Manager, LLM Architecture

It's fun to work in a company where people truly BELIEVE in what they're doing!

We're committed to bringing passion and customer focus to the business.

Job Description

Lead LLM Architect

This role requires working from our local Hyderabad office 2-3x a week.

Location: Hyderabad, Telangana, India

ABOUT THE TEAM

The Generative AI Platform team owns the company-wide stack for Large Language Model integration, including retrieval-augmented generation (RAG), prompt orchestration, vector-database indexing, and evaluation tooling. We partner with product, design, and domain experts to convert research breakthroughs into secure, scalable, and delightful customer experiences. As our Lead LLM Architect you will set the technical vision, design the architecture, and guide engineers building reusable AI services across web, mobile, and backend products.

At ABC, we love entrepreneurs because we are entrepreneurs. We know how much grit it takes to start your own business and grow it into something that lasts. We roll our sleeves up, we act fast, and we learn together. You will be working closely with talented software engineers and senior engineering leaders to optimize and build upon our single integration platform strategy.

WHAT YOU’LL DO:

• Partner with engineering, product, and design leaders to define a clear Gen-AI vision, strategy, and roadmap for ABC’s platform.
• Empower your team to deliver AI-powered solutions in a simple, reliable, and sustainable way.
• Develop and support engineers in their personal growth and career progression; act as a mentor and coach.
• Contribute to engineering-wide initiatives that make ABC an exceptional place to work for ambitious, highly motivated engineers.
• Gain deep product understanding of our fitness-tech domain and share that knowledge across the team.
• Teach, mentor, and coach other engineers on Gen-AI best practices, secure coding, and architectural excellence.
• Facilitate daily stand-ups and planning; remove blockers, ensure goals are met, and embed lessons learned from retrospectives and experiments.
• Provide hands-on leadership across our technical stack (Java, .NET, Golang, React/React Native) while introducing LLM components (OpenAI/Azure OpenAI, LangChain, vector DBs).
• Leverage cloud expertise (AWS or Azure) to deploy scalable, cost-efficient AI micro-services on Kubernetes with IaC (Terraform/Bicep).
• Architect and oversee API-orchestration layers that expose LLM capabilities as secure, versioned services consumed by web and mobile products.
• Embed Responsible-AI guardrails—bias checks, audit logging, safety filters—and champion enterprise-grade security, privacy, and compliance.
• Oversee best-practice adoption for software engineering, DevSecOps, and quality; ensure industry standards and legal requirements are built into every release.
• Communicate clearly with stakeholders and senior management—articulating platform strategy, trade-offs, and technical risks in plain English.

WHAT YOU’LL NEED:

• Bachelor’s degree in Computer Science (or related field) preferred; Master’s or PhD in AI / ML is a plus.
• 8–10 + years of hands-on software-engineering experience, including 3 + years in technical-lead or people-management roles driving architecture decisions.
•  Deep expertise with Gen-AI stacks: OpenAI / Azure OpenAI, LangChain / LlamaIndex, embedding workflows, and vector databases (Pinecone, Weaviate, Milvus, pgvector).
• Cloud mastery on AWS and/or Azure—including Kubernetes, GPU/TPU provisioning, Infrastructure-as-Code (Terraform/Bicep), and cost-governance.
• Strong coding skills in Node.js and/or Python, plus ability to dive into Java, .NET, React / React Native when needed (“roll-up-your-sleeves” mindset).
• Proven track record designing and operating large-scale, low-latency AI micro-services used by thousands of customers or millions of requests.
• Solid grounding in software-architecture patterns, system design, and DevSecOps; able to embed Responsible-AI guardrails (bias checks, safety filters, audit logging).
• Exceptional written and verbal communication, comfortable translating complex AI concepts for both technical and non-technical stakeholders.
• Demonstrated leadership and mentoring skills—cultivating curiosity, psychological safety, and an innovation mindset within diverse teams.
• Strategic-planning & project-management prowess: OKRs, Agile/Scrum, roadmap prioritisation, and cross-functional delivery in fast-moving environments.
• “Experience partnering with OpenAI or Azure OpenAI product teams, or contributing to OSS Gen-AI frameworks, is a strong plus.”

WHAT’S IN IT FOR YOU:  

• Purpose led company with a Values focused culture – Best Life, One Team, Growth Mindset 

• Time Off – competitive PTO plans with 15 Earned accrued leave, 12 days Sick leave, and 12 days Casual leave per year 

• 11 Holidays plus 4 Days of Disconnect – once a quarter, we take a collective breather and enjoy a day off together around the globe. #oneteam 

• Group Mediclaim insurance coverage of INR 500,000 for employee + spouse, 2 kids, and parents or parent-in-laws, and including EAP counseling 

• Life Insurance and Personal Accident Insurance 

• Best Life Perk – we are committed to meeting you wherever you are in your fitness journey with a quarterly reimbursement 

• Premium Calm App – enjoy tranquility with a Calm App subscription for you and up to 4 dependents over the age of 16 

• Support for working women with financial aid towards crèche facility, ensuring a safe and nurturing environment for their little ones while they focus on their careers. 

We’re committed to diversity and passion, and encourage you to apply, even if you don’t demonstrate all the listed skillsets!

ABC’S COMMITMENT TO DIVERSITY, EQUALITY, BELONGING AND INCLUSION:
ABC is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.  We are intentional about creating an environment where employees, our clients and other stakeholders feel valued and inspired to reach their full potential and make authentic connections. We foster a workplace culture that embraces each person’s diversity, including the extent to which they are similar or different. ABC leaders believe that an equitable and inclusive culture is not only the right thing to do, it is a business imperative. Read more about our commitment to diversity, equality, belonging and inclusion at abcfitness.com.

#LI-HYBRID

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!