Senior AI Engineer - LLM Evaluation
APAC - India - Bengaluru - Sunriver
INR 3740K-5876K (estimate) Senior-level Full Time
Tasks
- Build automated evaluation pipelines for continuous validation
- Build scalable RAG pipelines using enterprise data
- Contribute reusable AI platform capabilities and SDKs
- Define and track AI quality metrics
- Design scalable evaluation frameworks for LLMs and ML systems
- Ensure AI alignment with Responsible AI and governance
- Evaluate and adopt AI testing frameworks and benchmarking methods
- Fine tune evaluate benchmark and deploy large language models
- Implement evaluation and observability standards
- Integrate AI agents with enterprise services and platforms
- Lead AI evaluation and quality strategy
Perks/Benefits
- N/A
Skills/Tech-stack
AI Governance | AI Observability | AWS | Azure | Benchmarking | Bias Assessment | CI/CD | Claude SDK | Claude SDKs | CrewAI | Deep learning | DeepEval | Google Cloud | LLM Evaluation | Langgraph | Langsmith | Language Models | Large Language Models | Lightning | MCP | Machine Learning | Model Risk Management | Model risk | Monitoring | OpenAI SDKs | OpenAI evals | Promptfoo | PyTorch | PyTorch Lightning | Python | RAG | Ray | Regression testing | Responsible AI | Risk Management | SDK | Semantic Search | Tracing | Transformers | TruLens | Vector Search
Education
Roles
Related jobs
-
Junior Data Scientist- Computer Vision INR 350K-360KComputer Vision | Convolutional Neural Networks | Deep learning | Machine Learning | MatplotlibEntry-level Full TimeBengaluru, India2h ago
-
Data Engineer INR 1500K-2500KAWS Cloud | AWS Cloud Development Kit | AWS Glue | AWS IAM | AWS LambdaMid-level Full TimeIN-TN-Chennai3h ago
-
Gen Ai INR 2000K-3000KAI Agents | AI Governance | AWS CodePipeline | Agent architecture | AuthenticationMid-level Full TimeIN-KA-Bangalore3h ago
-
Senior-level Full TimeIN-KA-Bangalore3h ago
-
Agentic AI Engineer INR 1800K-2500KAI orchestration | API Integration | Automated testing | Automation | Azure DevOpsMid-level Full TimeIndia, West Bengal, Kolkata3h ago
-
Senior-level Full TimeGurugram, India4h ago
-
Data Engineer INR 1500K-3000KData Engineering | Governance | Information Systems | Management Information Systems | Management informationSenior-level Full TimeGurugram, Haryana, India5h ago
-
Senior QA Engineer(Data Platform Testing) INR 2515K-2829KAPI Gateway | API Testing | AWS Lambda | Alerting | CI/CDReward and recognition | Volunteering opportunities | Wellness | Work-life balanceSenior-level Full TimeHyderabad, India5h ago
-
Applied AI ML Associate INR 3799K-5000KAWS | Artificial Intelligence | Cloud platform | Data Analysis | Data PipelinesEntry-level Full TimeBengaluru, Karnataka, India6h ago
-
Lead Software Engineer INR 4000K-5000KAWS | Agent Development | Agent systems | Amazon ECS | Amazon EKSSenior-level Full TimeBengaluru, Karnataka, India6h ago
-
API Testing | Allure | BDD | CI/CD | CucumberExecutive-level Full TimePune, India6h ago
-
GCP Data Engineer INR 1500K-2500KAPI Integration | BigQuery | Cloud platform | Data Governance | Data ModelingSenior-level Full TimeGurugram, Haryana, India7h ago
-
Senior Consultant_AWS Developer INR 1500K-3500KAWS Glue | AWS Lambda | AWS Step Functions | AWS Storage Gateway | AWS storageSenior-level Full TimeThane, IN, 40060410h ago
-
Applied AI ML Lead INR 3264K-5000KAI Libraries | API Development | AWS | Agentic AI | Agentic AI librariesSenior-level Full TimeBengaluru, Karnataka, India10h ago
-
Mid-level Full TimeBengaluru, Karnataka, India11h ago
-
APIs | CI/CD | Cloud Computing | Distributed Systems | JavaCoworking stipend | Flexible savings accounts | Health coverage | Remote work | Sick leaveSenior-level Full TimeBengaluru, Karnataka, India12h ago
-
Senior Data Systems Software Engineer INR 1500K-3000KA/B | A/B Testing | API Versioning | API interoperability | AutomationLife insurance | Medical insurance | Retirement options | Volunteer programsSenior-level Full TimeBENGALURU, KARNATAKA, India13h ago
-
Mid-level Full TimePune, MH, IN, 41101413h ago
-
AI & Automation Engineer INR 3264K-5876KAPIs | Data Pipelines | Data Warehousing | Data models | EvaluationCareer development | Flexible vacation | Headspace app access | Hybrid work model | Mental health daysSenior-level Full TimeIndia, Bengaluru, Karnataka R15h ago
-
Lead Data Engineer/ Data Architect INR 2000K-2800KAgile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeHyderabad, Telangana, India - Remote R15h ago
-
Mid-level Full TimeBangalore, KA, IN15h ago
-
IN_Senior Manager_ AWS Bedrock Engineer_D&A_Advisory_Mumbai INR 1960K-4000KAPI Gateway | APIs | AWS Bedrock | AWS CloudFormation | AWS LambdaSenior-level Full TimeMumbai Shivaji Park, India15h ago
-
Agile | Angular | Bash | CI/CD | Continuous integrationSenior-level Full TimeMumbai Goregaon, India15h ago
-
Senior Engineer I Embedded Software Testing INR 3000K-3500KCase design | Embedded Systems | Firmware Testing | HVAC&R | Regression testingFlexible time off | Holiday leave | Paid parental leave | Vacation leaveSenior-level Full TimePune, India15h ago
-
IN_Senior Associate_GenAI and Agentic AI_GCC_Advisory_Bangalore INR 2500K-5000KAPI Development | Asynchronous programming | Autogen | CI/CD | CrewAISenior-level Full TimeBengaluru Millenia, India15h ago