Senior AI Engineer - LLM Evaluation
APAC - India - Bengaluru - Sunriver
INR 3740K-5876K (estimate) Senior-level Full Time
Tasks
- Build automated evaluation pipelines for continuous validation
- Build scalable RAG pipelines using enterprise data
- Contribute reusable AI platform capabilities and SDKs
- Define and track AI quality metrics
- Design scalable evaluation frameworks for LLMs and ML systems
- Ensure AI alignment with Responsible AI and governance
- Evaluate and adopt AI testing frameworks and benchmarking methods
- Fine tune evaluate benchmark and deploy large language models
- Implement evaluation and observability standards
- Integrate AI agents with enterprise services and platforms
- Lead AI evaluation and quality strategy
Perks/Benefits
- N/A
Skills/Tech-stack
AI Governance | AI Observability | AWS | Azure | Benchmarking | Bias Assessment | CI/CD | Claude SDK | Claude SDKs | CrewAI | Deep learning | DeepEval | Google Cloud | LLM Evaluation | Langgraph | Langsmith | Language Models | Large Language Models | Lightning | MCP | Machine Learning | Model Risk Management | Model risk | Monitoring | OpenAI SDKs | OpenAI evals | Promptfoo | PyTorch | PyTorch Lightning | Python | RAG | Ray | Regression testing | Responsible AI | Risk Management | SDK | Semantic Search | Tracing | Transformers | TruLens | Vector Search
Education
Roles
Related jobs
-
AI & Automation Engineer INR 3264K-5876KAPIs | Data Pipelines | Data Warehousing | Data models | EvaluationCareer development | Flexible vacation | Headspace app access | Hybrid work model | Mental health daysSenior-level Full TimeIndia, Bengaluru, Karnataka R13h ago
-
Lead Data Engineer/ Data Architect INR 2000K-2800KAgile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeHyderabad, Telangana, India - Remote R13h ago
-
Mid-level Full TimeBangalore, KA, IN13h ago
-
IN_Senior Manager_ AWS Bedrock Engineer_D&A_Advisory_Mumbai INR 1960K-4000KAPI Gateway | APIs | AWS Bedrock | AWS CloudFormation | AWS LambdaSenior-level Full TimeMumbai Shivaji Park, India13h ago
-
Agile | Angular | Bash | CI/CD | Continuous integrationSenior-level Full TimeMumbai Goregaon, India13h ago
-
Senior Engineer I Embedded Software Testing INR 3000K-3500KCase design | Embedded Systems | Firmware Testing | HVAC&R | Regression testingFlexible time off | Holiday leave | Paid parental leave | Vacation leaveSenior-level Full TimePune, India13h ago
-
IN_Senior Associate_GenAI and Agentic AI_GCC_Advisory_Bangalore INR 2500K-5000KAPI Development | Asynchronous programming | Autogen | CI/CD | CrewAISenior-level Full TimeBengaluru Millenia, India13h ago
-
IN_Senior Associate_GenAI and Agentic AI_GCC_Advisory_Bangalore INR 2500K-5000KAPI Development | Async Programming | Autogen | CI/CD | CrewAIFlexibility programs | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India13h ago
-
AI And A - Machine Learning Engineer - VG - W4 - CDAO0230 INR 3799K-5876KData Engineering | Data Mining | Data Pipelines | Data Quality | Data discoveryCareer development | Coaching and feedback | Discounts and offers | Flexible working | Health and well-being supportSenior-level Full TimeHyderabad, TG, IN, 500081 R13h ago
-
Entry-level Full TimeBangalore, IN13h ago
-
Agent systems | Amazon Web Services | Auditing | CI/CD | Cloud platformAccess to modern cloud infrastructure and engineering tooling | Collaborative high impact environment | Competitive compensation package | Exposure to advanced generative AI and MLOps practices | Flexible remote friendly working environmentSenior-level Full TimeIndia14h ago
-
Senior AI Engineer - Remote INR 3375K-5000KAWS CloudFormation | Apache Tomcat | CI/CD | CircleCI | Data PipelinesDistributed team collaboration | Remote workSenior-level Full TimeHyderabad, TS, India R17h ago
-
Senior AI Engineer - Remote INR 3375K-5000KApache | Apache Tomcat | CI/CD | CircleCI | CloudFormationRemote workSenior-level Full TimeGurugram, HR, India R17h ago
-
Senior AI Engineer - Remote INR 3375K-5000KAWS DocumentDB | Apache Tomcat | CI/CD | CircleCI | CloudFormationFlexible distributed team | Remote workSenior-level Full TimeBengaluru, KA, India R17h ago
-
Senior AI Engineer - Remote INR 3375K-5000KApache Avro | Apache Tomcat | CI/CD | CircleCI | CloudFormationGrowth mentoring | Mission-driven work | Remote workSenior-level Full TimeNew Delhi, DL, India R17h ago
-
Bash | Data Processing | Docker | GCP | LinuxAsynchronous culture | Competitive salary | Entrepreneurial team | Remote/distributed team | Supportive managementMid-level Full TimeBangalore, India22h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Friendly work environment | Opportunity for impactMid-level Full TimeDelhi, India22h ago
-
AI Engineer INR 3000K-5000KAPI Development | Agent Development | Agent Development Framework | Agent Development Kit | AnthropicMid-level Full TimePune, Maharashtra, India23h ago
-
Jr Data Engineer INR 360K-420KAWS Glue | Apache Airflow | Apache Flink | Apache Kafka | Apache SparkEntry-level Full TimeBangalore North, India1d ago
-
Senior-level Full TimeBangalore, IN1d ago
-
Data Engineer INR 2520K-3380KAWS | Airflow | Azure | Azure Bicep | Azure DataEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India1d ago
-
Senior Consultant INR 2000K-2700KAzure Data | Azure Data Lake | Azure Databricks | Azure Machine Learning | Data LakeSenior-level Full TimeIN-KA-Bangalore1d ago
-
Software Engineer – Data Platform INR 3000K-4000KAPIs | Alerting | Azure | Azure Data | Azure Data LakeFlexible working hours | Mobile working | Sabbatical | Training opportunitiesSenior-level Full TimeBengaluru, KA, India1d ago
-
Software Engineer – Data Platform INR 3000K-4000KAPIs | Alerting | Azure | Azure DevOps | CDCMobile flexible working models | Sabbaticals | Training opportunities | Work-life balanceSenior-level Full TimeBengaluru, KA, India1d ago
-
Software Engineer – Data Platform INR 3000K-4000KAPIs | Alerting | Audit Trail | Automated testing | AzureFlexible working | Mobile working | Sabbaticals | Training opportunities | Work-life balanceSenior-level Full TimeBengaluru, KA, India1d ago