Senior AI Engineer - LLM Evaluation
Tasks
- Accelerate delivery with AI coding assistants while maintaining testing and security
- Build RAG pipelines
- Build SDKs and orchestration patterns
- Build automated validation pipelines
- Define AI quality metrics tracking
- Deploy and operate LLMs in production
- Design evaluation frameworks
- Develop reusable AI platform capabilities
- Ensure responsible AI alignment
- Evaluate and adopt AI testing frameworks
- Fine tune and benchmark large language models
- Implement AI observability practices
- Integrate AI agents using Model Context Protocol
- Integrate evaluation into CI CD workflows
- Lead AI evaluation and quality strategy
Perks/Benefits
- N/A
Skills/Tech-stack
AI Governance | AWS | Agentic Workflows | CI/CD | Cloud Native | Cloud platform | Contextual retrieval | Continuous Deployment | Continuous integration | Evaluation automation | Feedback loops | Generative AI | Google Cloud | Google Cloud Platform | Knowledge graphs | LLM safety | Language Models | Large Language Models | Machine Learning | Microsoft Azure | Model Benchmarking | Monitoring | Observability | Prompt engineering | PyTorch | Python | RAG Pipelines | Regression testing | Responsible AI | Retrieval-Augmented Generation | Semantic Retrieval | Semantic Search | Tracing | Transformers | Vector Search
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Senior-level Full TimeIndia4h ago
-
Agile | Alerting | Automation | CI/CD | ClusteringAgile environment | Professional growthSenior-level Full TimeIndia5h ago
-
Databricks and Splunk Engineer - Senior Associate INR 3264K-4500KAmazon Web Services | Analytics | Azure | Cloud Computing | Cyber ThreatEmployee networks | Flexible work/life support | Hybrid work schedule | Inclusive development opportunities | Paid volunteer daysSenior-level Full TimeBangalore, India R8h ago
-
Senior Software Engineer INR 2771K-4064KAngular | Apache Hadoop | Apache Hive | Apache Spark | DockerSenior-level Full TimeBangalore, India10h ago
-
Senior-level Full TimeBangalore North, India16h ago
-
Senior-level Full TimeBangalore North, India16h ago
-
Senior-level Full TimeBangalore North, India16h ago
-
Data Engineer (ETL ODI) - AMS INR 1500K-2500KAPIs | Data Governance | Data Integrator | Data Modeling | Data QualityMid-level Full TimeBangalore North, India16h ago
-
Software Engineer INR 1200K-2500KAI Evaluation | Agile | Artifactory | Automated testing | Azure DevOpsEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India17h ago
-
Mid-level Full TimeGurugram - Haryana - India Pune …17h ago
-
AI Architect INR 2500K-5000KAccess Management | Agentic AI | Auditability | Compliance | Data ArchitectureSenior-level Full TimeBangalore - Karnataka - India Pune …17h ago
-
A1 / Analyst-GTS INR 360K-420KCNN | Data Preprocessing | Decision Tree | Deep learning | Dimensionality ReductionMentoring | Work reviewedEntry-level Full TimeHyderabad, Telangana, India18h ago
-
Advanced Engineer - AI/ML with Devops(GCP,AWS) INR 2500K-4200KAWS | Airflow | Amazon Web Services | Apache Spark | Artificial IntelligenceAccess to cutting-edge technologies | Collaborative work environment | Professional development | Training opportunitiesSenior-level Full TimeBangalore - Karnataka, India - Kalyani …18h ago
-
Sr AI Solutions Engineer (Generative AI & Agentic AI) INR 3242K-4062KAI Agent Builder | Agent Builder | Agent Orchestration | BigQuery | Claude APISenior-level Full TimePune, MH, India19h ago
-
Data Engineer INR 3000K-4000KAWS | Amazon Aurora | Automation | Azure | Data ManagementBike to work | Career development program | Educational assistance | Employee assistance program | Employee discountSenior-level Full TimeBengaluru, KA, India R19h ago
-
AI Development Strategy Lead INR 2500K-5000KAgent Orchestration | BI frameworks | Business Intelligence | C++ | Cloud DataSenior-level Full TimeBengaluru, Karnataka, India20h ago
-
Data and AI Engineer, Google Cloud INR 2000K-3300KAlgorithms | Architectural frameworks | Batch Processing | Cloud platform | Data PipelinesSenior-level Full TimeBengaluru, Karnataka, India; Hyderabad, Telangana, India20h ago
-
Software Engineer INR 1200K-2500KAgentic AI | Agile | Artifactory | Azure DevOps | Backend DevelopmentEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India20h ago
-
Senior-level Full TimeGurugram, Haryana, India20h ago
-
Mid-level Full TimeBengaluru, Karnataka, India20h ago
-
Principal Software Engineer INR 2475K-4800KAPI Design | Agent Orchestration | Agentic Systems | Automation | By DesignAgile environment | Feature flags | High autonomy | Inclusive culture | Mentorship and coachingSenior-level Full TimeIndia, Telangana, Hyderabad21h ago
-
FDH Support Leader INR 2500K-3500KAlerting | CI/CD | Data Contracts | Data Lineage | Data ObservabilitySenior-level Full TimePune, Maharashtra, IND, 41101321h ago
-
Associate Engineer INR 1500K-2500KAnalysis Services | Analysis Services Tabular | Azure Analysis | Azure Analysis Services | Azure Analysis Services TabularMid-level Full TimeHyderabad, IN21h ago
-
Robotics Engineer Fresher INR 300K-400KArduino | Artificial Intelligence | Automation | C plus plus | ElectronicsCareer growth opportunities | Exposure to robotics competitions | Hands on experience with robotics and AI | Opportunity to mentor students globallyEntry-level Full TimeBangalore, Karnataka, India21h ago
-
Senior Data Engineer- Managed services INR 3000K-4000KAzure Cloud | Azure Cloud Adoption | Azure Cloud adoption framework | Azure Data | Azure Data FactoryRemote work opportunitySenior-level Full TimePune, MH, India21h ago