AI QA Trainer - LLM Evaluation

India R

A INR 2300K-4200K (estimate) Senior-level Contract Full Time

@ J...

Apply Save

Found 22h ago

Tasks

Analyze model failures
Assess hallucination detection
Build test automation with Python and SQL
Collaborate on evaluation metrics dashboards and monitoring
Conduct robustness evaluations
Create regression suites
Define pass fail criteria
Design evaluation frameworks
Develop evaluation rubrics
Document findings and root cause analysis
Evaluate large language models
Execute test plans
Maintain evaluation documentation
Perform adversarial testing
Recommend prompt improvements and guardrails
Run red teaming exercises
Validate tool usage accuracy
Verify grounding and retrieval augmented generation outputs

Perks/Benefits

Skills/Tech-stack

Education

Bachelor of Engineering | Bachelor of Science | Master of Science | PhD

Roles

Regions

Asia/Pacific

Countries

India

Apply Save

Language: en Views: 0

Clicks: 0

Saves: 0

Related jobs

GTM & AI Engineer A INR 2000K-3465K

API Design | Anthropic | Automation | CRM Integrations | CrewAI

Distributed work flexibility | Fully remote | High ownership role

Mid-level Full Time

India R

18h ago
Senior Machine Learning Engineer / Tech Lead - AI & ML A INR 2040K-4200K

Cloud Computing | Containerization | Docker | GPU | Kubernetes

Distributed team | Four-day work week | Fully remote | Open source contribution opportunities | Unlimited paid time off

Senior-level Full Time

India R

22h ago
AI Engineer - FDE (Forward Deployed Engineer) INR 3242K-4500K

AWS | Agent systems | Apache Spark | Azure | Dspy

Remote work

Senior-level Full Time

Remote - India R

1d ago
Data Scientist INR 2500K-4500K

Azure | Computer Vision | Deep learning | Embeddings | Fine Tuning

Senior-level Full Time

Pune, India R

1d ago
Principal Machine Learning Engineer INR 5000K-6000K

Benchmarking | Cost Optimization | Deep learning | Keras | Language Processing

Senior-level Full Time

Bengaluru - India - Bengaluru, 560071 … R

1d ago
Senior BI & AI Engineer INR 3000K-4064K

Anomaly Detection | Azure Data | Azure Data Factory | Azure DevOps | DAX

ESG opportunities | Flexible vacation | Headspace app | Hybrid work model | Mental health days

Senior-level Full Time

India, Hyderabad, Telangana R

1d ago
Senior Data Analyst INR 3000K-4000K

Apache Airflow | BI | CI/CD | DAX | DBT

Employee incentive programs | Flex my way policies | Flexible vacation | Headspace access | Hybrid work model

Senior-level Full Time

India, Bengaluru, Karnataka R

1d ago
Data Engineer Azure Fabric INR 1500K-2200K

Active Directory | Azure | Azure Active Directory | Azure Data | Azure Data Factory

Eastern US time overlap | Proactive monitoring and alerting support | Remote work

Mid-level Contract

India - Remote R

1d ago
Principal Applied AI Engineer USD 185K-245K

Agent Frameworks | Agent SDK | Anthropic Agent SDK | Context engineering | Evals

Senior-level Full Time

Redesign Health R

1d ago
Lead Engineer, Testing INR 1500K-2500K

Case management | Data Pipeline Testing | Data Quality | Data Reconciliation | Data Validation

Senior-level Full Time

Remote Working, India R

1d ago
Senior Data Engineer INR 3600K-5000K

AWS | Data Modeling | Data Quality | Data Security | Data integration

Flexible work schedule | Hybrid work | Professional development | Remote work

Senior-level Full Time

Delhi NCR R

1d ago
Data Platform Engineer INR 1500K-3000K

AWS IAM | AWS KMS | AWS S3 | AWS Secrets | AWS Secrets Manager

Senior-level Full Time

Remote - India R

1d ago
Java, GCP, ReactJS/Consultant Specialist INR 2200K-3200K

Ansible | Apache Airflow | Apache Atlas | BigQuery | Claude

Flexible working | Professional development

Mid-level Full Time

Hyderabad, Telangana, India R

1d ago
AI Solutions Architect A INR 2000K-4680K

AI Governance | AI Security | AWS | Agent systems | Azure

Contract extension opportunity | Fully remote | Remote contract | US time zone overlap

Senior-level Contract Full Time

India R

1d ago
Java, GCP, ReactJS/Consultant Specialist INR 2200K-3200K

Ansible | Apache Airflow | Apache Atlas | BigQuery | Claude

Flexible working | Professional development

Mid-level Full Time

Hyderabad, Telangana, India R

1d ago
Machine Learning Engineer - LLM & GenAI A INR 2475K-5000K

Benchmarking | Chroma | CrateDB | Embedding Optimization | Faiss

Continuous learning | Flexible work hours | Fully remote

Senior-level Full Time

India R

1d ago
Senior Data Engineer A INR 2000K-3465K

Access Control | Agile | Analysis Services | App Service | Auditing

Collaboration opportunities | Continuous learning | Flexible home-office arrangements | Knowledge sharing | Professional growth

Senior-level Full Time

India R

1d ago
QA Engineer INR 940K-1600K

Data Quality | Data Reconciliation | Data Validation | Data Warehousing | ETL Testing

Career development | Flexible work hours | Inclusive work culture | Insurance | Learning opportunities

Mid-level Full Time

Pune, India R

1d ago
AP - Senior AI Engineer A INR 1860K-4680K

AI Agents | API | API Development | AWS | Autogen

Continuous learning | Exposure to cutting-edge AI technologies | Flexible location | Fully remote | International engineering team collaboration

Senior-level Full Time

India R

1d ago
Go Developer - AI & ML A INR 2040K-3065K

API Design | Channels | Distributed Systems | Docker | Go

Flexible schedule | Four-day work week | Fully remote | Professional growth opportunities | Uncapped holiday allowance

Senior-level Full Time

India R

2d ago
Principal AI Solutions Architect INR 3000K-5000K

API Design | Agent Frameworks | Artificial Intelligence | CI/CD | Cloud Architecture

Senior-level Full Time

India, Chennai (Hybrid) R

2d ago
AI Applications Engineer INR 3242K-5000K

Agent runtime | CI/CD | Data Pipelines | Docker | Google Cloud

Senior-level Full Time

India, Bengaluru (Hybrid) R

2d ago
Machine Learning Associate Advisor - HIH - Evernorth INR 2500K-3000K

API Integration | Artificial Intelligence | Authentication | C# | C++

Flexible hours | Remote work flexibility

Mid-level Full Time

HIH - Hyderabad, India R

2d ago
Digital Transformation & AI Scientist INR 4000K-6000K

Computational Fluid Dynamics | Data Ingestion | Digital Twin | Element analysis | Finite Element Analysis

Senior-level Full Time

India - Pune R

2d ago
Data Engineer (B) INR 1500K-2500K

AI machine learning | AWS | Azure | Azure Data | Azure Data Factory

Career growth | Collaborative work environment | Flexible hours | Professional development | Remote work

Senior-level Full Time

Pune, Maharashtra, India - Remote R

2d ago