QE for AI Engineer
Pune, Maharashtra, India
INR 3000K-4000K (estimate) Senior-level Full Time
Tasks
- Automate prompt regression testing
- Automate response comparison and evaluation
- Build offline online evaluation pipelines
- Create ground truth datasets and benchmarks
- Define evaluation frameworks for LLMs
- Design test strategies for non deterministic systems
- Ensure data lineage and traceability of outputs
- Evaluate LLM systems in production
- Evaluate chunking embedding and retrieval relevance
- Harden AI systems for reliability
- Integrate AI validation into CI CD pipelines
- Measure precision recall relevance and factuality
- Simulate adversarial and edge case inputs
- Test retrieval reranking and hallucination detection
- Validate RAG end to end pipelines
- Validate explainability and traceability
- Validate multi agent orchestration flows
Perks/Benefits
- N/A
Skills/Tech-stack
Adversarial Testing | Benchmarking | CI/CD | Data Lineage | Evaluation | Generative AI | Ground Truth | Language Models | Large Language Models | Machine Learning | Prompt engineering | RAG | Regression testing | Retrieval-Augmented Generation | Test automation | Test strategy
Education
N/A
Roles
AI | AI Engineer | AI Quality Engineer | Engineer | Quality Engineer
Related jobs
-
Sr.DataOPS Developer INR 2000K-3000KAWS Greengrass | Airflow | Apache Flink | Apache Kafka | Apache Kafka StreamsSenior-level Full TimeBangalore, India2h ago
-
Data Scientist INR 3000K-5000KAWS | Airflow | Amazon SageMaker | Apache Spark | Azure Machine LearningSenior-level Full TimeIN-MH-Pune4h ago
-
Net Core Azure Cosmos DB Gen AI INR 1800K-3300K.NET | API Management | ASP.Net Core | App Service | Application InsightsMid-level Full TimeIN-KA-Bangalore4h ago
-
Senior-level Full TimeIN-AP-Hyderabad4h ago
-
Senior Developer INR 2000K-5000KA/B | A/B Testing | API Development | Asynchronous programming | B testingSenior-level Full TimeIN-MH-Pune4h ago
-
Senior Applied AI/ML Engineer INR 3500K-5000KAgentic Systems | Autonomous Agents | Bias Mitigation | Data Analysis | Deep learningSenior-level Full TimeBengaluru, Karnataka, India5h ago
-
Forward Deployed Engineer, Google Cloud Business Platform INR 2500K-5000KArtificial Intelligence | Content Security | Content Security Policy | Data Storage | Distributed ComputingSenior-level Full TimeBengaluru, Karnataka, India5h ago
-
DE&A - Core - Advanced Data Engineering - Ab Initio INR 3000K-4200KAb Initio | Checkpointing | Data Lineage | Data Manipulation | Data Manipulation LanguageSenior-level Full TimeIndia5h ago
-
Assistant Manager- Python Gen AI with GCP- Bangalore INR 1500K-2500KADK | AI Agents | API Development | Agile | Cloud platformWork from client office 5 days per weekMid-level Full TimeBangalore, Karnataka, India5h ago
-
Senior-level Full TimeBangalore, Karnataka, India6h ago
-
Data Engineer (.NET Specialist) INR 2400K-3380K.Net Core | ASP.NET | Angular | App Service | Azure AppCustomer-facing role | On call support rewardsSenior-level Full TimePune, MH, India6h ago
-
Business Intelligence Engineer - Snowflake Developer INR 2500K-4132KAgile | Airflow | Bitbucket | CI/CD | Common table expressionsFlexible remote days | Health insurance | Hybrid work model | Life insurance | Paid time offSenior-level Full TimeHyderabad, India R7h ago
-
GenAI Engineer INR 2500K-5000KAPI Integration | Agent Orchestration | Anthropic Claude | Anthropic Claude Code | CRMInclusive workplace | Knowledge sharing | Mentoring | Professional developmentSenior-level Full TimePune, Maharashtra, India7h ago
-
Mid-level Full TimeMumbai, Maharashtra, India8h ago
-
Staff Engineer INR 4000K-5500KAgentic Workflows | Artificial Intelligence | BigQuery | Cloud platform | CrewAISenior-level Full TimeChennai, Tamil Nadu, India10h ago
-
Compliance Engineering - Data Eng - Analyst - Bengaluru INR 1685K-2700KAI code generation | API | Automated testing | Code generation | DatabaseMid-level Full TimeBengaluru, Karnataka, India10h ago
-
Senior Manager of Software Engineering - Data Engineer INR 4000K-6000KAPI Development | AWS | AWS Glue | AWS Lambda | Apache KafkaSenior-level Full TimeHyderabad, Telangana, India10h ago
-
Lead - AI Engineer INR 2000K-5000KAgentic AI | Generative AI | Governance | Language Models | Large Language ModelsSenior-level Full TimeBangalore, Bangalore, IN10h ago
-
Asst Mgr - AI Solutions INR 96K-180KAPIs | Artificial Intelligence | Feature Engineering | Generative AI | GitEntry-level Full TimeMumbai A.O, MUM-KND-AFS(FD), IN10h ago
-
Software Engineer, Evals INR 1500K-3000KBackend Engineering | C++ | Cloud Native | Dashboards | Data PipelinesMid-level Full TimeBangalore, India10h ago
-
Senior-level Full TimeBangalore, India10h ago
-
IN_RBIN_Assistant Manager/Senior Engineer_Data Scientist INR 3000K-4200KAgentic AI | Agile | Application development | Azure Databricks | Big DataSenior-level Full TimeNashik, Maharashtra, India10h ago
-
Principal Software Engineer INR 2500K-5000K.Net Core | .Net Framework | AI Agents | AI code analysis | AI code generationSenior-level Full TimeBengaluru, India11h ago
-
Data Preprocessing | Deep learning | Feature Engineering | Git | Jupyter NotebooksFlexible hours | Part-time schedule | Professional development opportunity | Remote workMid-level Full TimeIndia R12h ago
-
Python With Gen AI INR 1500K-3000KData Analysis | Data Preprocessing | Deep learning | Generative AI | Machine LearningCareer growth programs | Flexible work hours | Remote work | Work-life balanceMid-level Full TimeBangalore, IN17h ago