AI QA Engineer
Tasks
- Design edge case and negative tests for AI failure modes
- Design test cases for LLM agents and RAG pipelines
- Detect hallucinations reasoning gaps source fabrication misattributions
- Maintain structured QA sign off in JIRA
- Perform regression testing after model upgrades prompt changes and backend fixes
- Run multi model comparative testing
- Test prompt versions iteratively and track accuracy changes
- Validate AI outputs against ground truth
- Validate citation accuracy document ingestion pipelines and cross document context
Perks/Benefits
Skills/Tech-stack
AWS Bedrock | Azure OpenAI | Case design | Content Filtering | Evaluation rubrics | Ground Truth | Ground truth validation | Hallucination detection | Jira | LLM-as-a-Judge | Language Models | Large Language Models | Machine Learning | Model Testing | Multi Model Testing | Multi-model | PII Redaction | Prompt engineering | Quality Assurance | Regression testing | Retrieval-Augmented Generation | SharePoint | Test Case | Test Case Design
Education
N/A
Roles
AI | AI QA | AI QA Engineer | Engineer | QA Engineer
Related jobs
-
Mid-level Full TimeIN-AP-Hyderabad7h ago
-
AI Engineer INR 1500K-3000KAzure Databricks | Data Engineering | Data engineering pipelines | Embeddings | Engineering PipelinesMid-level Full TimeBengaluru, Karnataka, India8h ago
-
Senior Engineer Digital & AI Solutions INR 4000K-6000KAPI Development | Agent systems | CI/CD | Cloud Computing | ContainerizationFood allowance | Health insurance | Hybrid work | Internet allowance | Learning and development programsSenior-level Full TimeHyderabad, Telengana, India9h ago
-
AI QA Engineer - Bangalore, IND INR 2000K-3000KAPI Development | Claude | Docker | Embeddings | Evaluation FrameworksMid-level Full TimeIndia10h ago
-
Lead Software Engineer - Java/Python, AWS,LLM INR 3500K-5000KA/B | A/B Testing | AWS Lambda | Agentic AI | AgileSenior-level Full TimeMumbai, Maharashtra, India10h ago
-
Staff Engineer, Data Analytics Engineering INR 2500K-3500KAPIs | AWS Glue | AWS Lambda | AWS S3 | Amazon BedrockSenior-level Full TimeBengaluru, KA, India10h ago
-
Senior Associate - Data Science/Applied AI ML INR 1400K-2400KAnomaly Detection | Backtesting | Bias/fairness | CI/CD | ClassificationMid-level Full TimeMumbai, Maharashtra, India12h ago
-
Senior Associate - Data Science/Applied AI ML INR 1400K-2400KAnomaly Detection | Bias and fairness | CI/CD | Classification | ClusteringMid-level Full TimeMumbai, Maharashtra, India12h ago
-
Senior-level Full Timetelengana, India13h ago
-
Consultant - Python Developer with Gen AI INR 1500K-2500KAI Search | Agile | Azure OpenAI | CSS | DjangoMid-level Full TimeBangalore, Karnataka, India13h ago
-
Mid-level Full TimeBangalore, Karnataka, India13h ago
-
Mid-level Full TimeBengaluru, Karnataka, India14h ago
-
Senior Machine Learning Engineer - Advertising Engineering INR 2520K-3880KAttribution Modeling | C++ | Classification | Data Lake | Data PipelinesFlexible remote work on Fridays | Health and wellness support | Hybrid work schedule | Paid time offSenior-level Full TimeBengaluru, India15h ago
-
Technical specialist- Data Engineer, AS INR 2475K-3465KAgile | Airflow | Apache Beam | Apache Hadoop | Apache SparkAccident insurance | Best in class leave policy | Certification sponsorship | Childcare assistance reimbursement | Employee assistance programMid-level Full TimePune - Margarpatta, India20h ago
-
Technical specialist- Data Engineer, AS INR 2475K-3465KAgile | Airflow | Apache Beam | Apache Spark | Cause analysisAccident and term life insurance | Childcare assistance reimbursement | Employee assistance program | Flexible leave policy | Gender neutral parental leavesMid-level Full TimePune - Margarpatta, India20h ago
-
Automation Excellence Senior Specialist INR 3700K-5572K.NET | AI Document Intelligence | AI Services | API | Access ManagementSenior-level Full TimeINBLR02 - Bangalore - Milesstone Buildcon, …20h ago
-
Data Engineer INR 3000K-4000KAgile | Azure Data | Azure Data Factory | Bash | CollibraFlexible working options | Health and wellbeing services | Hybrid work | Workplace inclusionSenior-level Full TimeBengaluru, IN20h ago
-
.NET Web | .NET Web API | AI Safety | AJAX | API DevelopmentGlobal team collaboration | Minimal office visits | Remote-first cultureEntry-level Full TimeIndia - Remote R20h ago
-
Gen AI Engineer (Data science) INR 2000K-4500KAWS | Angular | Apache Spark | Azure | Computer VisionFully remote | Minimal travelSenior-level Full TimeBangalore, KA, IN, 560100 R20h ago
-
Agile | Continuous Testing | JMeter | Jira | PostmanMid-level Full TimeBengaluru Millenia, India20h ago
-
Acceptance Testing | Agile | Automated testing | Automation Testing | Change ManagementFlexibility programs | Inclusive benefits | Mentorship | Wellbeing supportMid-level Full TimeBengaluru Millenia, India20h ago
-
Automated Test | Automated test cases | Automation Testing | Change Management | Continuous TestingFlexibility programs | Inclusive benefits | MentorshipMid-level Full TimeBengaluru Millenia, India20h ago
-
Acceptance Testing | Agile | Automation Testing | Continuous Testing | Database ManagementMid-level Full TimeBengaluru Millenia, India20h ago
-
Acceptance Testing | Automation Testing | Continuous Process | Continuous Process Improvement | Continuous TestingSenior-level Full TimeBengaluru Millenia, India20h ago
-
IN_Director_Gen AI_CEDA Central_Advisory_Bangalore INR 3200K-5199KAWS | Azure | Business Intelligence | Cloud Computing | Data analyticsFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportExecutive-level Full TimeBengaluru Millenia, India20h ago