Senior AI Quality Assurance (QA) Engineer
Tasks
- Build Python test automation
- Create benchmark golden datasets
- Evaluate LLM powered applications
- Implement unit testing integration testing and API testing
- Perform root cause analysis for AI failures
- Run AI regression tests
- Test REST APIs and distributed backend services
- Test conversational AI systems
- Use AI evaluation and observability platforms
- Validate AI evaluation metrics
- Validate SQL generated queries and results
Perks/Benefits
- N/A
Skills/Tech-stack
AI benchmarking | API Testing | Cause analysis | Cost analysis | Exact Match | Execution Accuracy | F1-score | Fixtures | Integration Testing | LLM Evaluation | Language Processing | Latency P50 | Latency P95 | Latency P99 | Mean Reciprocal Rank | Mocking | Natural Language | Natural Language Processing | Observability | Parameterized Testing | Precisión | Pytest | Python | REST APIs | Recall | Regression testing | Root Cause Analysis | Root cause | SQL | Token usage | Unit Testing
Education
N/A
Related jobs
-
Agentic AI Data Engineer INR 2500K-5000KAWS Glue | AWS Lambda | AWS Step Functions | Amazon Athena | Amazon BedrockSenior-level Full TimeGurugram, Haryana, India3h ago
-
Data Scientist INR 2500K-4200KAmazon SageMaker | Azure AI | Azure Synapse | Azure Synapse Analytics | C++Mid-level Full TimeHyderabad, Telangana, India7h ago
-
Data Engineer INR 1500K-2980KAPI Security | AWS | Angular | CSS3 | CypressEmployee assistance program | Employee discounts | Gym reimbursement | Healthcare coverage | Learning platform accessMid-level Full TimeBangalore, Karnataka, India7h ago
-
Software Engineer - Tableau INR 2520K-4000KA/B | A/B Testing | Agile | Attribution Modeling | Audience SegmentationFlexible work modelSenior-level Full TimeHyderabad, India8h ago
-
Senior-level Full TimePune, MH, India9h ago
-
Data Engineer INR 2771K-4064KAgile | Azure Data | Azure Data Services | Azure DevOps | Cloud ConceptsLearning and career development | Management training | Modern infrastructure | Online learningSenior-level Full TimeChennai, IN, 600 03213h ago
-
AI Engineering INR 3242K-5000KAgent workflows | Audit Logging | Authentication | Authorization | Automated testingSenior-level Full TimeBangalore, KA, IN13h ago
-
Kafka Administrator INR 2800K-4200KAWS EC2 | AWS Glue | AWS IAM | AWS MSK | AWS SecretsBudget for courses and technical conferences | Career progression | Funded certification paths | Hybrid/Remote flexibility | MentorshipSenior-level Full TimeBengaluru, Karnataka, India18h ago
-
Lead MySQL Database Engineer INR 2000K-3100KAWS EC2 | AWS RDS | Amazon Aurora | Ansible | AppDynamicsIn-office work culture | Mentoring and developmentSenior-level Full TimeBengaluru, Karnataka, India18h ago
-
Senior-level Full TimeHyderabad, India23h ago
-
Senior Software Engineer INR 2800K-4200KAWS | Bash | CI/CD | CICS | CobolReward and recognition | Volunteering opportunities | Wellness | Work-life balanceSenior-level Full TimeHyderabad, India23h ago
-
Software Engineer III - Data Engineer INR 2500K-4000KAgile | Application Resiliency | Artificial Intelligence | Automated testing | CI/CDSenior-level Full TimeHyderabad, Telangana, India23h ago
-
Senior manager- Data Engineer INR 3000K-4000KAPI Integration | AWS | Ads API | Agile | Batch dataDental insurance | Employee discounts | Health insurance | Mental Health Awareness Days | Paid time offSenior-level Full TimeBangalore, India; Gurgaon, India23h ago
-
Applied AI ML Lead INR 3799K-5572KAWS Bedrock | AWS ECS | AWS Lambda | AWS SageMaker | Apache SparkSenior-level Full TimeBengaluru, Karnataka, India23h ago
-
Senior Agentic AI Engineer - Bangalore INR 3000K-3500KAI Workflow Orchestration | AI workflow | APIs | AWS | CI/CDMid-level Full TimeBengaluru, Karnātaka, India1d ago
-
Senior-level Full TimeHyderabad, India1d ago
-
AI Assisted Development | Claude Code | Cursor | JavaScript | OpenAI CodexSenior-level Full TimeMiami1d ago
-
Data Engineer, Cloud Supply Chain Data Engineering INR 1500K-2200KBigQuery | Data Governance | Data Monitoring | Data Pipelines | Data QualityCollaborative team environment | Mentoring opportunitiesMid-level Full TimeBengaluru, Karnataka, India1d ago
-
Senior Staff Software Engineer, Geo INR 2500K-3465KAnomaly Detection | C++ | Continuous Learning | Data Governance | Data InfrastructureSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Application Engineer, General Software Development INR 1500K-3500KAI integration | Agentic integration | C++ | Cloud platform | Data exchangeMid-level Full TimeHyderabad, Telangana, India1d ago
-
Senior Staff Software Engineer INR 3417K-4800K.Net Core | AWS | Azure | CI/CD | Cloud NativeEnhanced medical benefits | Paid time off | Wellbeing programsSenior-level Full TimeHyderabad, India1d ago
-
Mid-level Full TimePune, Maharashtra, India1d ago
-
Mid-level Full TimePune1d ago
-
APIs | Azure | Azure AI | Databases | GPU ComputingComprehensive benefits package | Inclusive workplace culture | International team collaboration | Professional growth opportunities | Remote work flexibilitySenior-level Full TimeIndia1d ago
-
Principal Data Science - Associate Director- Bangalore INR 1500K-2700KAI Governance | AI System Design | Advanced Analytics | Apache Spark | Artificial IntelligenceMid-level Full TimeBangalore, KA, India1d ago