Senior AI QA Engineer
Bengaluru/Hyderabad/Mumbai/Gurugram, IN
INR 2000K-4200K (estimate) Senior-level Full Time
Tasks
- Build automated LLM evaluation pipelines
- Define quality gates
- Design LLM benchmarks
- Develop Playwright UI test suites
- Develop Python test automation frameworks
- Evaluate prompts models retrieval pipelines and agents
- Implement unit integration API regression and end to end tests
- Integrate automated tests into CI/CD pipelines
- Perform root cause analysis for regressions and production issues
- Test agent reasoning flow memory and task completion
- Test retrieval quality context relevance and tool invocation
- Use OpenTelemetry for tracing and observability
- Validate AI performance and scalability
- Validate RAG pipelines
Perks/Benefits
- N/A
Skills/Tech-stack
Benchmarking | CI/CD | Cause analysis | End to End | End-to-End Testing | Integration Testing | LLM Evaluation | Latency | Observability | OpenTelemetry | Playwright | Precisión | Prompt Regression | Prompt Regression Testing | Pytest | Python | RAG | REST APIs | Recall | Regression testing | Root Cause Analysis | Root cause | Token usage | Unit Testing | Vector Databases | Vector Search
Education
N/A
Related jobs
-
Agentic AI Data Engineer INR 2500K-5000KAWS Glue | AWS Lambda | AWS Step Functions | Amazon Athena | Amazon BedrockSenior-level Full TimeGurugram, Haryana, India3h ago
-
Data Scientist INR 2500K-4200KAmazon SageMaker | Azure AI | Azure Synapse | Azure Synapse Analytics | C++Mid-level Full TimeHyderabad, Telangana, India6h ago
-
Data Engineer INR 1500K-2980KAPI Security | AWS | Angular | CSS3 | CypressEmployee assistance program | Employee discounts | Gym reimbursement | Healthcare coverage | Learning platform accessMid-level Full TimeBangalore, Karnataka, India7h ago
-
Senior-level Full TimePune, MH, India9h ago
-
AI Engineering INR 3242K-5000KAgent workflows | Audit Logging | Authentication | Authorization | Automated testingSenior-level Full TimeBangalore, KA, IN13h ago
-
Kafka Administrator INR 2800K-4200KAWS EC2 | AWS Glue | AWS IAM | AWS MSK | AWS SecretsBudget for courses and technical conferences | Career progression | Funded certification paths | Hybrid/Remote flexibility | MentorshipSenior-level Full TimeBengaluru, Karnataka, India18h ago
-
Lead MySQL Database Engineer INR 2000K-3100KAWS EC2 | AWS RDS | Amazon Aurora | Ansible | AppDynamicsIn-office work culture | Mentoring and developmentSenior-level Full TimeBengaluru, Karnataka, India18h ago
-
Associate TSE II INR 1400K-2100KApplication Architecture | Atlas | Benchmarking | Cloud Manager | Data ArchitectureHybrid workMid-level Full TimeGurugram21h ago
-
Senior-level Full TimeHyderabad, India22h ago
-
Senior Software Engineer INR 2800K-4200KAWS | Bash | CI/CD | CICS | CobolReward and recognition | Volunteering opportunities | Wellness | Work-life balanceSenior-level Full TimeHyderabad, India22h ago
-
Software Engineer III - Data Engineer INR 2500K-4000KAgile | Application Resiliency | Artificial Intelligence | Automated testing | CI/CDSenior-level Full TimeHyderabad, Telangana, India22h ago
-
Senior manager- Data Engineer INR 3000K-4000KAPI Integration | AWS | Ads API | Agile | Batch dataDental insurance | Employee discounts | Health insurance | Mental Health Awareness Days | Paid time offSenior-level Full TimeBangalore, India; Gurgaon, India23h ago
-
Applied AI ML Lead INR 3799K-5572KAWS Bedrock | AWS ECS | AWS Lambda | AWS SageMaker | Apache SparkSenior-level Full TimeBengaluru, Karnataka, India23h ago
-
Senior Agentic AI Engineer - Bangalore INR 3000K-3500KAI Workflow Orchestration | AI workflow | APIs | AWS | CI/CDMid-level Full TimeBengaluru, Karnātaka, India1d ago
-
Senior-level Full TimeHyderabad, India1d ago
-
AI Assisted Development | Claude Code | Cursor | JavaScript | OpenAI CodexSenior-level Full TimeMiami1d ago
-
Data Engineer, Cloud Supply Chain Data Engineering INR 1500K-2200KBigQuery | Data Governance | Data Monitoring | Data Pipelines | Data QualityCollaborative team environment | Mentoring opportunitiesMid-level Full TimeBengaluru, Karnataka, India1d ago
-
Senior Staff Software Engineer, Geo INR 2500K-3465KAnomaly Detection | C++ | Continuous Learning | Data Governance | Data InfrastructureSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Application Engineer, General Software Development INR 1500K-3500KAI integration | Agentic integration | C++ | Cloud platform | Data exchangeMid-level Full TimeHyderabad, Telangana, India1d ago
-
Senior Staff Software Engineer INR 3417K-4800K.Net Core | AWS | Azure | CI/CD | Cloud NativeEnhanced medical benefits | Paid time off | Wellbeing programsSenior-level Full TimeHyderabad, India1d ago
-
Mid-level Full TimePune, Maharashtra, India1d ago
-
Mid-level Full TimePune1d ago
-
APIs | Azure | Azure AI | Databases | GPU ComputingComprehensive benefits package | Inclusive workplace culture | International team collaboration | Professional growth opportunities | Remote work flexibilitySenior-level Full TimeIndia1d ago
-
Data Engineer INR 3000K-4000KData Ingestion | Data Pipelines | Data Quality | Data Transformation | Data ValidationSenior-level Full TimePune, Maharashtra, India1d ago
-
EAI Software Engineer INR 1500K-2100KAgile | Airflow Orchestration | Apache Airflow | CI/CD | Cloud ComposerMid-level Full TimeHyderabad, India1d ago