Senior AI QA Engineer
Bengaluru/Hyderabad/Mumbai/Gurugram, IN
INR 2000K-4200K (estimate) Senior-level Full Time
Tasks
- Build automated LLM evaluation pipelines
- Define quality gates
- Design LLM benchmarks
- Develop Playwright UI test suites
- Develop Python test automation frameworks
- Evaluate prompts models retrieval pipelines and agents
- Implement unit integration API regression and end to end tests
- Integrate automated tests into CI/CD pipelines
- Perform root cause analysis for regressions and production issues
- Test agent reasoning flow memory and task completion
- Test retrieval quality context relevance and tool invocation
- Use OpenTelemetry for tracing and observability
- Validate AI performance and scalability
- Validate RAG pipelines
Perks/Benefits
- N/A
Skills/Tech-stack
Benchmarking | CI/CD | Cause analysis | End to End | End-to-End Testing | Integration Testing | LLM Evaluation | Latency | Observability | OpenTelemetry | Playwright | Precisión | Prompt Regression | Prompt Regression Testing | Pytest | Python | RAG | REST APIs | Recall | Regression testing | Root Cause Analysis | Root cause | Token usage | Unit Testing | Vector Databases | Vector Search
Education
N/A
Language: en
Views: 0
Clicks: 2
Saves: 0
Related jobs
-
Senior-level ContractShaikpet, India14h ago
-
Senior-level Full TimeHyderabad, India15h ago
-
Data Engineer II INR 2000K-3000KAWS | Amazon Kinesis | Amazon Redshift | Apache Airflow | Apache BeamCareer growth opportunities | Complimentary games | Family Leaves | Healthcare coverage | Mental well-being supportSenior-level Full TimeLocations: Hyderabad, Telangana, India15h ago
-
Advanced Engineer-AI/ML INR 2500K-4680KAI orchestration | API Integration | ASR | AWS | Action modelsEmployee discounts | Training opportunities | Tuition reimbursement | Wellness benefitsSenior-level Full TimeBangalore - Karnataka, India - Kalyani …16h ago
-
Senior Software Engineer, AI/ML, Google Cloud INR 2500K-5000KC plus plus | Data Processing | Data Structures | Data structures algorithms | DebuggingSenior-level Full TimeHyderabad, Telangana, India18h ago
-
Principal Silicon Engineer - Storage INR 1163K-3500KAmazon S3 | Azure Blob | Azure Blob Storage | Azure Files | Backup and RecoverySenior-level Full TimeIndia18h ago
-
Lead Software Engineer Java INR 3000K-4500KAI code assistants | AWS | Agile | Apache Spark | Application ResiliencySenior-level Full TimeMumbai, Maharashtra, India18h ago
-
AI/ML Engineer INR 2000K-3500KAzure | Data Pretraining | Data Structures | DeepSpeed | Distributed SystemsCar lease benefits | Career advancement programs | Certification programs | Continuous learning opportunities | Corporate NPSMid-level Full TimeBangalore, Karnataka, India R19h ago
-
Senior / Lead Data Engineer - Azure Databricks INR 1500K-3000KAzure | Azure Databricks | Azure DevOps | CI/CD | Conformed DimensionsSenior-level Full TimeHyderabad, TS, India19h ago
-
Senior-level Full TimeHyderabad, Telangana, India19h ago
-
Advanced Robotics Simulation Engineer INR 2000K-4680KDocker | GPU Computing | Isaac Lab | Isaac Sim | Isaac-GymSenior-level Full TimeChennai, Tamil Nadu, India19h ago
-
Java Full Stack AI Engineer- LINA GI INR 2200K-3200KAPI Security | Angular | App Service | Azure | Azure AppCertification programs | Corporate NPS | Dental insurance | Education reimbursement | Employee assistance programSenior-level Full TimeTelangana, India R20h ago
-
Senior-level Full TimeHyderabad, TS, India20h ago
-
Junior AI Specialist INR 336K-480KAWS | Airflow | Azure | DBT | Data AnalysisCertification programs | Health insurance | Internship opportunities | Language courses | MentorshipEntry-level Full TimeAll Cities , India R20h ago
-
Python, LLM, GenAI/Senior Consultant Specialist INR 3000K-5000KAsynchronous programming | Authentication | Authorization | CI/CD | CachingContinuous professional development | Flexible workingSenior-level Full TimePune, Maharashtra, India R20h ago
-
Data Protection Lead Engineer INR 1500K-3000KArtificial Intelligence | Azure | Cloud platform | DLP | Data ClassificationSenior-level Full TimeIndia21h ago
-
Senior-level Full TimeChennai, Tamil Nadu, India21h ago
-
SO_Cloud_Devops_Engineer_SME_2026 INR 400K-500KAzure Data | Azure Data Factory | CI/CD | Command Line | Command-line scriptingEntry-level Full Timepune, India22h ago
-
Analyst, Business Intelligence INR 3000K-5000KAWS | Azure | Azure OpenAI | CI/CD | Distributed SystemsEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India22h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeEntrepreneurial team | Fast-growing environment | Flexible asynchronous culture | Remote/distributed workMid-level Full TimeBangalore, India23h ago
-
Bash | Data Ingestion | Data Processing | Docker | GCPAsynchronous culture | Flexible remote setting | Friendly laid-back atmosphere | Opportunity for high impactMid-level Full TimeDelhi, India23h ago
-
Sr Data Engineer INR 3000K-4000KAPIs | Airflow | Apache Iceberg | CI/CD | Control-MAnnual leave | Development courses | Employee assistance program | Flexible working options | Life insuranceSenior-level Full TimeBangalore, IN23h ago
-
Senior-level Full TimeBengaluru, Karnataka, India23h ago
-
Advanced Software Engr INR 2500K-4500KAI Agents | API Design | Cloud Platforms | Distributed Systems | DjangoSenior-level Full TimeBengaluru, Karnataka, India23h ago
-
Data Engineer III INR 1500K-2500KAPI Integration | Apache Airflow | Automated testing | CI/CD | Change Data CaptureSenior-level Full TimeChennai, Tamil Nadu, India1d ago