Senior Software Development Test Engineer - AI
Bengaluru, Karnataka, India
INR 2500K-4800K (estimate) Senior-level Full Time
Tasks
- Build LLM as judge and automated scoring pipelines
- Build automated testing frameworks for ML models and AI agents
- Build regression suites for quality drift detection
- Create evaluation frameworks for non deterministic model outputs
- Define quality metrics guardrails and automated monitoring for model degradation
- Detect hallucinations unsafe outputs and edge cases
- Generate test cases and synthetic data using AI and LLMs
- Measure response accuracy relevance and consistency
- Perform root cause analysis for production model and behavioral issues
- Validate integration of ML models and AI agents in end to end product flows
- Validate model correctness accuracy latency and behavioral consistency
Perks/Benefits
- N/A
Skills/Tech-stack
API Testing | AWS | Artificial Intelligence | Azure | CI/CD | Containerization | DeepEval | Docker | Drift Detection | Embeddings | Evaluation metrics | F1-score | GCP | Hallucination detection | Integration Testing | Java | Kubernetes | LLM | Langsmith | MLflow | Machine Learning | Model Observability | Precision-recall | Production Monitoring | Prompt engineering | Python | RAG | Ragas | Regression testing | Retrieval-Augmented Generation | Synthetic data | Test automation | Vector Search | Weights and Biases
Education
N/A
Related jobs
-
Sr. Software Development Engineer in Test - Python | Automation | Robot Framework | 5y to 7y INR 2300K-3465KAPI Testing | CI/CD | Distributed Systems | Docker | Event DrivenFamily-friendly benefits | Flexible work hours | Medical, dental & vision coverage | Paid parental leave | Paid time offSenior-level Full TimeIndia, Bangalore2d ago
-
Senior SDET – Performance Engineering INR 2500K-4000KAKS | APM | Azure | C# | CI/CD401k | Cafe Amenities | Fitness Center | Health insurance | Recreational facilitiesSenior-level Full TimePune, MH, India5d ago
-
Sr Software Development Engineer, Selection Monitoring INR 3242K-4580KAgentic AI | Big Data | Build Automation | Cloud Computing | Code reviewSenior-level Full TimeBengaluru, Karnataka, IND7d ago
-
Senior AI Software Development Engineer INR 3700K-5000KAPI Design | Agent Frameworks | Authentication | Authorization | BatchingSenior-level Full TimeHyderabad, India10d ago
-
Senior AI Software Development Engineer INR 3584K-5000KAPI Design | API Integrations | Agent Frameworks | Authentication | AuthorizationSenior-level Full TimeHyderabad, India10d ago
-
Senior AI Software Development Engineer INR 3700K-5000KAPI Design | Agent Frameworks | Authentication | Authorization | BatchingSenior-level Full TimeHyderabad, India10d ago
-
Staff SDET - Open Data Platform (ODP) INR 1600K-2500KCI/CD | Defect Management | Distributed Systems | Functional testing | GitContinuous improvement culture | Cross-functional collaboration | MentorshipSenior-level Full TimeBengaluru11d ago
-
Staff Software Development Test Engineer INR 2000K-3500KAWS | Anthropic API | Azure | CI/CD | CypressSenior-level Full TimeBengaluru, Karnataka, India16d ago
-
Software Development Engineer in Test (SDET) INR 2300K-4200KAWS | Azure | CI/CD | Cloud platform | CypressSenior-level Full TimeBengaluru, Karnataka, India16d ago
-
Staff Software Development Test Engineer INR 1632K-3000KAPI Testing | Cause analysis | Data Lake | Data Modeling | Data PipelinesSenior-level Full TimeBengaluru, Karnataka, India24d ago
-
Senior Software Development Test Engineer INR 2000K-3000KAPI Testing | Cause analysis | Data Lineage | Data Pipeline Testing | Data QualitySenior-level Full TimeBengaluru, Karnataka, India24d ago
-
Software Development Engineer.5 INR 3264K-5000KAI Agents | AI Evaluation | AWS | Anthropic Claude | ArgoCDSenior-level Full TimeBangalore, India R25d ago
-
Senior-level Full TimePune, Gera Commerzone SEZ, India26d ago
-
Lead Software Development Engineer in Test - AI INR 2000K-3500KAI Model Testing | AI model | Automation Framework | Bias Testing | Continuous integrationCareer growth | Remote-first culture | Training and upskillingSenior-level Full TimeIndia1mo ago