Senior Software Development Test Engineer - AI
INR 2500K-4800K (estimate) Senior-level Full Time
Tasks
- Build LLM as judge and automated scoring pipelines
- Build automated testing frameworks for ML models and AI agents
- Build regression suites for quality drift detection
- Create evaluation frameworks for non deterministic model outputs
- Define quality metrics guardrails and automated monitoring for model degradation
- Detect hallucinations unsafe outputs and edge cases
- Generate test cases and synthetic data using AI and LLMs
- Measure response accuracy relevance and consistency
- Perform root cause analysis for production model and behavioral issues
- Validate integration of ML models and AI agents in end to end product flows
- Validate model correctness accuracy latency and behavioral consistency
Perks/Benefits
- N/A
Skills/Tech-stack
API Testing | AWS | Artificial Intelligence | Azure | CI/CD | Containerization | DeepEval | Docker | Drift Detection | Embeddings | Evaluation metrics | F1-score | GCP | Hallucination detection | Integration Testing | Java | Kubernetes | LLM | Langsmith | MLflow | Machine Learning | Model Observability | Precision-recall | Production Monitoring | Prompt engineering | Python | RAG | Ragas | Regression testing | Retrieval-Augmented Generation | Synthetic data | Test automation | Vector Search | Weights and Biases
Education
N/A
Related jobs
-
Staff Software Development Test Engineer INR 1632K-3000KAPI Testing | Cause analysis | Data Lake | Data Modeling | Data PipelinesSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Senior Software Development Test Engineer INR 2000K-3000KAPI Testing | Cause analysis | Data Lineage | Data Pipeline Testing | Data QualitySenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Software Development Engineer.5 INR 3264K-5000KAI Agents | AI Evaluation | AWS | Anthropic Claude | ArgoCDSenior-level Full TimeBangalore, India R2d ago
-
Senior-level Full TimePune, Gera Commerzone SEZ, India3d ago
-
Sr. Software Development Engineer in Test - Python | Automation | Robot Framework | 5y to 7y INR 2475K-3500KAI tools | API Testing | CI/CD | Debugging | Distributed SystemsFlexible work hours | Medical, dental, and vision coverage | Paid parental leave | Paid time off | Retirement plansSenior-level Full TimeIndia, Bangalore14d ago
-
Lead Software Development Engineer in Test - AI INR 2000K-3500KAI Model Testing | AI model | Automation Framework | Bias Testing | Continuous integrationCareer growth | Remote-first culture | Training and upskillingSenior-level Full TimeIndia17d ago
-
AI Software Development Engineer in Test (AI-SDET) INR 2000K-3000KAI evals | API Testing | AWS | Agile Scrum | AutogenAnnual bonus | Extra days off | Health care benefits | In-company English lessons | Internet reimbursementSenior-level Full TimeMumbai, India1mo ago
-
Senior Software Development Engineer INR 2000K-3500KAWS | Agile | Amazon SNS | Amazon SQS | Automated testingSenior-level Full TimeOffice - Noida, India1mo ago