Quality Assurance (QA) - Lead - MiDAS
Tasks
- Architect api test automation
- Architect deterministic ui test automation
- Automate evaluation of non deterministic LLM outputs
- Define quality metrics for AI outputs
- Design and maintain CI/CD pipelines
- Develop genAI evaluation framework
- Embed automated quality checks into github workflows
- Implement faithfulness detection
- Implement hallucination detection
- Implement relevancy detection
- Lead JMeter performance testing
- Monitor probabilistic quality metrics
Perks/Benefits
Skills/Tech-stack
API Testing | CI/CD | GitHub Actions | JMeter | LLM Evaluation | LLM-as-a-Judge | Langfuse | Performance Testing | Playwright | Pytest | Python | RAG | Requests | Retrieval-Augmented Generation | Test automation | UI Testing | Vector Databases
Education
Related jobs
-
Senior-level Full TimeIN-AP-Hyderabad5h ago
-
Senior-level Full TimePune, Maharashtra, India10h ago
-
Lead AI/ML Engineer – Security AI & LLMs INR 3000K-5000KAPI Design | AWS SageMaker | Azure ML | Deep learning | DockerSenior-level Full TimePune, Maharashtra, India10h ago
-
Tech Lead - Java (Java, React, Kong) INR 1500K-3000KAPI Development | API Gateway | API Management | API Security | AgileSenior-level Full TimePune, IN, 41105713h ago
-
Lead Software Engineer - Python, Data, Cloud INR 3000K-4500KAPI Development | AWS Glue | Amazon Athena | Amazon Redshift | Amazon S3Senior-level Full TimeBengaluru, Karnataka, India13h ago
-
Technical Lead-Azure Databricks INR 1500K-2700KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Senior-level Full TimeINDIA - BENGALURU - HP, IN18h ago
-
Lead Data & Analytics -Vice President-Software Engineer INR 3000K-4064KAPI Versioning | Azure | Core Java | Data Modeling | Data WarehousingSenior-level Full TimeCOMMERZ III, OBEROI GARDEN CITY, India18h ago
-
Lead Data Scientist- AML INR 3600K-5000KComputer Vision | Data Analysis | Data Structures | Data Structures and Algorithms | Data VisualizationSenior-level Full TimeTower 02, Manyata Embassy Business Park, …18h ago
-
Java Technical Lead- Career INR 2000K-3000KAWS | Airflow | Apache Beam | Apache Kafka | AzureHealthcare benefits | Hybrid work | Online learning platform | Paid time offSenior-level Full TimeIND-Trivandrum-Equifax Analytics-PEC, India18h ago
-
Lead Data Scientist INR 2817K-5000KAdvanced Excel | Artificial Intelligence | Automation | Credit Risk | Credit risk analyticsFlexible working hours | Hybrid work arrangements | Paid parental leave | Training and development programsSenior-level Full TimeBangalore RMZ-ECO WORLD, India18h ago
-
WFM Data Science Lead INR 2000K-3000KAnomaly Detection | Business Intelligence | Customer Experience | Dashboarding | Data PipelinesSenior-level Full TimeBengaluru, KA, IN1d ago
-
WFM Data Science Lead INR 2000K-3000KAmazon Redshift | Anomaly Detection | BPO | Contact center | Contact center operationsSenior-level Full TimeHyderabad, TS, IN1d ago
-
APIs | Agent systems | Architecture Design | Data Engineering | Data ModelingEquipment support | Flexible working hours | Gifts and rewards | Lunch allowance | Medical insuranceSenior-level Full TimeIndia R1d ago
-
AI Data Engineer III INR 3000K-4000KAgile | Amazon Web Services | Apache Airflow | CI/CD | DBTComprehensive insurance | Education assistance program | Fitness benefits | Flexible working hours | Health checkupsSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Big Data Lead INR 1500K-2500KAWS S3 | Amazon ECS | Amazon IAM | Apache Airflow | CI/CDClient facing communication | Team collaborationSenior-level Full TimeIndia1d ago
-
Lead Software Engineer -Python or Java, big data INR 3000K-4500KAWS | AWS Lambda | Agile | Amazon EMR | Amazon Elastic Compute ServiceSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Lead Software Engineer - AL/ML (Implementation Engineer) INR 4000K-5000KAWS | Azure | CI/CD | Distributed Systems | DjangoFlexible-hybrid work | Health insurance | Paid parental leave | Paid time offSenior-level Full TimeBengaluru1d ago
-
Assistant Director,Collections Analytics Lead ASP INR 2817K-4000KAML | Alteryx | BigQuery | Capacity Planning | Collections analyticsSenior-level Full TimeKolkata, West Bengal, India R1d ago
-
Senior Process Manager INR 2500K-5000KAI system architecture | AWS Bedrock | Autogen | Automation | Azure OpenAISenior-level Full TimePune, Maharashtra, India1d ago
-
Senior-level Full TimePune City,India1d ago
-
Lead Data Analyst INR 2817K-4064KAPI Integration | Dashboarding | Data Automation | Data Visualization | Data cleaningSenior-level Full TimeBengaluru1d ago
-
Lead, Health Analytics and AI Unit, Digital Health INR 3000K-4200KAdvanced Analytics | Artificial Intelligence | By Design | Data Engineering | Data GovernanceSenior-level Full TimeNew Delhi, DL, India1d ago
-
AI Governance | Airflow | Alerting | Anomaly Detection | Artifact RegistryMid-level Full TimePune, MH, IN, 4110061d ago
-
Data Science Engagement Lead INR 2755K-4000KAWS | Agile | Artificial Intelligence | Azure | CI/CDCareer growth | Inclusive workplace | International collaboration | Training and development programmes | Work-life balanceSenior-level Full TimeSHELL CENTRE – CHENNAI, India1d ago
-
Data Analyst - Lead INR 2500K-4000KAdvanced Statistics | Algorithm Development | Artificial Intelligence | Campaign analytics | Customer SegmentationCareer mobility and development | Flexible work arrangements | Leadership programs | Training programs | Well-being programsSenior-level Full TimeFXE-MEISA/IND/GUXAT/GUXAT/Gurugram, India1d ago