Quality Assurance (QA) - Lead - MiDAS
Tasks
- Architect api test automation
- Architect deterministic ui test automation
- Automate evaluation of non deterministic LLM outputs
- Define quality metrics for AI outputs
- Design and maintain CI/CD pipelines
- Develop genAI evaluation framework
- Embed automated quality checks into github workflows
- Implement faithfulness detection
- Implement hallucination detection
- Implement relevancy detection
- Lead JMeter performance testing
- Monitor probabilistic quality metrics
Perks/Benefits
Skills/Tech-stack
API Testing | CI/CD | GitHub Actions | JMeter | LLM Evaluation | LLM-as-a-Judge | Langfuse | Performance Testing | Playwright | Pytest | Python | RAG | Requests | Retrieval-Augmented Generation | Test automation | UI Testing | Vector Databases
Education
Related jobs
-
Apache Airflow | Apache Beam | Apache Flink | Asynchronous Messaging | BigQueryFlexible working | Opportunities for growth | Professional developmentMid-level Full TimePune, Maharashtra, India R12h ago
-
AI/ML Engineering Lead/(Gen AI,Python,LLM,ETL)Associate Director INR 2000K-4000KAgent Orchestration | Agentic AI | Apache Airflow | Apache Beam | Audit LoggingFlexible working | Inclusive environment | Professional developmentMid-level Full TimePune, Maharashtra, India R14h ago
-
Lead Data Engineer INR 2500K-4500KAWS | AWS Glue | Access Control | Airflow | AzureCareer mobility | Cross portfolio opportunities | Leadership development | Learning opportunitiesSenior-level Full TimeIndia R20h ago
-
Senior-level Full TimePune, India23h ago
-
Lead Security Engineer - Data Security & RL for Cyberecurity INR 2500K-6000KAWS | Access Control | Adaptive Security | Anomaly Detection | Automated incident responseSenior-level Full TimeBengaluru, Karnataka, India23h ago
-
Gen AI QA - Automation Testing (Senior Engineer / Engineer) INR 1860K-3000KAWS | AWS Glue | Amazon S3 | Data Migration | Data ValidationSenior-level Full TimeBengaluru, India23h ago
-
Aeron | Algorithmic trading | Automated testing | CI/CD | CucumberSenior-level Full Time110380-IND-BENGALURU-INTL BLR Twr-1&2 CARNATION, India23h ago
-
AI tools | Consumer lending | Credit Bureau Reporting | Credit bureau | Data GovernanceSenior-level Full Time111442-IND-BENGALURU-INTL BLR BLK B3 PETUNIA, India23h ago
-
Acquisition Analytics | Artificial Intelligence | Azure AI | Clustering | Copilot StudioSenior-level Full Time110380-IND-BENGALURU-INTL BLR Twr-1&2 CARNATION, India23h ago
-
Lead Developer, Java, Spark and GCP INR 2771K-4064KGCP | Hadoop | Hive | Java | Logical Flow EngineHybrid work | Work-life flexibilitySenior-level Full TimePune, India R23h ago
-
Data Analytics Lead - Data Scientist INR 2771K-5000KBig Data | Data Warehousing | ETL | Explainability | FairnessSenior-level Full TimeLocation(s): Pune, Maharashtra, India23h ago
-
Senior-level Full TimeBengaluru, Maruthi Onyx - TESCO TSA, …23h ago
-
ABAC | Airflow | Audit Logging | BigQuery | CDCFlexible working | Inclusive and diverse environment | Professional development opportunitiesMid-level Full TimePune, Maharashtra, India R1d ago
-
Lead Software Engineer - Platform Data Team INR 2500K-4200KAWS | Ansible | Azure | Benchmarking | CAP TheoremCareer growth mentoring | Global team collaboration | Remote workSenior-level Full TimeIndia1d ago
-
Software Engineering Lead INR 3672K-5000KAI Observability | API Design | Agent systems | Agentic AI | Amazon Web ServicesCareer development opportunities | Comprehensive benefits | Inclusion-focused cultureSenior-level Full TimePrimary location: Hyderabad, India1d ago
-
Lead Software Engineer - Workday INR 1800K-2600KAPI Gateway | AWS CloudWatch | AWS Elastic Load Balancing | AWS Lambda | AWS VPCSenior-level Full TimeOff Embassy Golf Links Business Park, …2d ago
-
Access Control | Agile | Alerting | Apache Airflow | AuditabilityContinuous professional development | Flexible working | Inclusive and diverse work environmentMid-level Full TimePune, Maharashtra, India3d ago
-
Lead Software Engineer - (SRE principles , AI tools, AWS) INR 3000K-6000KAIOps | API Gateway | AWS ECS | AWS Lambda | Amazon APISenior-level Full TimeBengaluru, Karnataka, India3d ago
-
Senior Data Analyst INR 2817K-4000KAlation | Analysis Services | Azure Analysis | Azure Analysis Services | Azure DataSenior-level Full TimeTelangana, India R3d ago
-
Data Science & Engineering Lead INR 3000K-4500KAPI Gateway | AWS Glue | AWS Lambda | Airbyte | Amazon EC2Senior-level Full TimeIndia - Remote R3d ago
-
Lead Software Engineer INR 1500K-3000KAWS Cloud | Apache Airflow | Azure Data | Azure Data Factory | Data FactorySenior-level Full TimeBangalore, India3d ago
-
Lead Software Engineer INR 1500K-3000KAWS Cloud | Airflow | Apache Airflow | Azure Data | Azure Data FactorySenior-level Full TimeBangalore, India3d ago
-
QA Lead - Manual, Automation & AI Testing INR 1500K-3000KAI Testing | API Integration | API Testing | Appium | Automation TestingSenior-level Full TimeBengaluru, Karnataka, India3d ago
-
Quality Engineering Lead – Data & Reporting Platforms INR 1600K-3000KAI Testing | API Testing | AWS Athena | Access Control | AgileSenior-level Full TimeLocation(s): Pune, Maharashtra, India3d ago
-
Engineering & Delivery Lead - Data Services INR 3672K-5000KAWS Athena | Apache Drill | Apache Flink | Apache Kafka | Apache RangerSenior-level Full TimeLocation(s): Pune, Maharashtra, India3d ago