Associate Lead - Testing (QA + MLOps)
IN KA Bengaluru, India
INR 1500K-2000K (estimate) Mid-level Full Time
Tasks
- Assess bias and fairness
- Build reusable evaluation harnesses
- Create AI test automation frameworks
- Define AI ML testing strategy
- Design validation frameworks
- Detect data drift and concept drift
- Drive QA transformation programs
- Establish AI quality scorecards
- Generate synthetic data strategies
- Implement CI/CD quality gates
- Lead AI ML quality strategy workshops
- Mentor QA teams
- Own AI testing delivery end to end
- Perform AI risk and readiness assessments
- Perform safety and toxicity testing
- Present quality architecture to CXOs
- Test LLM based applications
- Test LLM hallucinations
- Test chatbot and voice bot conversations
- Test multi agent orchestration
- Test prompt robustness
- Validate API contract for LLM integrations
- Validate RAG retrieval accuracy
- Validate blue green releases
- Validate canary releases
- Validate conversation drift and fallback
- Validate deployment
- Validate end to end ML lifecycle
- Validate intent classification
- Validate model training and hyperparameter tuning
- Validate model versioning and registry
- Validate prompt engineering
Perks/Benefits
- N/A
Skills/Tech-stack
AI Quality Scorecard | AI quality | API Gateway | API Testing | AWS DevOps | AWS Glue | AWS Lambda | Amazon API | Amazon API Gateway | Amazon Bedrock | Amazon CloudWatch | Amazon Feature Store | Amazon IAM | Amazon Kinesis | Amazon Model Monitor | Amazon S3 | Amazon SageMaker | Amazon SageMaker Pipelines | Automation Testing | BLEU | Bias and fairness | Blue-Green Deployment | Blue/green | CI/CD | Canary Release | CloudFormation | CodeBuild | CodePipeline | Concept drift | Data Drift | Data Generation | Embedding similarity | Feature Store | Git | Infrastructure as Code | LLM | Langchain | Machine Learning | Observability | Prompt engineering | PyTorch | Python | RAG | ROUGE | Safety testing | Sagemaker Pipelines | Scikit-learn | Synthetic Data Generation | Synthetic data | TensorFlow | Terraform | Toxicity Testing | “as-code”
Education
N/A
Related jobs
-
Python Developer INR 2000K-2500KAPI Gateway | AWS CDK | AWS CloudFormation | AWS Lambda | AWS Step FunctionsMid-level Full TimeHyderabad, India9h ago
-
Data Engineer INR 2000K-2000KDBT | Data Governance | Data Modeling | Data Warehousing | Dimensional ModelingSenior-level Full TimeHyderabad, India9h ago
-
Business Analyst, Data Engineer INR 1500K-2000KAWS Glue | AWS Lambda | AWS Step Functions | Amazon Web Services | CI/CDAI certifications | Mentorship | TrainingMid-level Full TimeIndia-Bangalore9h ago
-
Principal Consultant – Alteryx, SQL, Pentaho INR 2520K-4000KAlteryx | Analytics reporting | Cloud Platforms | Data Reconciliation | Data ValidationSenior-level Full TimeIndia-Mumbai9h ago
-
Lead Consultant – Alteryx, SQL, Pentaho INR 2520K-4000KAlteryx Designer | Cloud Computing | Data Reconciliation | Data Validation | Data WarehousingSenior-level Full TimeIndia-Mumbai9h ago
-
Data Engineer (Pyspark+SQL, AWS) INR 2000K-2500KAWS Glue | Amazon Athena | Amazon Aurora | Amazon DynamoDB | Amazon EC2Mid-level Full TimeHyderabad, Telangana, India10h ago
-
Data Engineer (Python+SQL, AWS) INR 1500K-2000KAWS Glue | AWS Lambda | AWS S3 | Amazon CloudWatch | Amazon SNSMid-level Full TimeNoida, Uttar Pradesh, India10h ago
-
API Gateway | AWS Bedrock | AWS Lambda | AWS Step Functions | Amazon CloudWatchEntry-level Full TimeMumbai, MH, India10h ago
-
Senior PostgreSQL Developer / Database Engineer INR 2000K-2040KAWS | AWS RDS | Azure | Backup and Recovery | BashSenior-level Full TimeTamil Nadu, Chennai, India11h ago
-
Senior Staff Software Engineer, Applied AI, Search INR 2500K-4500KC++ | Data Processing | Debugging | Fine Tuning | Information RetrievalSenior-level Full TimeBengaluru, Karnataka, India11h ago
-
Data Domain Architect - Associate INR 2000K-2040KAI/ML | Alteryx | Cloud Data | Cloud Data Warehouse | Data EngineeringSenior-level Full TimeBengaluru, Karnataka, India11h ago
-
Automation Engineer – Analytics and Automation INR 1500K-2156KAI | API Integration | CMDB | Cloud Platforms | Cloud platformMid-level Full TimeIndia11h ago
-
Consultant INR 1500K-2300KAWS | Azure | Cloud platform | Data Engineering | Feature EngineeringCollaborative cross-functional team | Competitive benefits package | Cutting-edge technology exposure | End-to-end ownership | High-impact projectsMid-level Full TimeBengaluru, Karnataka, India12h ago
-
Staff SW Engineer- Java and GenAI- 6+yrs experience INR 3200K-4600KAWS | Agile | CI/CD | Docker | GANContinued learning opportunities | MentorshipSenior-level Full TimeBengaluru, INDIA, India12h ago
-
Assistant Manager - AI Research INR 2500K-3199KAWS | Azure | CI/CD | Cloud platform | Data Leakage PreventionMid-level Full TimeGurgaon, HR, India13h ago
-
AWS | Apache Spark | CI/CD | Git | HadoopMid-level Full TimeBengaluru, Karnataka, India13h ago
-
Senior-level Full TimeHyderabad, Telangana, India14h ago
-
AI/ML Engineer INR 1500K-3000KAzure | C++ | Deep learning | Generative AI | Hugging FaceCertification programs | Corporate pension scheme | Dental coverage | Education reimbursement program | Employee assistance programMid-level Full TimeBangalore, Karnataka, India14h ago
-
Performance Optimization Engineer INR 1500K-2245KCommunication Protocols | DCS | Dashboards | Data Analysis | Data Historians401k | Dental coverage | Employee assistance program | Flexible time off | Medical insuranceMid-level Full TimeNOIDA, UTTAR PRADESH, India14h ago
-
AI Engineer (Manager level) INR 1500K-3000KAI Search | Agile | Azure AI | Azure AI Search | Azure CIMid-level Full TimeHyderabad, India15h ago
-
Software Engineer - Sr. Consultant (10+ years of experience - Python - Gen AI and Agentic AI) INR 2500K-3800KAPIs | Agentic AI | Application development | Debugging | Generative AIHybrid workSenior-level Full TimeBengaluru, INDIA, India15h ago
-
Associate Principal, SOX Analytics INR 1120K-1600KAPI Integration | Audit Methodology | Coso | Data Analysis | Data AutomationMid-level Full TimeIndia15h ago
-
AWS | Apache Spark | CI/CD | Git | HadoopExecutive-level Full TimeBengaluru, Karnataka, India16h ago
-
Mid-level Full TimeBangalore, KA, IN, 56007616h ago
-
Senior-level Full TimeMumbai, Maharashtra, India17h ago