Associate Lead - Testing (QA + MLOps)
IN KA Bengaluru, India
INR 1500K-2000K (estimate) Mid-level Full Time
Tasks
- Assess bias and fairness
- Build reusable evaluation harnesses
- Create AI test automation frameworks
- Define AI ML testing strategy
- Design validation frameworks
- Detect data drift and concept drift
- Drive QA transformation programs
- Establish AI quality scorecards
- Generate synthetic data strategies
- Implement CI/CD quality gates
- Lead AI ML quality strategy workshops
- Mentor QA teams
- Own AI testing delivery end to end
- Perform AI risk and readiness assessments
- Perform safety and toxicity testing
- Present quality architecture to CXOs
- Test LLM based applications
- Test LLM hallucinations
- Test chatbot and voice bot conversations
- Test multi agent orchestration
- Test prompt robustness
- Validate API contract for LLM integrations
- Validate RAG retrieval accuracy
- Validate blue green releases
- Validate canary releases
- Validate conversation drift and fallback
- Validate deployment
- Validate end to end ML lifecycle
- Validate intent classification
- Validate model training and hyperparameter tuning
- Validate model versioning and registry
- Validate prompt engineering
Perks/Benefits
- N/A
Skills/Tech-stack
AI Quality Scorecard | AI quality | API Gateway | API Testing | AWS DevOps | AWS Glue | AWS Lambda | Amazon API | Amazon API Gateway | Amazon Bedrock | Amazon CloudWatch | Amazon Feature Store | Amazon IAM | Amazon Kinesis | Amazon Model Monitor | Amazon S3 | Amazon SageMaker | Amazon SageMaker Pipelines | Automation Testing | BLEU | Bias and fairness | Blue-Green Deployment | Blue/green | CI/CD | Canary Release | CloudFormation | CodeBuild | CodePipeline | Concept drift | Data Drift | Data Generation | Embedding similarity | Feature Store | Git | Infrastructure as Code | LLM | Langchain | Machine Learning | Observability | Prompt engineering | PyTorch | Python | RAG | ROUGE | Safety testing | Sagemaker Pipelines | Scikit-learn | Synthetic Data Generation | Synthetic data | TensorFlow | Terraform | Toxicity Testing | “as-code”
Education
N/A
Related jobs
-
Data Engineer (MS) (Remote) INR 2040K-3380KCI/CD | Data Transformation | Data Validation | Date normalization | ETLMentorship opportunities | Professional growth | Remote workSenior-level Full TimeMaharashtra, Pune, India R9h ago
-
AI / LLM Integration Engineer (Remote) INR 1800K-2800KAPI Integration | Anthropic SDK | Async APIs | Benchmarking | Confidence scoring100 percent remote | Collaborative culture | Inclusive team culture | Professional growthMid-level Full TimeMaharashtra, Pune, India R9h ago
-
ML / LLM Engineer (Remote) INR 2500K-3000KAmazon Web Services | Azure | Classification | Feature Engineering | Language ModelsRemote workMid-level Full TimeMaharashtra, Pune, India R9h ago
-
Sr. Computer Vision & Edge AI Engineer INR 2500K-5000KComputer Vision | Deep learning | FP16 | GPU Acceleration | Human pose estimationSenior-level Full TimeMaharashtra, Pune, India10h ago
-
Azure Databricks INR 1500K-2000KAccess Controls | Apache Spark | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimeIN-AP-Hyderabad10h ago
-
Senior-level Full TimeIN-TN-Chennai10h ago
-
Mid-level Full TimeIN-KA-Bangalore10h ago
-
Software Engineer - AI/ML with Python INR 2500K-3500KAI | AWS | Airflow | Apache Spark | BazelCollaborative work environment | Professional growth | Training and developmentSenior-level Full TimeBangalore - Karnataka, India - Kalyani …11h ago
-
Mid-level Full TimeBangalore - Karnataka, India - Kalyani …11h ago
-
Business glossary | Data Catalog | Data Contracts | Data Lineage | Data ModelingAsync first | Flexible overlap hours | RemoteSenior-level Full TimePune, Maharashtra, India11h ago
-
Senior Data Engineer INR 3000K-4000KADLS Gen2 | Audit Logging | Azure Data | Azure Data Factory | Azure Data LakeSenior-level Full TimePune, Maharashtra, India11h ago
-
Engineering Analyst II, Gemini and Labs INR 1000K-1500KAdversarial techniques | Automation | Classifier | Data Analysis | Fine TuningHoliday coverage | Rotating on call coverage | Weekend coverageMid-level Full TimeBengaluru, Karnataka, India12h ago
-
Senior Database Engineer INR 2520K-3380KAerospike | Azure | Azure Cosmos | Azure Cosmos DB | Azure DataOn-call support | Relocation assistance | Rotational shifts 24x7Senior-level Full TimeBengaluru, KA, IN13h ago
-
Lead Software Engineer - Data Engineer INR 1500K-2400KABAC | AWS | AWS Glue | AWS Lambda | AWS Step FunctionsSenior-level Full TimeBengaluru, Karnataka, India13h ago
-
Senior Architect, ML Engineering INR 2500K-4500KAWS | Agentic Workflows | Async Programming | Azure | Circuit BreakersSenior-level Full TimePune, Maharashtra, India14h ago
-
Entry-level Full TimePune, Maharashtra, India14h ago
-
Data Engineer - Senior INR 3000K-3500KAnomaly Detection | Application Insights | Azure | Azure Application Insights | Azure SQLSenior-level Full TimePune, Maharashtra, India15h ago
-
Engineer (India Office)-Power BI INR 1500K-2000KData Quality | Data Validation | Data Visualization | Power BI | Predictive ModelingMid-level Full TimeBangalore, Karnataka, India15h ago
-
Staff Machine Learning Manager INR 2160K-2800KAnomaly Detection | Attribution Modeling | Auction theory | Audience Segmentation | Bid optimizationBring children to work | Bring pets to work | Food provided | Gym | Training opportunitiesSenior-level Full TimeBangalore16h ago
-
Software Engineer INR 1100K-1500KAirflow | Azure Data | Azure Data Factory | Control-M | Data FactoryEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India16h ago
-
Germany (FS- D&A)- Executive-BLR INR 500K-500KAgile | Azure Data | Azure Data Factory | Azure Databricks | Azure DevOpsHybrid workExecutive-level Full TimeBangalore, Karnataka, India16h ago
-
Cloud Support Engineer – Backup & Disaster Recovery INR 938K-1500KAWS | Backup and Disaster Recovery | Cloud Computing | Cloud Storage | DatabasesMid-level Full TimeBangalore, India16h ago
-
Embedded Linux Engineer - Platform Software INR 980K-1200KALSA | ARM | Asynchronous programming | BLE | BluetoothMid-level Full TimeGurugram, Haryana, India17h ago
-
Data Operations Engineer INR 2040K-3380KApache Airflow | Data Pipelines | Data Refresh | Data Warehousing | Data pipelineEmergency incident response support | On-call rotationSenior-level Full TimeBengaluru, INDIA, India R17h ago
-
Mid-level Full TimeChennai, Tamil Nadu, India17h ago