AI Test Engineer - VLabs
Bengaluru, Karnataka, India
INR 2500K-4800K (estimate) Senior-level Full Time
Tasks
- Build Python evaluation frameworks
- Build and maintain ground truth datasets
- Continuously update datasets for business requirements
- Define classification and extraction accuracy standards
- Design adversarial and boundary test inputs
- Design and operate drift detection frameworks
- Embed evaluation into CI CD pipelines
- Establish performance degradation thresholds
- Execute API testing for AI pipeline endpoints
- Implement AI as judge evaluation
- Perform end to end workflow and agent testing
- Support responsible AI governance requirements
- Translate AI testing strategy into test scenarios
- Translate evaluation results into actionable insights
- Validate LLM and document model outputs
- Validate data persistence with SQL
Perks/Benefits
Skills/Tech-stack
AI Foundry | AI as Judge | API Testing | AWS Bedrock | Adversarial Testing | Azure AI | Azure AI Foundry | CI/CD | Data Validation | DeepEval | Document Classification | Drift Detection | HTTPx | Hallucination detection | Information Extraction | LLM Evaluation | Langchain | Llamaindex | Newman | OCR | Postman | Prompt design | Prompt engineering | Promptfoo | Python | Ragas | Requests | SQL | Statistical Analysis | VLM
Education
Roles
AI | AI Test Engineer | Engineer | Software Test Engineer | Test Engineer
Related jobs
-
Agentic AI Data Engineer INR 2500K-5000KAWS Glue | AWS Lambda | AWS Step Functions | Amazon Athena | Amazon BedrockSenior-level Full TimeGurugram, Haryana, India4h ago
-
Data Scientist INR 2500K-4200KAmazon SageMaker | Azure AI | Azure Synapse | Azure Synapse Analytics | C++Mid-level Full TimeHyderabad, Telangana, India7h ago
-
Data Engineer INR 1500K-2980KAPI Security | AWS | Angular | CSS3 | CypressEmployee assistance program | Employee discounts | Gym reimbursement | Healthcare coverage | Learning platform accessMid-level Full TimeBangalore, Karnataka, India7h ago
-
Software Engineer - Tableau INR 2520K-4000KA/B | A/B Testing | Agile | Attribution Modeling | Audience SegmentationFlexible work modelSenior-level Full TimeHyderabad, India8h ago
-
Senior-level Full TimePune, MH, India9h ago
-
Data Engineer INR 2771K-4064KAgile | Azure Data | Azure Data Services | Azure DevOps | Cloud ConceptsLearning and career development | Management training | Modern infrastructure | Online learningSenior-level Full TimeChennai, IN, 600 03214h ago
-
AI Engineering INR 3242K-5000KAgent workflows | Audit Logging | Authentication | Authorization | Automated testingSenior-level Full TimeBangalore, KA, IN14h ago
-
Kafka Administrator INR 2800K-4200KAWS EC2 | AWS Glue | AWS IAM | AWS MSK | AWS SecretsBudget for courses and technical conferences | Career progression | Funded certification paths | Hybrid/Remote flexibility | MentorshipSenior-level Full TimeBengaluru, Karnataka, India18h ago
-
Lead MySQL Database Engineer INR 2000K-3100KAWS EC2 | AWS RDS | Amazon Aurora | Ansible | AppDynamicsIn-office work culture | Mentoring and developmentSenior-level Full TimeBengaluru, Karnataka, India18h ago
-
Senior-level Full TimeHyderabad, India23h ago
-
Senior Software Engineer INR 2800K-4200KAWS | Bash | CI/CD | CICS | CobolReward and recognition | Volunteering opportunities | Wellness | Work-life balanceSenior-level Full TimeHyderabad, India23h ago
-
Software Engineer III - Data Engineer INR 2500K-4000KAgile | Application Resiliency | Artificial Intelligence | Automated testing | CI/CDSenior-level Full TimeHyderabad, Telangana, India23h ago
-
Senior manager- Data Engineer INR 3000K-4000KAPI Integration | AWS | Ads API | Agile | Batch dataDental insurance | Employee discounts | Health insurance | Mental Health Awareness Days | Paid time offSenior-level Full TimeBangalore, India; Gurgaon, India23h ago
-
Applied AI ML Lead INR 3799K-5572KAWS Bedrock | AWS ECS | AWS Lambda | AWS SageMaker | Apache SparkSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Product and Solutions Development Manager INR 2200K-3000KAgile | Apache Kafka | Apache Spark | CI/CD | ContainerizationMid-level Full TimeBangalore - Karnataka - India1d ago
-
Senior Agentic AI Engineer - Bangalore INR 3000K-3500KAI Workflow Orchestration | AI workflow | APIs | AWS | CI/CDMid-level Full TimeBengaluru, Karnātaka, India1d ago
-
Senior AI Architect INR 2500K-5000KAWS | Agent Development | Agent Development Kit | Agent systems | Agentic AISenior-level Full TimeIndia1d ago
-
Senior-level Full TimeHyderabad, India1d ago
-
AI Assisted Development | Claude Code | Cursor | JavaScript | OpenAI CodexSenior-level Full TimeMiami1d ago
-
Data Engineer, Cloud Supply Chain Data Engineering INR 1500K-2200KBigQuery | Data Governance | Data Monitoring | Data Pipelines | Data QualityCollaborative team environment | Mentoring opportunitiesMid-level Full TimeBengaluru, Karnataka, India1d ago
-
Senior Staff Software Engineer, Geo INR 2500K-3465KAnomaly Detection | C++ | Continuous Learning | Data Governance | Data InfrastructureSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Application Engineer, General Software Development INR 1500K-3500KAI integration | Agentic integration | C++ | Cloud platform | Data exchangeMid-level Full TimeHyderabad, Telangana, India1d ago
-
Senior Staff Software Engineer INR 3417K-4800K.Net Core | AWS | Azure | CI/CD | Cloud NativeEnhanced medical benefits | Paid time off | Wellbeing programsSenior-level Full TimeHyderabad, India1d ago
-
Mid-level Full TimePune, Maharashtra, India1d ago
-
Mid-level Full TimePune1d ago