AI Test Engineer - VLabs
Bengaluru, Karnataka, India
INR 2500K-4800K (estimate) Senior-level Full Time
Tasks
- Build Python evaluation frameworks
- Build and maintain ground truth datasets
- Continuously update datasets for business requirements
- Define classification and extraction accuracy standards
- Design adversarial and boundary test inputs
- Design and operate drift detection frameworks
- Embed evaluation into CI CD pipelines
- Establish performance degradation thresholds
- Execute API testing for AI pipeline endpoints
- Implement AI as judge evaluation
- Perform end to end workflow and agent testing
- Support responsible AI governance requirements
- Translate AI testing strategy into test scenarios
- Translate evaluation results into actionable insights
- Validate LLM and document model outputs
- Validate data persistence with SQL
Perks/Benefits
Skills/Tech-stack
AI Foundry | AI as Judge | API Testing | AWS Bedrock | Adversarial Testing | Azure AI | Azure AI Foundry | CI/CD | Data Validation | DeepEval | Document Classification | Drift Detection | HTTPx | Hallucination detection | Information Extraction | LLM Evaluation | Langchain | Llamaindex | Newman | OCR | Postman | Prompt design | Prompt engineering | Promptfoo | Python | Ragas | Requests | SQL | Statistical Analysis | VLM
Education
Roles
AI | AI Test Engineer | Engineer | Software Test Engineer | Test Engineer
Related jobs
-
Software Engineer - AI/ML INR 2500K-3500KAI Search | AWS | Agent systems | Agentic AI | AutogenAccess to cutting-edge technologies | Collaborative work environment | Professional growth and development | Training and developmentSenior-level Full TimeBangalore - Karnataka, India - Kalyani …3h ago
-
Senior Data Engineer (Python, SQL, Snowflake, Devops, Azure/ AWS, Co-pilot, Kubernetes, Docker) INR 2520K-3500KADF | AWS | Airflow | Automated Data Profiling | AzureEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India5h ago
-
AI Solutions Consultant INR 1500K-3000KAI Agent | AI Agent Development | Agent Development | Agent systems | Application developmentMid-level Full TimeBengaluru, Karnataka, India; Mumbai, Maharashtra, India5h ago
-
Software Engineer (AI/ML) INR 500KAWS | Attention Mechanisms | Azure | Cloud platform | Deployment PipelinesBirthday day off | Career advancement | Casual leave | Employee assistance program | Learning and development opportunitiesMid-level Full TimeBengaluru, Onsite7h ago
-
Technical Lead INR 2000K-5000KAzure Machine Learning | CI/CD | Copilot Studio | Data Pipelines | DjangoSenior-level Full TimeChennai, Tamil Nadu, India7h ago
-
Mid-level Full TimeKarnataka, India7h ago
-
Senior-level Full TimeHyderabad, TS, India8h ago
-
Sr AI Engineer (Generative AI & Pharmacovigilance) INR 3264K-5000KAWS | Agentic AI | Azure OpenAI | CI/CD | CrewAISenior-level Full TimePune, MH, India8h ago
-
Backend Engineer (Cloud & AI Integration) INR 1500K-3000KAWS | Git | NoSQL | Production Support | PythonMid-level Full TimeBengaluru, KA, India8h ago
-
Senior Software Engineer - Full Stack & GenAI INR 3242K-4680KAPI Security | AWS | Amazon RDS | Amazon S3 | AngularHybrid work | Professional developmentSenior-level Full TimeBengaluru, KA, India R8h ago
-
Principal Member Technical Staff - (MTS 4) - Machine learning INR 3200K-4580KAWS | Anomaly Detection | Azure | Cassandra | Cosmos DBSenior-level Full TimeBengaluru, India8h ago
-
DevOps Track Sr.Engineer INR 1500K-3500KAccess Management | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksSenior-level Full TimeIndia9h ago
-
Senior Data Engineer INR 1500K-2600KDBT | Data Marts | Data Modeling | Data Quality | Data ReconciliationSenior-level Full TimeBangalore, KA, IN, 56007610h ago
-
Senior Data Engineer INR 3000K-4000KAmazon Web Services | Cloud platform | Continuous Delivery | Data Engineering | Data GovernanceFlexible work arrangements | Learning and development programs | MentorshipSenior-level Full TimeBangalore, India; Chennai, India; Hyderabad, India; …10h ago
-
Lead, AI Engineer INR 2500K-5000KAWS Bedrock | Agentic Workflows | Anthropic Claude | Automated Regression | Automated regression testingCareer growth | Company-sponsored certifications | Mentorship | Recognition rewards | SnacksSenior-level Full TimeHyderabad, TS, India10h ago
-
Senior Software Engineer, Machine Learning INR 3600K-5400KA/B | A/B Testing | Amazon S3 | Apache Airflow | B testingDental insurance | Financial wellness support | Health insurance | Mental health support | Paid time offSenior-level Full TimeBengaluru, India R10h ago
-
Senior-level Full TimeBangalore, Karnataka, India10h ago
-
Senior Staff Engineer INR 2500K-4000KAlgorithms | Amazon Web Services | Big Data | Cloud Computing | Code reviewCustomer-focused culture | Flexible work arrangementsSenior-level Full TimeBangalore11h ago
-
Senior Software Engineer(AI/ML), Trust INR 4500K-6500KA/B | A/B Testing | Anomaly Detection | Apache Airflow | Apache KafkaSenior-level Full TimeBangalore, India11h ago
-
Senior-level Full TimeGurugram, India11h ago
-
Apache Airflow | Apache Spark | Batch Processing | Cloud infrastructure | Data GovernanceHigh impact ownership | Professional growth opportunities | Remote first working environment | Technical leadership opportunitiesSenior-level Contract Full TimeIndia12h ago
-
AI Agents | AWS Bedrock | AWS Lambda | Agent Orchestration | Agent systemsContinuous learning | Fixed working hours | Fully remote | Knowledge sharing | Laptop providedMid-level Full TimeIndia R12h ago
-
ABAP | BDC | Business Intelligence | Data Modeling | Data PipelinesCareer development | Employee communities | Global onboarding | Health insurance | Hybrid workSenior-level Full TimeIndia R12h ago
-
API Versioning | Asynchronous processing | Authentication | CI/CD | CeleryCollaborate with backend and frontend teams | Full-time employment | Fully remote | Hands on experience with production LLM and VLM systems | Work on AI infrastructure model routing and microservicesMid-level Full TimeIndia R13h ago
-
ACID transactions | Apache Spark | Azure Data | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeIndia14h ago