Senior Lead Quality Engineer (Gen AI)
Tasks
- Build AI scoring rubrics and dashboards
- Build AI specific KPIs
- Conduct rate limiting abuse testing
- Create benchmark datasets
- Define AI quality engineering strategy
- Define latency benchmarks and SLAs
- Design prompt engineering strategies
- Design quality gates and release readiness
- Enable continuous AI quality monitoring
- Enforce role-based access control
- Ensure PII protection
- Ensure backward compatibility
- Establish AI quality standards
- Evaluate agent workflows
- Implement human in the loop evaluation
- Integrate AI validation into CI CD pipelines
- Lead AI risk assessment
- Monitor token usage and cost efficiency
- Perform API security validation
- Perform scalability and concurrency testing
- Test OAuth authentication flows
- Test data isolation
- Validate RAG pipelines
- Validate WCAG accessibility compliance
- Validate fine tuned models
Perks/Benefits
- N/A
Skills/Tech-stack
API Security | API Testing | API security testing | AWS | Access Control | Agent workflows | CI/CD | GenAI testing | LLM Evaluation | OAuth | PII protection | Prompt engineering | Quality Engineering | RAG | Rate Limiting | Release governance | Role-Based Access | Role-Based Access Control | Security Testing | Test automation | WCAG
Education
N/A
Roles
Engineer | Lead | Lead Quality Engineer | Quality Engineer | Senior Quality Engineer
Related jobs
-
Senior AI Engineer INR 2755K-4590KAgent Orchestration | Anthropic | Azure AI | CI/CD | Enterprise SearchAsynchronous work | Continuous learning culture | Mentorship opportunities | Remote workSenior-level Full TimeRemote, Bangalore R23h ago
-
Assistant Director, Decision Systems Data INR 2000K-4000KAgile | Apache Airflow | CI/CD | Cloud Bigtable | Cloud DataflowExecutive-level Full TimeBangalore, Karnataka, India R1d ago
-
Senior AI Engineer - Remote INR 2500K-4200KAWS CloudFormation | Apache Tomcat | Best practices | CI/CD | CircleCIMentorship | Professional development | Remote workSenior-level Full TimeBengaluru, KA, India R1d ago
-
Senior AI Engineer - Remote INR 2500K-4200KAWS | AWS DocumentDB | Apache Tomcat | CI/CD | CircleCIDistributed team culture | Remote workSenior-level Full TimePune, MH, India R1d ago
-
Senior AI Engineer - Remote INR 2500K-4200KAWS Bedrock | AWS CloudFormation | Apache | Apache Tomcat | Azure AICareer growth opportunities | Mission-driven work | Remote workSenior-level Full TimeNew Delhi, DL, India R1d ago
-
Lead Data Integration Engineer INR 3200K-4000KAirflow | Airflow DAGs | Autosys | Bitbucket | CI/CDFlexible work arrangements | Health insurance | Hybrid work model | Life insurance | Paid time offSenior-level Full TimeHyderabad, India R1d ago
-
Senior-level Full TimeIND - Remote - Bangalore Based … R1d ago
-
Senior Machine Learning Engineer - AI-Assisted Data Annotation INR 2400K-3500KConfidence scoring | Few-Shot Learning | Few-shot | Fine Tuning | Human-in-the-loopAccidental insurance | Flexible hours | Hybrid work | Life insurance | Medical insuranceSenior-level Full TimeBangalore, India (Hybrid) R1d ago
-
RAG + Agentic AI Engineer (India) (Remote) INR 2500K-3500KAccess Control | Agent Orchestration | Asynchronous programming | Auditability | BM25Growth opportunities | Ownership and autonomy | Remote workMid-level Full TimeMaharashtra, Pune, India R2d ago
-
Senior Generative AI Engineer (India) (Remote) INR 2000K-4800KAI orchestration | AI/ML stack | API Development | API Gateway | AWS AIAgile environment | Professional growth | Remote workSenior-level Full TimeMaharashtra, Pune, India R2d ago
-
Data Engineer/Senior Consultant Specialist INR 2400K-3500KAccess Control | Apache Spark | Azure | Azure Data | Azure Data LakeFlexible working | Inclusive work environment | Professional development opportunitiesSenior-level Full TimeHyderabad, Telangana, India R2d ago
-
AWS | BigQuery | ClickHouse | Cloud infrastructure | Data EngineeringFully remote | Inclusive organization | Remote-first cultureMid-level Full TimeIndia R2d ago
-
AWS | Asynchronous processing | Authentication | CI/CD | CeleryCareer growth opportunities | Collaborative product driven environment | Flexible work environment | Fully remote opportunities | Technical ownershipSenior-level Full TimeIndia R2d ago
-
AWS | Azure | Cloud platform | Deep learning | Generative AIContinuous learning and growth | Fully remote work | Technical excellence and experimentation cultureSenior-level Full TimeIndia R2d ago
-
Senior-level Full TimeAnywhere in India R2d ago
-
Senior Machine Learning Engineer INR 2500K-4800KAPI Development | AWS | Amazon Bedrock | Amazon EKS | Amazon SageMakerSenior-level Full TimeBangalore,Karnataka,India R2d ago
-
Staff Engineer - Devops INR 3000K-4000KAnsible | Argo CD | ArgoCD | As-a-Service | BashOn-call rotationSenior-level Full TimePune, MH, India R2d ago
-
Computer Scientist ( Java Backend ) INR 2500K-3500KApache Kafka | CI/CD | Docker | Event Driven | Event-driven architectureSenior-level Full TimeBangalore, India R2d ago
-
Computer Scientist 2 ( Data Engineering + AI ) INR 2475K-3465KAWS | Apache Spark | Azure | CI/CD | DatabricksSenior-level Full TimeNoida, India R2d ago
-
Computer Scientist - II ( Data Engineering ) INR 1500K-2000KAWS | Apache Spark | Azure | CI/CD | Data ModelingMid-level Full TimeNoida, India R2d ago
-
Engineer III - Data Platform (Hybrid, IND) INR 2500K-3500KAWS | Airflow | Azure | CI/CD | ContainerizationCompetitive vacation and holidays | Comprehensive wellness programs | Employee networks | Paid parental and adoption leave | Paid time offSenior-level Full TimeBangalore, India R2d ago
-
Principal Data Science Engineer INR 2500K-3500KArtifacts registry | Artificial neural networks | Autoscaling | BigQuery | C++Senior-level Full TimeIndia Remote R2d ago
-
Lead AIML INR 4000K-4500KAWS | AWS Lambda | Amazon Comprehend | Amazon EKS | Amazon LexRemote workSenior-level Full TimeHyderabad, Telangana, India - Remote R2d ago
-
Senior-level Full TimeNoida, India R2d ago
-
Computer Scientist-II (Fullstack) INR 3150K-4500KAccessibility | Adobe Analytics | Adobe Experience | Adobe Experience Manager | AngularSenior-level Full TimeNoida, India R2d ago