Test Lead - API Automation
Tasks
- Collaborate with Agile teams and cross functional AI teams
- Design test cases for non deterministic AI systems
- Design test strategies for Generative AI systems
- Execute automated testing for API services
- Implement prompt engineering and optimization
- Perform exploratory negative and adversarial testing
- Test AI model endpoints
- Validate accuracy relevance consistency and factuality
- Validate data quality using statistics and AI metrics
- Validate fairness transparency and responsible AI requirements
Perks/Benefits
- N/A
Skills/Tech-stack
AI Performance Metrics | AI performance | API Testing | AWS Bedrock | Adversarial Testing | Azure OpenAI | Bias detection | Data Quality | Exploratory Testing | Factual accuracy | Generative AI | Google Vertex | Google Vertex AI | Hallucination detection | JSON | Language Models | Language Processing | Large Language Models | Machine Learning | Machine learning lifecycle | Model versioning | NLP | Natural Language | Natural Language Processing | Negative Testing | Output Consistency | Performance Metrics | Prompt engineering | Python | RAG | REST | Regression testing | Retrieval-Augmented Generation | Statistics | Test automation | Vertex AI
Education
N/A
Roles
Engineer | Lead | QA Engineer | Software Test Engineer | Test Engineer | Test Lead
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R26d ago
-
Principal AI Software Engineering Lead / IT Architect 3 USD 140K-213KAPI | AWS | Agentic development | Agile | Artificial IntelligenceHybrid onsite 3 days per weekSenior-level ContractIrving, United States2h ago
-
Mid-level Full TimeUS-Kansas-Wichita4h ago
-
Gen AI Developer Specialist USD 118K-204KAI Evaluation | AWS | AWS Bedrock | Access Management | Cause analysisSenior-level Full TimeUnited States4h ago
-
Data Engineer (Multiple Positions) USD 148K-300KBig Data | Data Modeling | Data Quality | Data Reliability | ETLEntry-level Full TimeSeattle, Washington, United States4h ago
-
Mid-level Full TimeUS-Kansas-Wichita4h ago
-
AI infrastructure | Academic research | Cloud infrastructure | Distributed Systems | Machine LearningEntry-level Full TimeSeattle, Washington, United States4h ago
-
Academic research | Artificial Intelligence | Cloud infrastructure | Machine Learning | Open SourceEntry-level Full TimeSan Jose, California, United States4h ago
-
Data cleaning | Data collection | Deep learning | Machine Learning | Model EvaluationSenior-level Full TimeSan Jose, California, United States4h ago
-
3D Generation | AIGC | Agentic Models | Computer Vision | Fine TuningEntry-level Full TimeSan Jose, California, United States4h ago
-
AI | CPU | GPU | Kubernetes | Machine LearningSenior-level Full TimeSan Jose, California, United States4h ago
-
Containerd | Docker | Go | Java | KubernetesSenior-level Full TimeSeattle, Washington, United States4h ago
-
AI | AI workloads | C++ | CPU | Distributed SystemsSenior-level Full TimeSeattle, Washington, United States4h ago
-
Research Scientist - LLM Training System as a Service - Global Frontier Tech Recruitment Program - 2027 Start (PhD) USD 212K-450KCUDA | Deep learning | Distributed Training | GPU Performance | Language ModelsEntry-level Full TimeSan Jose, California, United States4h ago
-
AI Engineer Graduate (Client Arch) - 2027 Start USD 100K-118KArtificial Intelligence | Automation | CI/CD | Code Refactoring | Continuous DeliveryEntry-level Full TimeSan Jose, California, United States5h ago
-
Algorithms | C++ | Data Mining | Data Pipelines | Data StructuresDevelopment workshops | Learning programs | Social eventsEntry-level InternshipSan Jose, California, United States5h ago
-
Data Engineer Graduate (Monetization Data) - 2027 Start USD 115K-177KAmazon Redshift | Apache Airflow | Apache Flink | Apache Hive | Apache KafkaEntry-level Full TimeSan Jose, California, United States5h ago
-
Machine Learning Engineer Graduate (TikTok-Data-Search-Recommendation-Global E-Commerce) - 2027 Start USD 116K-177KC++ | Computer Vision | Data Mining | Data Pipelines | DatabasesEntry-level Full TimeSeattle, Washington, United States5h ago
-
Machine Learning Engineer, AI Agent USD 148K-180KAI Agents | Agent systems | C++ | Context engineering | Deep learningMid-level Full TimeSan Jose, California, United States5h ago
-
Advertising Systems | Algorithms | C# | C++ | Data StructuresEntry-level Full TimeSan Jose, California, United States5h ago
-
Machine Learning/Research Engineer Graduate (Monetization Technology-Ads Core Global) - 2027 Start (PhD) USD 113K-250KAlgorithms | C++ | Computer Vision | Counterfactual Optimization | Data StructuresEntry-level Full TimeSan Jose, California, United States5h ago
-
Agent Registry | Agentic AI | C++ | Computer Vision | Feature StoreEntry-level Full TimeSan Jose, California, United States5h ago
-
ML Infra Engineer Intern (Ads Infra) - 2027 Summer USD 113K-250KAutomated Operations | C++ | Cloud Native | Data Pipelines | Data QualityEntry-level InternshipSan Jose, California, United States5h ago
-
(General Hire) Machine Learning Engineer Graduate (Performance Monetization) - 2027 Start (PhD) USD 118K-250KC++ | Debugging | Go | MXNet | Machine LearningEntry-level Full TimeSan Jose, California, United States5h ago
-
(General Hire) Machine Learning Engineer Graduate (Performance Monetization) - 2027 Start USD 113K-250KC# | C++ | Debugging | Go | MXNetEntry-level Full TimeSan Jose, California, United States5h ago