AI Test Engineer
Baltimore, MD, United States
A USD 98K-150K (estimate) Mid-level Full Time
Tasks
- Build automated testing frameworks for AI applications
- Define and validate AI guardrails and fallback logic
- Design AI testing strategies
- Detect hallucinations bias and model drift risks
- Develop test cases for edge cases and prompt behavior
- Document testing results and communicate findings
- Execute manual and automated AI output validation
- Identify workflow failures and performance bottlenecks
- Integrate automated tests into CI/CD pipelines
- Measure model performance against success criteria
- Perform data validation across ingestion and processing pipelines
- Support root cause analysis for AI production issues
- Test AI agent workflows and orchestration logic
- Validate AI agent integrations with APIs and data services
- Validate training validation and test dataset quality
Perks/Benefits
Skills/Tech-stack
API Testing | AWS | Agile | Bias detection | CI/CD | Case Development | Cause analysis | Data Quality | Data Validation | Databricks | Distributed Systems | Integration Testing | Language Models | Large Language Models | Machine Learning | Model Drift | Model Evaluation | Prompt engineering | Regression testing | Root Cause Analysis | Root cause | SQL | Scenario testing | Test Case | Test Planning | Test automation | Test case development
Education
Bachelor of Arts | Bachelor of Engineering | Bachelor of Science
Roles
AI | AI Test Engineer | Engineer | Machine Learning Test Engineer | Test Engineer
Related jobs
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R10d ago
-
Apache Spark | Dask | Data Augmentation | Data Generation | Deep learningCollaborative environment | Cutting-edge infrastructure | Health coverage | Professional growth | Retirement plansMid-level Full TimeRedmond, United States3h ago
-
Machine Learning Engineer, Ecommerce Risk Control USD 175K-266KData Analysis | Data Visualization | Experiment design | Machine Learning | Model DeploymentSenior-level Full TimeSan Jose, California, United States5h ago
-
Brand recognition | Computer Vision | Data Mining | Deep learning | Entity MatchingMid-level Full TimeSeattle, Washington, United States5h ago
-
Click Through Rate | Click Through Rate Prediction | Conversion Rate | Conversion Rate Prediction | Data AnalysisSenior-level Full TimeSeattle, Washington, United States5h ago
-
Computer Vision | Fraud Detection | Language Processing | Machine Learning | Natural LanguageSenior-level Full TimeSeattle, Washington, United States5h ago
-
A/B | A/B Testing | B testing | Data Mining | Distributed ComputingMid-level Full TimeSeattle, Washington, United States5h ago
-
Machine Learning Engineer, Pricing - TikTok E-commerce Global Supply Chain and Logistics USD 129K-194KCausal Inference | Data Mining | Forecasting | Graph Databases | Knowledge graphsMid-level Full TimeSeattle, Washington, United States5h ago
-
Alignment | Chain-of-Thought | Computer Vision | Data Mining | Fine TuningSenior-level Full TimeSeattle, Washington, United States5h ago
-
Machine Learning Engineer, Recommendation - E-Commerce USD 145K-337KClick Through Rate | Click Through Rate Modeling | Computer Vision | Conversion Rate | Conversion rate modelingSenior-level Full TimeSan Jose, California, United States5h ago
-
Machine Learning Engineer, Ecommerce Risk Control USD 175K-266KA/B | A/B Testing | B testing | Data Visualization | Deep learningSenior-level Full TimeSeattle, Washington, United States5h ago
-
Mid-level Full TimeSan Jose, California, United States5h ago
-
Brand recognition | Chain-of-Thought | Computer Vision | Data Mining | Feature extractionSenior-level Full TimeSeattle, Washington, United States5h ago
-
Alignment | Audio Processing | COT | Clustering | Computer VisionSenior-level Full TimeSan Jose, California, United States5h ago
-
Benchmarking | Causal Inference | Data Mining | Data Pipelines | Data PreprocessingMid-level Full TimeSeattle, Washington, United States5h ago
-
Big Data | Click Through Rate | Click Through Rate Optimization | Conversion Rate | Conversion rate optimizationSenior-level Full TimeSeattle, Washington, United States5h ago
-
Machine Learning Engineer - Marketplace Growth (TikTok Global E-commerce Supply Chain and Logistics) USD 136K-280KBenchmarking | Causal Inference | Click Through Rate | Click Through Rate Optimization | Customer SegmentationEntry-level Full TimeSan Jose, California, United States5h ago
-
A/B | A/B Testing | B testing | Continual Learning | Data SetSenior-level Full TimeSeattle, Washington, United States5h ago
-
Benchmarking | Causal Inference | Data Mining | Data Pipelines | Data PreprocessingMid-level Full TimeSan Jose, California, United States5h ago
-
A/B | A/B Testing | B testing | Continual Learning | Dataset ConstructionSenior-level Full TimeSan Jose, California, United States5h ago
-
Senior Machine Learning Engineer - TikTok E-commerce Global Supply Chain and Logistics USD 136K-205KData Mining | Deep learning | Knowledge graphs | Machine Learning | Operations ResearchSenior-level Full TimeSeattle, Washington, United States5h ago
-
Machine Learning Engineer, Ecommerce Risk Control USD 175K-266KData Analysis | Data Pipelines | Data Visualization | Deep learning | Experiment designSenior-level Full TimeSeattle, Washington, United States5h ago
-
Applied Scientist, Search, E-Commerce Alliance USD 136K-205KData Analysis | Language Processing | Machine Learning | Model Deployment | Natural LanguageMid-level Full TimeSan Jose, California, United States5h ago
-
AI Agents | Causal Inference | Continual pretraining | Deep learning | Fine TuningEntry-level Full TimeSeattle, Washington, United States5h ago
-
Data Analysis | Large Scale Data | Large-scale | Machine Learning | RankingEntry-level Full TimeSan Jose, California, United States5h ago