AI Validation Engineer
Auburn Hills, MI, United States
USD 140K-220K (estimate) Mid-level Full Time
Tasks
- Analyze test results to identify failure patterns and root causes
- Build evaluation datasets and benchmarks
- Create AI assisted test generation tools
- Design validation frameworks for deep learning models
- Develop automated test pipelines for LLM features
- Develop model monitoring and drift detection systems
- Implement automated regression testing for ML model updates
- Integrate AI model validation into test bench and CI CD pipelines
Perks/Benefits
Skills/Tech-stack
Adversarial Input | Adversarial Input Testing | BLEU | Bias Fairness Analysis | Bias/fairness | CI/CD | Dataset curation | Deep learning | Drift Detection | Evaluation metrics | Fairness analysis | GitHub Actions | GitLab CI | Hallucination detection | Human-in-the-loop | Jenkins | Kubeflow | LLM-as-judge | Language Models | Language Processing | Large Language Models | MLflow | Machine Learning | Model Monitoring | Natural Language | Natural Language Processing | Out of Distribution | Out-of-distribution Detection | Prompt Regression | Prompt Regression Testing | Pytest | Python | ROUGE | Regression testing | Robot Framework | Statistical Analysis | The Loop | Uncertainty Quantification | Weights and Biases
Education
Language: en
Views: 0
Clicks: 0
Saves: 0
Related jobs
-
Senior Machine Learning Engineer USD 150K-225KAIOps | AWS | Azure | Data Analysis | Data Visualization401k match | Career growth opportunities | Dental insurance | Employee resource groups | Flexible paid time offSenior-level Full TimeRemote, United States2h ago
-
Senior Data Engineer USD 120K-175K.Net Core | .Net Framework | Apache Airflow | Azure | Azure Data401k match | Career growth | Dental insurance | Employee resource groups | Life insuranceSenior-level Full TimeRemote, United States2h ago
-
Lead AI Engineer-Machine Learning USD 110K-198KAWS | Agile | Azure | CI/CD | Cloud platform401k match | Dental insurance | Disability insurance | Growth opportunities | Holiday paySenior-level Full TimeParsippany, NJ, United States2h ago
-
Lead AI Engineer USD 180K-280KAWS SageMaker | Azure Machine Learning | CI/CD | ColBERT | DockerChildcare benefits | Onsite gym | Tuition assistanceSenior-level Full TimeQuincy, MA, United States3h ago
-
Senior Cloud Data Ops Engineer USD 140K-180KAWS | Amazon Aurora | Amazon RDS | Bash | BigQuery401k match | Bonus eligibility | Childcare benefits | Gym access | Insurance coverageSenior-level Full TimeQuincy, MA, United States3h ago
-
Senior-level ContractBeaverton, OR, US4h ago
-
Data Engineer USD 90K-130KBusiness Objects | Change Management | Data Governance | Data Lineage | Data ModelingMid-level Full TimeNashville, TN, United States4h ago
-
API | Agile Software Development | Agile software | Apache Kafka | Application developmentSenior-level Full TimeDearborn, United States5h ago
-
Data Engineer CAD 100K-170KAWS | Apache Airflow | Azure | BigQuery | Cloud Computing401k | Asynchronous work culture | Autonomy | Company paid life insurance | Disability coverageMid-level Full TimeRemote - US or Canada R5h ago
-
Senior Data Engineer ID71671 USD 155K-170KAWS Kinesis | Apache Airflow | Apache Kafka | Automated testing | Batch ProcessingAnnual learning budget | Flexible hours | Learning budget | Mentorship | Mentorship supportSenior-level Full TimeNew York, United States5h ago
-
Senior Data Engineer ID71671 USD 155K-170KAWS Kinesis | Apache Airflow | Automated testing | CI/CD | DBTCollaborative culture | Flexible hours | Internal tech talks | Learning budget | MentorshipSenior-level Full TimeRichmond, United States5h ago
-
Senior Data Engineer ID71671 USD 155K-170KAWS Kinesis | Apache Airflow | Apache Kafka | Automated testing | CI/CDAnnual learning budget | Flexible hours | Internal tech talks | Local well being programs | MentorshipSenior-level Full TimeMiami, United States5h ago
-
Senior Data Engineer ID71671 USD 155K-170KApache Airflow | Automated testing | CI/CD | Dagster | Data LineageAnnual learning budget | Collaborative culture | Flexible hours | Mentorship | People focused supportSenior-level Full TimeOrlando, United States5h ago
-
Senior Data Engineer ID71671 USD 155K-170KAWS Kinesis | Apache Airflow | Apache Kafka | Automated testing | CI/CDAnnual learning budget | Flexible hours | Mentorship | Performance reviews | Remote workSenior-level Full TimeJacksonville, United States5h ago
-
Senior Data Engineer ID71671 USD 155K-170KAWS | AWS Kinesis | Apache Airflow | Apache Kafka | AzureAnnual learning budget | Flexible hours | Health and wellbeing programs | Learning budget | MentorshipSenior-level Full TimeWest Palm Beach, United States5h ago
-
Senior Data Engineer ID71671 USD 155K-170KApache Airflow | CI/CD | DBT | Dagster | Data LineageFlexible hours | Internal tech talks | Learning budget | Local support programs | MentorshipSenior-level Full TimeDallas, United States5h ago
-
Senior Data Engineer ID71671 USD 155K-170KAWS Kinesis | Apache Airflow | Apache Kafka | Automated testing | Batch ProcessingAnnual learning budget | Collaborative culture | Flexible hours | Mentorship | People focused supportSenior-level Full TimeAtlanta, United States5h ago
-
Senior Data Engineer ID71671 USD 155K-170KAWS Kinesis | Amazon Web Services | Apache Airflow | Apache Kafka | Automated testingAnnual learning budget | Flexible hours | Mentorship | Remote work | TechtalksSenior-level Full TimeSan Francisco, United States5h ago
-
Senior Data Engineer ID71671 USD 155K-170KAWS Kinesis | Apache Airflow | Apache Kafka | Automated testing | Batch ProcessingFlexible hours | Internal tech talks | Learning budget | Mentorship | People focused supportSenior-level Full TimeChicago, United States5h ago
-
Senior Data Engineer ID71671 USD 155K-170KAmazon Kinesis | Apache Airflow | Apache Kafka | CI/CD | DBTFlexible hours | Learning budget | Mentorship | Remote work | Well-being programsSenior-level Full TimeTallahassee, United States5h ago
-
Senior Data Engineer ID71671 USD 155K-170KAWS Kinesis | Apache Airflow | Automated testing | CI/CD | DBTFlexible hours | Learning budget | Mentorship | Performance reviews | Remote workSenior-level Full TimeBoston, United States5h ago
-
Senior Data Engineer ID71671 USD 155K-170KAWS Kinesis | Apache Airflow | Apache Kafka | CI/CD | DBTAnnual learning budget | Flexible hours | Mentorship | Performance and compensation reviews | Remote workSenior-level Full TimeIrving, United States5h ago
-
Senior Data Engineer ID71671 USD 155K-170KApache Airflow | Automated testing | Batch Processing | CI/CD | DBTAnnual learning budget | Flexible hours | Mentorship | Performance reviews | Remote workSenior-level Full TimeTexas City, United States5h ago
-
Senior Data Engineer ID71671 USD 155K-170KAWS Kinesis | Apache Airflow | Apache Kafka | Automated testing | CI/CDFlexible hours | Health and wellness programs | Learning budget | Mentorship | Remote workSenior-level Full TimeJersey City, United States5h ago
-
SOFTWARE ENGINEER – Algorithm Development - 7+ yrs of Experience - TS/SCI w/Poly clearance is required- RC - 306/309 A USD 181K-192KAlgorithm Development | C++ | Data queries | Database interfaces | Documentation401k retirement plan | Dental insurance | Floating holidays | Holidays | Life insuranceSenior-level Full TimeColumbia, United States6h ago