AI/ML Evaluation Specialist, Human Data
Tasks
- Analyze annotator behavior patterns
- Automate low judgment decisions
- Build multilingual annotation workflows
- Collaborate with cross-functional stakeholders
- Design human data collection programs
- Develop annotator calibration systems
- Enforce quality metrics and statistical process controls
- Implement data quality frameworks
- Monitor program performance
- Optimize data pipelines and workflows
Perks/Benefits
- N/A
Skills/Tech-stack
AI Safety | Data Annotation | Data Quality | Data Quality Management | Data pipeline | Human-in-the-loop | Language Processing | Language Understanding | Machine Learning | Multilingual Data | Natural Language | Natural Language Processing | Natural Language Understanding | Privacy Compliance | Process control | Python | Quality management | R | Responsible AI | SQL | Statistical Process Control | The Loop
Education
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R25d ago
-
Senior Robotics Software Engineer USD 162K-221K3D Geometry | APIs | C++ | CI/CD | Collision Checking401k matching | Dental insurance | Equity | Free Lunches | Free parkingSenior-level Full TimeBerkeley, California6h ago
-
PIM - Sr Databricks & Reporting Analyst USD 123K-178KAI | Apache Spark | Artificial Intelligence | Azure Data | Azure Data FactoryRelocation assistance | TeleworkSenior-level Full TimeReston, VA, US6h ago
-
AI Engineer, Senior - Enterprise Value Creation (NJUS) USD 131K-190KAWS | Azure | CI/CD | Data Engineering | Distributed Computing401k match | Dental insurance | Employee assistance program | Flexible spending account | Health savings accountSenior-level Full TimeColumbus, OH, US6h ago
-
Quality Analytics Site Lead USD 165K-218KAutomated Monitoring | Cause analysis | Dashboards | Data Modeling | Data QualityHealth insurance | On-site work | Travel up to 25 percentSenior-level Full TimeAshville, Ohio, United States9h ago
-
Senior Analytics Engineer, Product USD 155K-233KAmplitude | CI/CD | DBT | Data Governance | Data ModelingSenior-level Full TimeSan Francisco11h ago
-
Machine Learning Engineer, Platform USD 170K-220KAgentic Systems | Continuous Improvement | Deep learning | Distribution Shift | Fine TuningMid-level Full TimeSan Francisco Bay Area11h ago
-
Applied AI Architect, National Security USD 240K-270KCloud Architecture | Enterprise Integration | LLM | Machine Learning | PythonSenior-level Full TimeWashington, DC12h ago
-
Machine Learning Engineer, Assistant Quality USD 180K-205KAgent Orchestration | C++ | Evaluation | Experimentation | Go401k plan | Dental | Education stipend | Healthy lunches | Home office stipendMid-level Full TimeSan Francisco, CA13h ago
-
Machine Learning Engineer, Search Quality USD 140K-265KC++ | Experimentation | Go | Information Retrieval | JavaMid-level Full TimeMountain View, CA13h ago
-
Lead AI Engineer USD 198K-261KAgentic Frameworks | CI/CD | Cloud Computing | Containers | Fine TuningSenior-level Full TimeUSA, USA R14h ago
-
Staff AI Engineer USD 193K-235KAPI | Access Control | Audit Logging | Automated testing | CI/CD401k plan with employer match | Commuter benefit | Dental insurance | FSA program | Flexible time offSenior-level Full TimeDallas, Texas, United States14h ago
-
Staff Machine Learning Engineer USD 238K-297KDeep learning | Embedding-based retrieval | Experiment tracking | Explainable AI | Fairness401k match | Equity | Flexible PTO | Free lunch | GymSenior-level Full TimeSan Francisco, CA15h ago
-
Bash | C# | C++ | Cgroups | ChrootSenior-level Full TimeFoster City, CA15h ago
-
A/B | A/B Testing | AWS | AWS SageMaker | Apache Spark401k employer match | Commuter benefit | Dental insurance | Dependent Care Flexible Spending Account FSA | Flexible time offSenior-level Full TimeLos Angeles, California, United States; San … R15h ago
-
Agent Orchestration | Agentic AI | Artificial Intelligence | Autogen | Autonomous ReasoningSenior-level Full TimeRemote-USA R15h ago
-
Senior Data Engineer (Azure & Databricks) USD 149K-190KAWS | Apache Spark | Azure | Data Governance | Data LineageHybrid work schedule | In office 4 days per weekSenior-level Full TimeNew York, NY, United States17h ago
-
Mid-level Full TimeNew York, NY, United States17h ago
-
AI Inference Engineer USD 165K-330KAI | AI Inference | API Development | ComfyUI | Docker401k | Dental insurance | Equity | Fertility and family building stipend | Flexible PTOMid-level Full TimeSan Francisco17h ago
-
Multimodal AI Agent Algorithm Engineer USD 122K-160KAgent architecture | C++ | Deep learning | Distributed Training | Environmental simulationSenior-level Full Time圣何塞17h ago
-
Senior-level Full Time圣何塞17h ago
-
Staff Machine Learning Engineer USD 152K-261KC++ | Data Analysis | Deep learning | Distributed Systems | Experimentation401(k) plan match | Commuter benefits | Dental insurance | Electric car charging | Employee assistance programSenior-level Full TimeMountain View, USA18h ago
-
Specialist I, QC Analytics (Wed-Sat Shift) USD 86K-114KCMMS | Capillary isoelectric focusing | ERP | Flow Cytometry | GMPMid-level Full TimeWaltham, Massachusetts, United States18h ago
-
Sr Machine Learning Engineer I USD 150K-241KAutomated retraining | Bayesian Filter | CI/CD | Data Association | Extended KalmanDiscretionary paid time off | Emotional and mental wellness support | Employee resource groups | Fitness programs | Learning and development programsSenior-level Full TimeSeattle, Washington, United States R18h ago
-
Senior-level Full TimeUSA - Sandy, UT19h ago