AI/ML Evaluation Specialist, Human Data
Tasks
- Analyze annotator behavior patterns
- Automate low judgment decisions
- Build multilingual annotation workflows
- Collaborate with cross-functional stakeholders
- Design human data collection programs
- Develop annotator calibration systems
- Enforce quality metrics and statistical process controls
- Implement data quality frameworks
- Monitor program performance
- Optimize data pipelines and workflows
Perks/Benefits
- N/A
Skills/Tech-stack
AI Safety | Data Annotation | Data Quality | Data Quality Management | Data pipeline | Human-in-the-loop | Language Processing | Language Understanding | Machine Learning | Multilingual Data | Natural Language | Natural Language Processing | Natural Language Understanding | Privacy Compliance | Process control | Python | Quality management | R | Responsible AI | SQL | Statistical Process Control | The Loop
Education
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R4d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R17d ago
-
AWS Glue | AWS IAM | AWS Kinesis | AWS Lambda | AWS Step Functions100 percent on siteSenior-level ContractMc Lean, United States5h ago
-
AI | Adversarial Machine Learning | Bot Detection | Data Analysis | Data IntegrityMid-level Full TimeMountain View, CA, USA7h ago
-
Staff Software Engineer, AI/ML, Google Cloud Security USD 207K-301KAgentic AI | Cloud platform | Data Processing | Debugging | Distributed SystemsSenior-level Full TimeReston, VA, USA; New York, NY, …7h ago
-
Senior-level Full TimeUS - Remote; Canada - Remote R14h ago
-
Senior-level Full TimeMountain View, CA16h ago
-
Staff Software Engineer, AI Integrations USD 170K-240KAgile | Alerting | Amazon Web Services | C++ | CSSSenior-level Full TimeMountain View, CA16h ago
-
Senior Principal AI Engineer USD 152K-254KData Processing | Git | Hugging Face | Language Models | Language ProcessingAdoption Assistance | Dental vision prescription coverage | Disability benefits | Employee assistance program | Health benefitsSenior-level Full TimeCambridge, United States18h ago
-
Principal / Sr Principal Data Analytics Engineer USD 98K-184KAgile | CI/CD | Data Governance | Data Modeling | Data MonitoringCompany-Paid Holidays | Disability insurance | Health insurance | Hybrid work | Life insuranceSenior-level Full TimeFLME229, United States18h ago
-
Sr. Data Engineer USD 109K-145KAWS | Agile | CI/CD | Compliance | Data Governance401k matching | Employee assistance program | Flexible working hours | Learning platform access | Medical, dental, and vision insuranceSenior-level Full TimeRemote, TX, United States R18h ago
-
Sr. Principal Embedded Software Engineer USD 142K-213KADA | Automated Regression | Bitbucket | C# | C++On-site work required | Relocation assistance not provided | Travel includedSenior-level Full TimeCACM01, United States18h ago
-
Senior Machine Learning Engineer USD 134K-168KData Drift | Databricks | Ethical AI | Feature Engineering | KubernetesSenior-level Full TimeNew York, NY, United States18h ago
-
Senior Software Engineer Remote/Travel- Active Secret -Top Secret clearance(SCI-eligibility) USD 145K-204KAPI Testing | Cypher | Data Processing | DataOps | DevOpsCompetitive benefits | Growth opportunity | Remote work | TravelSenior-level Full TimeReston, VA, United States R1d ago
-
AWS | AWS CloudFormation | Amazon ECS | Automation | CI/CD401-k plan | Company match | Employee stock purchase plan | Healthcare benefits | Life insuranceMid-level Full TimeAtlanta, GA, United States1d ago
-
Machine Learning Engineer USD 124K-170KAWS Bedrock | AWS Fargate | AWS Lambda | AWS Step Functions | Amazon ECSCareer development opportunities | Equal employment opportunityMid-level Full TimePlano, Texas, United States1d ago
-
Senior Staff Generative AI Scientist USD 185K-210KAWS | Apache Spark | Artificial Intelligence | Azure | Cloud platform401k matching | Dental insurance | Disability insurance | Health insurance | Life insuranceSenior-level Full TimeRemote, United States R1d ago
-
Machine Learning Engineer USD 117K-177KAWS | Azure | Cloud Computing | Data Visualization | Google CloudMid-level Full TimeSt. Louis, Missouri, United States1d ago
-
Software Data Engineer 4 (Enterprise Infrastructure) USD 150K-237KCloud Security | Cloud infrastructure | IT Operations | Linux | Network Automation401k matching | Career counseling | Career development | Dental insurance | Health insuranceSenior-level Full TimeLaurel, MD1d ago
-
Principal Software/Data Engineer USD 135K-258KCloud Security | Cloud infrastructure | Linux | Network Automation | OpenStack401k matching | Career development | Employee referral bonuses | Flexible work-life balance | Health, dental, and vision plansSenior-level Full TimeLaurel, MD1d ago
-
Systems Engineer (Computer Network Support - AI/ML) USD 146K-206KAI/ML | Agile | Confluence | Jira | Process Engineering401k match | Career counseling and training | Health, dental, vision coverage | Paid Holidays | Paid time offMid-level Full TimeFort Meade, MD1d ago
-
Agile Development | Artificial Intelligence | Confluence | Jira | LLMCareer development | Education & training | Health, dental & vision plans | Holidays | Paid time offSenior-level Full TimeFort Meade, MD1d ago
-
Software Engineer - X Data USD 125K-400KApache Flink | Apache Kafka | Apache Spark | BigQuery | Cause analysis401k retirement plan | Dental insurance | Discounts | Health insurance | Life insuranceMid-level Full TimePalo Alto, CA1d ago
-
Embedded Software Engineer Intern (Fall 2026) USD 108K-108KADC | Antenna Testing | C# | C++ | CANHousing stipend | Overtime pay | Paid sick time | Relocation supportEntry-level InternshipSouth San Francisco, California, USA1d ago
-
AI Engineer – Decision & Optimization Systems USD 80K-210KAMPL | C++ | CPLEX | Constraint Satisfaction | Go401k | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago