Senior Manager, Machine Learning (Data Operations)
Tasks
- Collaborate with ML and product teams on labeling tasks
- Correct mislabeled data
- Define annotation guidelines and taxonomies
- Design labeling workflows
- Establish gold standard datasets
- Identify active learning opportunities
- Manage Label Studio projects and integrations
- Manage outsourced labeling vendors
- Measure IAA and cost per label
- Monitor inter annotator agreement
- Run audit sampling for labeled data
- Run calibration sessions with vendors
- Set labeling quality frameworks
- Set quality SLAs for vendors
- Track label accuracy metrics
Perks/Benefits
Skills/Tech-stack
Annotation Guidelines | Calibration | Data Operations | Data labeling | Document extraction | Entity recognition | Error Taxonomy | Gold Standard | Gold Standard Datasets | Intent detection | Inter-annotator agreement | Label Studio | Machine Learning | Named Entity Recognition | Python | SQL | Text Classification
Education
N/A
Related jobs
-
Product Manager, Research USD 190K-260KData Annotation | Data Pipelines | Data Quality | Documentation | Evaluation401k access | Company-sponsored fitness classes | Dental coverage | FSA | HSAMid-level Full TimeSan Francisco1d ago
-
AI | Audio/Video | Audio/Video Processing | Backend Development | Computer VisionSenior-level Full TimeSan Jose, California, United States1d ago
-
C++ | Cloud Computing | Computer Vision | Data Processing | Data StructuresSenior-level Full TimeMountain View, CA, USA1d ago
-
Senior Manager, AI Software Engineering USD 199K-235KAWS | Agile | Automated testing | CI/CD | Cloud Native401k match | Flexible time off | Hybrid work model | Medical insuranceSenior-level Full TimeNew York, NY R1d ago
-
Data Operations Manager, Human Data USD 270K-365KData Analysis | Data pipeline | Data strategy | Project Management | PythonFlexible working hours | Generous vacation | Hybrid office policy | Optional equity donation matching | Parental leaveMid-level Full TimeSan Francisco, CA | New York … R1d ago
-
Sr Manager Analytics Engineer USD 157K-235KAirflow | Anomaly Detection | Apache Spark | Data Architecture | Data Compliance401k plan | Bonus eligible | Disability benefits | Life insurance | Medical/Dental/Vision insuranceSenior-level Full TimeWest Hollywood, CA, US, 900691d ago
-
Sr. Manager, People Analytics USD 119K-172KData Integrity | Data Management | Hypothesis Testing | Machine Learning | Microsoft Excel401k match | Backup child and elder care | Dental insurance | EAP counseling | Headspace accessSenior-level Full Time09510 - Washington Remote, United States R1d ago
-
Manager, Strategic Analytics USD 180K-220KBusiness Analytics | Dashboards | Data Analysis | Data Quality | Data metricsMid-level Full TimeRemote-Washington, DC, United States R1d ago
-
Manager, Data Science - Pricing Algorithms USD 120K-192KData Analysis | Data Science | Experiment design | Feature Engineering | Machine LearningBereavement leave | Jury duty leave | Paid Holidays | Paid sick leave | Paid time offMid-level Full Time8901 - Corp Office West Crk, …1d ago
-
Manager, Retail Strategy & Analytics USD 102K-163KA/B | A/B Testing | Analytics | B testing | DashboardsFlexible time off | Paid Holidays | Paid sick time | Paid vacation timeMid-level Full Time8116 - Midtown Office, United States1d ago
-
Vice President, Data & Analytics Engineering USD 255K-320KAPI Management | Access Control | BigQuery | Business Intelligence | ComplianceEmployee discount | Financial security benefits | Free executive coaching on demand | Health and wellness benefits | Paid volunteer daysExecutive-level Full TimeJersey City, NJ, United States1d ago
-
Head of BP&AD Expense Management and Analytics USD 128K-168KAdvanced Analytics | Alation | Automation | Business Intelligence | Data GovernanceEmployee networks and communities | Learning and development programsSenior-level Full TimeSpringfield, Massachusetts; Boston, Massachusetts1d ago
-
Senior Manager, People Analytics, Insights and AI USD 154K-246KChange Management | Classification | Clustering | Compensation | DAX401k plan | Annual bonus | Anxiety Management Program | Commuter benefits | Dental insuranceSenior-level Full TimeLocation Franklin Lakes, New Jersey1d ago
-
Senior Data Analyst (Hybrid) USD 74K-124KAccess Management | Application Security | Dashboards | Data Lake | Data MappingHybrid workSenior-level Full TimeBoston - 20 Overland Street, United …1d ago
-
Director Analytics & Reporting USD 165K-213KAI-assisted analytics | Care Gap Analysis | Claims data | Clinical data | Data GovernancePeriodic travel to office | Remote workExecutive-level Full TimeRemote R1d ago
-
Senior Engineering Manager, ML Platform USD 240K-340KAWS | Databricks | Docker | Feature Engineering | Fraud Detection401k plan | Education reimbursement | Flexible time off | Medical/Dental/Vision | Wellness reimbursementSenior-level Full TimeRemote - USA R1d ago
-
Analytics | Artificial Intelligence | Data Visualization | Machine Learning | SQL401k matching | Commuter benefits match | Hybrid work | Medical, dental, vision benefits | Mental health programMid-level Full TimeSan Francisco, CA1d ago
-
Engineering Manager – Applied AI USD 220K-300KAI Agents | LLM Applications | Language Models | Large Language Models | Python401k retirement plan | Dental insurance | Medical insurance | Vision insuranceMid-level Full TimeLos Angeles, CA1d ago
-
Manager, Delivery Solutions Architects - Public Sector USD 197K-270KArchitecture | Artificial Intelligence | Data Engineering | Data Science | Data WarehousingSenior-level Full TimeMaryland; Remote - Virginia; Remote - … R1d ago
-
Manager, Customer Relationship Management Analytics USD 105K-124KCampaign Execution | Campaign Execution Tools | Database Marketing | Pega | Portfolio ManagementEmployee resource groups | Matching gift program | Professional development opportunities | Sustainability initiatives | Volunteerism programMid-level Full TimeBuffalo, New York, United States R1d ago
-
Manager, Data Platform Engineering USD 163K-244KAWS | Access Management | Alerting | Audit Logging | CI/CDMid-level Full TimeBoston, Massachusetts, United States1d ago
-
Senior AI Program Manager USD 146K-195KAcademic collaboration | Artificial Intelligence | Biology | Computational Biology | Computational ScienceCampus perks | Commuter benefits | Discounts | Family and caregiver support | Medical, dental, and vision insuranceSenior-level Full TimeBoston, MA, United States1d ago
-
Manager, Applied AI Engineering (Digital Natives) USD 251K-335KAPI Integration | Application Architecture | Cloud infrastructure | Deployment engineering | Generative AIHybrid work model | Relocation assistanceSenior-level Full TimeSan Francisco R1d ago
-
Data Analytics Manager USD 145K-210KBusiness Intelligence | Cloud Data | Cloud data warehousing | DBT | DashboardingRemote work opportunityMid-level Full TimeRemote - US R1d ago
-
Sr. Manager, Financial Services & Analytics USD 94K-140KA/B | A/B Testing | Authorization Rate | Authorization Rate Analysis | B testingSenior-level Full TimeAtlanta, GA1d ago