Senior Manager, Machine Learning (Data Operations)
Tasks
- Collaborate with ML and product teams on labeling tasks
- Correct mislabeled data
- Define annotation guidelines and taxonomies
- Design labeling workflows
- Establish gold standard datasets
- Identify active learning opportunities
- Manage Label Studio projects and integrations
- Manage outsourced labeling vendors
- Measure IAA and cost per label
- Monitor inter annotator agreement
- Run audit sampling for labeled data
- Run calibration sessions with vendors
- Set labeling quality frameworks
- Set quality SLAs for vendors
- Track label accuracy metrics
Perks/Benefits
Skills/Tech-stack
Annotation Guidelines | Calibration | Data Operations | Data labeling | Document extraction | Entity recognition | Error Taxonomy | Gold Standard | Gold Standard Datasets | Intent detection | Inter-annotator agreement | Label Studio | Machine Learning | Named Entity Recognition | Python | SQL | Text Classification
Education
N/A
Related jobs
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R21d ago
-
Manager, IT Engineering USD 98K-134KAgile | Azure | Cloud Computing | Data Engineering | Data Lake401k program | Employee assistance program | Flexible spending accounts | Health benefits | Pet insuranceMid-level Full TimeFort Worth, TX, US4h ago
-
Mid-level Full TimeMcCarran, NV6h ago
-
Engineering Manager, Enterprise Applied AI USD 300K-520KAPIs | Agent Orchestration | Benchmarking | Cloud Native | Data SourcesDental insurance | Equity grant | Free gym membership | Health insurance | Laundry reimbursementMid-level Full TimeSan Francisco10h ago
-
Manager of Data Engineering USD 200K-230KAWS | Amazon Databricks | Amazon EMR | Amazon Kinesis | Amazon RedshiftSenior-level Full TimeBeverly Hills, California, United States12h ago
-
Engineering Manager (TLM, Agents) USD 300K-405KAI Evaluation | Artificial Intelligence | Browser technologies | CDP | Code QualityMid-level Full TimeSan Francisco13h ago
-
Associate Manager, Revenue Analytics USD 93K-172KExcel | Financial Modeling | Rate analysis | Revenue Forecasting | SQL401k matching | Dental insurance | Employee stock purchase plan | Health insurance | Paid time offMid-level Full TimeNew York; San Francisco17h ago
-
Marketing Analytics and Digital Growth Manager USD 90K-105KAd Spend | Adobe Analytics | Customer Acquisition | Customer Lifetime Value | Customer RetentionMid-level Full TimeSan Antonio17h ago
-
APIs | Artificial Intelligence | Business Intelligence | Data Governance | Data PipelinesMid-level Full TimeNotre Dame, IN, United States18h ago
-
Sr. Product Manager AI USD 137K-180KAI Platform | B2B SaaS | Cross-functional | Cross-functional leadership | Enterprise B2BSenior-level Full TimeRemote US R18h ago
-
Associate Director, Data Strategy & Analytics USD 103K-120KAttribution Modeling | BigQuery | Cohort Analysis | DBT | Data Modeling401k match | Dental insurance | Flexible parental leave | Health insurance | Hybrid work optionsMid-level Full TimeNew York R19h ago
-
Manager, Data Engineering USD 125K-174KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage401k match | Charitable giving programs | Dental insurance | Life insurance | Medical insuranceSenior-level Full TimeAnkeny, IA, United States20h ago
-
Senior Product Manager, Robotics & Autonomy USD 150K-221KAgile | Analytics | Autonomy | Computer Vision | Data AnalysisSenior-level Full TimeAustin, Texas, United States21h ago
-
Quantitative Project Manager USD 125K-150KAnalytical Skills | Communication skills | Dashboard Development | Financial markets | Investment Management401k | Casual dress code | Employee discounts | Employee resource groups | Flexible spending accountMid-level Full TimeOld Greenwich22h ago
-
Technical Specialist, ML Data USD 159K-202KActive Learning | Data Curation | Data Mining | Data Pipelines | Data QualityDiscretionary annual bonus | Equity incentive plan | Health benefitsSenior-level Full TimeMountain View, CA, United States23h ago
-
Head of AI Documentation USD 205K-286KArtificial Intelligence | Content Operations | Cross-functional | Cross-functional leadership | Developer documentationExecutive-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA1d ago
-
Product Manager II, YouTube Shorts Gen AI Creation USD 163K-237KCross-Functional Collaboration | Cross-functional | Functional collaboration | Generative AI | Go-to-marketMid-level Full TimeSan Bruno, CA, USA1d ago
-
Abuse prevention | Agentic solutions | Applied AI | Cross-Functional Collaboration | Cross-functionalMid-level Full TimeAustin, TX, USA1d ago
-
Machine Learning Research Manager USD 200K-230KDeep learning | Experiment design | Language Models | Language Processing | Large Language Models401k | Annual company offsite | Dental insurance | Equipment stipend | Flexible PTOMid-level Full TimeUnited States1d ago
-
Engineering Manager - Map AI Platform USD 148K-217K3D Reconstruction | Backend Engineering | Computer Vision | Data Analysis | Data Processing401k program | Commuter benefits | FSA | Family leave | Medical, dental, vision benefitsMid-level Full TimeSan Francisco, CA1d ago
-
Data Engineering Manager USD 200K-335KApache Airflow | Apache NiFi | Cloud platform | Data Governance | Data ManagementSenior-level Full TimeUS - San Francisco1d ago
-
Analytics Product Manager USD 111K-138KAgile | Azure DevOps | Backlog Grooming | Backlog Management | Business Intelligence401k retirement plan | Bonding Leave | Dental insurance | Employee assistance program | Employee referral programMid-level Full TimeWI, United States1d ago
-
Product Management Intern (Summer 2027) USD 108K-112KData Analysis | Machine Learning | Python | SQLDedicated manager | MentorshipEntry-level InternshipBellevue, Washington; Mountain View, California; San …1d ago
-
Associate Product Manager, New Grad (2027 Start) USD 133K-150KAI Tooling | Data Analysis | Python | SQLDedicated manager | MentorshipEntry-level Full TimeBellevue, Washington; Mountain View, California; San …1d ago
-
Senior Manager, Fraud Analytics & Operations USD 70K-92KAnalytics | Control Coverage | Data Analysis | Decision logic | False PositiveHealth and wellness care | Hybrid work model | Visa sponsorship | Work-life balanceSenior-level Full TimeMalvern, PA, United States1d ago