Senior Manager, Machine Learning (Data Operations)
Tasks
- Audit labeled dataset quality
- Define labeling guidelines and taxonomies
- Define labeling pipeline and integration requirements
- Design gold standard datasets
- Drive continuous labeling operations improvement
- Identify and remediate mislabeled data
- Manage Label Studio projects and workflows
- Measure cost per label and turnaround time
- Partner with machine learning and product teams on labeling task design
- Set inter annotator agreement targets
- Set quality SLAs and run calibration sessions
- Source and manage labeling vendors and BPOs
- Support active learning and model assisted labeling initiatives
- Track label accuracy and IAA metrics
- Translate model requirements into labeling programs
Perks/Benefits
- Flexible PTO
- Home office stipend
- Medical, dental, and vision coverage
- WeWork access
- Wellness programs
Skills/Tech-stack
Active Learning | Annotation quality | Calibration | Data Engineering | Data Operations | Data labeling | Document extraction | Entity recognition | Error Taxonomy | Gold Standard | Gold Standard Evaluation | Intent detection | Inter-annotator agreement | Label Studio | Machine Learning | Named Entity Recognition | Python | SQL | Text Classification
Education
N/A
Related jobs
-
Manager, Commercial Analytics CAD 94K-120KAESO | Advanced Analytics | Congestion analysis | Cost modeling | Dispatch AnalysisCompressed work week option | Employee and family assistance program | Flexible benefits program | Flexible work hours option | Hybrid work flexworkSenior-level Full TimeCalgary, Canada1d ago
-
Data Product Manager – Analytical Data Products CAD 130K-160KAgile | Amazon Redshift | Backlog Management | BigQuery | ClickHouseContinuous learning opportunities | Dental insurance coverage | Health insurance coverage | Hybrid work flexibility | Vision insurance coverageMid-level Full TimeToronto, Ontario1d ago
-
Lead, AI Program Management CAD 118K-160KArtificial Intelligence | Business Intelligence | Confluence | Data Science | GovernanceSenior-level Full TimeToronto, ON, Canada1d ago
-
Engineering Manager - Machine Learning CAD 210K-282KAgentic Systems | BigQuery | CUDA | Distributed Computing | DockerRemote workMid-level Full TimeRemote Opportunity - Canada; Toronto, Ontario R1d ago
-
AI initiatives | Agile | Cloud Data | Cloud Data Platforms | Data EngineeringSenior-level Full TimeToronto, Ontario, Canada1d ago
-
Engineering Manager - Data Platform CAD 120K-175KAnalytics Platforms | Apache Spark | Automated testing | Azure Data | Azure Data FactoryMid-level Full TimeVancouver, British Columbia, Canada2d ago
-
Manager, Supply Chain Analytics CAD 94K-115KAnalytics visualization | Business Intelligence | Dashboard Development | Data Governance | Data Pipeline DevelopmentEmployee share purchase plan | Extra days off purchase | Flexible work arrangements | Hybrid work model | Pension planMid-level Full TimeToronto, 700 University Avenue, Canada2d ago
-
Manager, Analytics CAD 100K-120KAWS | Agile | Apache Airflow | Backlog Management | CI/CDCareer growth | Flexible work hours | Professional development | Remote work optionMid-level Full TimeCAN - ON - Yonge St, …2d ago
-
Performance Analytics Reporting & GenAI Lead CAD 134K-186KAlteryx | Analytics governance | Cross-Functional Collaboration | Cross-functional | Dashboard DevelopmentFlexible working | Global recognition programs | Hybrid working | Insurance plans | Paid parental leaveSenior-level Full TimeToronto, Canada2d ago
-
Data, Analytics & Platform Head CAD 140K-194KAI Governance | Analytics | Artificial Intelligence | Budget Management | Change ManagementExecutive-level Full TimeToronto, Canada2d ago
-
Data, Analytics & Platform Head CAD 140K-194KAI Model Deployment | AI model | Analytics | Artificial Intelligence | Budget ManagementHybrid workExecutive-level Full TimeToronto, Canada2d ago
-
Senior Product Operations Manager, Business Analytics CAD 124K-171KDashboarding | Data Governance | Data Modeling | Data Requirements | Data StandardsDental insurance | Health insurance | Healthcare spending | Paid leave | RRSP matchSenior-level Full TimeToronto, Ontario, Canada2d ago
-
Staff Data Analytics Engineer USD 159K-220KApache Airflow | Apache Spark | BigQuery | DBT | Data ContractsFlexible working hours | Remote work flexibilitySenior-level Full TimeAmericas - Remote, US - Remote, … R3d ago
-
Senior Manager, Machine Learning CAD 103K-192KAWS SageMaker | Artificial Intelligence | AzureML | Big Data | ClassificationAccident insurance | Health insurance | Life insurance | Retirement savings plan | Tuition reimbursementSenior-level Full TimeBMOPLACE, Canada3d ago
-
Senior Manager, Analytics and BI CAD 81K-115KBigQuery | Cloud Computing | Dashboard automation | Data Science | ExcelAllyship | Diversity, equity & inclusion | High-performing culture | Inclusive workplaceSenior-level Full TimeToronto, ON, CA, M5C2T63d ago
-
Senior Manager, D2D Analytics CAD 81K-115KBigQuery | Business Intelligence | Campaign Measurement | Cloud Computing | Customer AnalyticsDiversity, equity & inclusion | Inclusive culture | Professional development | Team collaborationSenior-level Full TimeToronto, ON, CA, M5C2W13d ago
-
Analytics Manager CAD 135K-155KAPI | AWS | Git | Microservices | PythonAdditional paid time off | Annual bonus eligibility | Career development opportunities | Employee diversity inclusion programs | Health benefitsMid-level Full TimeCanada - Markham ON 10 Aviva …3d ago
-
Senior Manager, Engineering AI / ML CAD 149K-183KArtificial Intelligence | CI/CD | Data Pipelines | Feature Engineering | MLOpsSenior-level Full TimeTD Centre - TD Tower - …4d ago
-
Senior Product Manager, AI/ML Platform CAD 122K-179KAPI Integration | Analytics ecosystems | Data Exploration | Data analytics | ExperimentationComprehensive benefitsSenior-level Full TimeAMER - Canada - Ontario - …4d ago
-
Manager, Enterprise Data and Analytics CAD 106K-118KAI integration | Agentic AI | Azure | Data Lakehouse | Data ManagementFlexible work hours | Four weeks vacation | Health and dental benefits | Pension | Professional development allowanceSenior-level Full TimeOttawa, Ontario, Canada4d ago
-
Revenue Analytics Manager USD 126K-213KAnalytics engineering | DBT | Data Modeling | Data Warehousing | Forecast accuracyEquity Compensation & Employee Stock Purchase Plan | Flexible paid time off | Growth and development fund | Parental leave | Team member resource groupsMid-level Full TimeRemote, Canada; Remote, US R4d ago
-
Manager, Engineering and Data Platform CAD 130K-150KAI | API Management | Agile | Azure DevOps | Data GovernanceFlexible summer hours | Paid time off | Parental leave | Peer recognition platform | Professional developmentMid-level Full TimeOttawa, ON, Canada4d ago
-
AWS | AWS Glue | Amazon EMR | Amazon Redshift | Apache FlinkCareer development | Employee incentives | Remote work optionSenior-level Full TimeCanada5d ago
-
Adjudication | Audio evaluation | Calibration | Dashboards | Data AnnotationFlexible work location | Fully remote | Influence over quality strategy | International team collaboration | Professional growth opportunitiesSenior-level Full TimeCanada R5d ago
-
Acceptance Testing | Business Intelligence | Clinical data | Clinical data standards | DashboardsDental insurance | Disability insurance | Employee assistance program | Fitness reimbursement | Life insuranceSenior-level Full TimeCanada5d ago