Technical Program Manager III, AI/ML Data Analysis, Search
Tasks
- Apply classical test theory and item response theory
- Architect automated evaluation pipelines
- Communicate technical tradeoffs
- Conduct statistical calibration
- Design human evaluation systems
- Detect rater bias and data integrity threats
- Detect rater drift
- Establish inter rater reliability metrics
- Identify differential item functioning
- Identify risks
- Implement data quality checks
- Integrate human assessment workflows into model development lifecycle
- Lead complex multi-disciplinary projects
- Manage project schedules
- Plan requirements
- Translate latent constructs into behavioral assessments
Perks/Benefits
- N/A
Skills/Tech-stack
Bayesian Modeling | Classical Test Theory | Cohen Kappa | Computational Linguistics | Data Pipelines | Differential Item Functioning | Fleiss Kappa | Hierarchical modeling | Human Feedback | Inter-rater reliability | Intraclass Correlation | Item Response Theory | Language Models | Language Processing | Large Language Models | Learning from Human Feedback | Machine Learning | Multilevel modeling | Natural Language | Natural Language Processing | Program Management | Python | R | Reinforcement Learning | Reinforcement Learning from Human Feedback | Response Theory | Statistical Analysis | Statistical calibration | Test Theory
Education
Roles
Regions
Countries
States
Related jobs
-
Staff Product Manager, Agentic AI Applications USD 172K-258KAPI Design | Agentic Systems | Artificial Intelligence | CI/CD | Command LineSenior-level Full TimeMountain View, California; San Francisco, California10h ago
-
Manager, Analytics - Business Insights USD 100K-151KAgile | Automation | BigQuery | DBT | Data ModelingMid-level Full TimeCharlotte, North Carolina, United States; Denver, …12h ago
-
Data Manager - Senior (KP) Job#818 USD 129K-155KAI machine learning | Agile | Cloud Architecture | Data Pipelines | Data VisualizationSenior-level Full TimeSpringfield, Virginia, United States14h ago
-
Senior-level Full TimeCenter, Center District, IL17h ago
-
Manager, Forward Deployed Engineering - CMEG USD 211K-291KApache Spark | Data Governance | Data Migration | Databricks | Delta LakeMid-level Full TimeRemote - Washington D.C. R17h ago
-
Manager, Forward Deployed Engineering - Manufacturing USD 211K-291KAnalytics | Apache Spark | Data Migration | Databricks | Delta LakeMid-level Full TimeRemote - California R17h ago
-
ASICs | Agent Orchestration | Artificial Intelligence | Bias Mitigation | CommunicationMid-level Full TimeSunnyvale, CA | Bellevue, WA | …20h ago
-
DRAM | Design for test | Design verification | HBM | Machine LearningSenior-level Full TimeSunnyvale, CA, USA20h ago
-
Social Insights and Analytics Manager, AI Marketing USD 141K-206KAI | Audience engagement | Dashboarding | Data Analysis | ForecastingSenior-level Full TimeSan Francisco, CA, USA; New York, …20h ago
-
Product Manager - Data & Analytics USD 110K-145KAgile | Amazon Web Services | Cloud Computing | Cloud platform | Data Architecture401k | Dental insurance | Discounts | Medical insurance | Paid leaveMid-level Full TimeUniversal City, CALIFORNIA, United States R1d ago
-
Principal Program Manager, Commercial AI Programs USD 105K-190KAI Platform | AI Platform Program Management | Agile | Budget Management | Commercial Go to Market401k | Employee stock grants | Employee stock purchase plan | Flexible spending account | Home internet discountsSenior-level Full TimeWA-Headquarters, Building 3, United States1d ago
-
Manager, Cybersecurity - Process and Analytics USD 126K-228KAI Driven | AI-driven detection | Artificial Intelligence | Automation workflows | Cybersecurity401k | Commuter and transit programs | Employee stock grants | Employee stock purchase plan | Home internet discountMid-level Full TimeKS-Headquarters, Building 6360 OP, United States1d ago
-
People Analytics Manager - Workday USD 100K-150KAdaptive Planning | Composite Reports | Correlation | Data Governance | Data QualityAccident insurance | Critical illness insurance | Dental insurance | Disability benefits | Hybrid work arrangementMid-level Full TimeRemote - Illinois, United States R1d ago
-
Data Science Manager USD 140K-160KAgile | Artificial Intelligence | CPLEX | Data Mining | Data VisualizationMid-level Full TimeAtlanta, GA1d ago
-
AlertMedia | Corporate Resilience | Crisis management | Emergency response | EverbridgeSenior-level Full TimeCalifornia - San Francisco, United States1d ago
-
Manager, Fraud Detection and Analytics USD 83K-158KAnomaly Detection | Card Fraud | Data Analysis | Data Visualization | Fraud DetectionMid-level Full TimeOne Destiny Way, Westlake TX, United …1d ago
-
Quant Analytics Manager - Credit Risk Modeling USD 116K-216KCloud Computing | Data Modeling | Distributed Computing | ETL | Machine LearningFlexible work options | In office presence with flexibilityMid-level Full Time4900 Tiedeman Road, Brooklyn, OH, United … R1d ago
-
SVP of Product – Data, Analytics, AI/ML USD 270K-330KAnalytics | Business Intelligence | Data Governance | Data Lakes | Data Privacy401k match | Career development opportunities | Donation matching | Flexible spending account | Health and wellness programExecutive-level Full TimeRemote - USA, United States R1d ago
-
Manager, Applied AI Engineer USD 171K-375KAI/ML | ASR | CRM Integration | Go | GuardrailsHybrid work | Remote work | Work-life balanceSenior-level Full TimeRemote (US), United States R1d ago
-
AI/ML Engineering Manager 2 (Travelbank) USD 119K-140KAPI Design | Agile Development | Artificial Intelligence | Backend Development | Code review401k | Adoption Assistance | Healthcare (Medical Dental Vision) | Life insurance | Long-term disabilityMid-level Full TimeMinneapolis, MN, United States1d ago
-
Capacity modeling | Data Governance | Data Integrity | ERP | Forecasting401k match | Dental insurance | Employee assistance program | Employee scholar program | Flexible spending accountsSenior-level Full TimeUS-GA-MIDLAND-MPC ~ 8801/8987 Macon Rd ~ …1d ago
-
Technical Program Manager – AI & Data Platforms USD 119K-140KCloud Migration | Dependency management | Executive reporting | Financial Forecasting | Issue management401k retirement plan | Healthcare (Medical Dental Vision) | Life insurance | Long-term disability | Paid HolidaysMid-level Full TimeHopkins, MN, United States1d ago
-
Artificial Intelligence | Cloud Computing | Content Writing | Generative AI | Language Models401k matching | Employee assistance program | Flexible spending accounts | Flexible work arrangements | Health insuranceSenior-level Full TimeSeattle, Washington, USA1d ago
-
Amazon Web Services | Apache Iceberg | Apache Spark | Business Development | Cloud platformEquity plan | Health benefits | Paid time off | Travel opportunitiesSenior-level Full TimeRedwood City, California, United States1d ago
-
Data Science Manager - Market Origination USD 160K-190KForecasting | Predictive Modeling | Python | R | SQL401k match | Career development | Company equipment | Company events | Disability benefitsMid-level Full TimeAtlanta, GA preferred, Remote R1d ago