Technical Program Manager III, AI/ML Data Analysis, Search
Tasks
- Apply classical test theory and item response theory
- Architect automated evaluation pipelines
- Communicate technical tradeoffs
- Conduct statistical calibration
- Design human evaluation systems
- Detect rater bias and data integrity threats
- Detect rater drift
- Establish inter rater reliability metrics
- Identify differential item functioning
- Identify risks
- Implement data quality checks
- Integrate human assessment workflows into model development lifecycle
- Lead complex multi-disciplinary projects
- Manage project schedules
- Plan requirements
- Translate latent constructs into behavioral assessments
Perks/Benefits
- N/A
Skills/Tech-stack
Bayesian Modeling | Classical Test Theory | Cohen Kappa | Computational Linguistics | Data Pipelines | Differential Item Functioning | Fleiss Kappa | Hierarchical modeling | Human Feedback | Inter-rater reliability | Intraclass Correlation | Item Response Theory | Language Models | Language Processing | Large Language Models | Learning from Human Feedback | Machine Learning | Multilevel modeling | Natural Language | Natural Language Processing | Program Management | Python | R | Reinforcement Learning | Reinforcement Learning from Human Feedback | Response Theory | Statistical Analysis | Statistical calibration | Test Theory
Education
Roles
Regions
Countries
States
Related jobs
-
Senior-level Full TimeCenter, Center District, IL16h ago
-
ASICs | Agent Orchestration | Artificial Intelligence | Bias Mitigation | CommunicationMid-level Full TimeSunnyvale, CA | Bellevue, WA | …19h ago
-
DRAM | Design for test | Design verification | HBM | Machine LearningSenior-level Full TimeSunnyvale, CA, USA19h ago
-
Social Insights and Analytics Manager, AI Marketing USD 141K-206KAI | Audience engagement | Dashboarding | Data Analysis | ForecastingSenior-level Full TimeSan Francisco, CA, USA; New York, …19h ago
-
Product Manager - Data & Analytics USD 110K-145KAgile | Amazon Web Services | Cloud Computing | Cloud platform | Data Architecture401k | Dental insurance | Discounts | Medical insurance | Paid leaveMid-level Full TimeUniversal City, CALIFORNIA, United States R1d ago
-
Principal Program Manager, Commercial AI Programs USD 105K-190KAI Platform | AI Platform Program Management | Agile | Budget Management | Commercial Go to Market401k | Employee stock grants | Employee stock purchase plan | Flexible spending account | Home internet discountsSenior-level Full TimeWA-Headquarters, Building 3, United States1d ago
-
Manager, Cybersecurity - Process and Analytics USD 126K-228KAI Driven | AI-driven detection | Artificial Intelligence | Automation workflows | Cybersecurity401k | Commuter and transit programs | Employee stock grants | Employee stock purchase plan | Home internet discountMid-level Full TimeKS-Headquarters, Building 6360 OP, United States1d ago
-
People Analytics Manager - Workday USD 100K-150KAdaptive Planning | Composite Reports | Correlation | Data Governance | Data QualityAccident insurance | Critical illness insurance | Dental insurance | Disability benefits | Hybrid work arrangementMid-level Full TimeRemote - Illinois, United States R1d ago
-
Data Science Manager USD 140K-160KAgile | Artificial Intelligence | CPLEX | Data Mining | Data VisualizationMid-level Full TimeAtlanta, GA1d ago
-
AlertMedia | Corporate Resilience | Crisis management | Emergency response | EverbridgeSenior-level Full TimeCalifornia - San Francisco, United States1d ago
-
Manager, Fraud Detection and Analytics USD 83K-158KAnomaly Detection | Card Fraud | Data Analysis | Data Visualization | Fraud DetectionMid-level Full TimeOne Destiny Way, Westlake TX, United …1d ago
-
Quant Analytics Manager - Credit Risk Modeling USD 116K-216KCloud Computing | Data Modeling | Distributed Computing | ETL | Machine LearningFlexible work options | In office presence with flexibilityMid-level Full Time4900 Tiedeman Road, Brooklyn, OH, United … R1d ago
-
SVP of Product – Data, Analytics, AI/ML USD 270K-330KAnalytics | Business Intelligence | Data Governance | Data Lakes | Data Privacy401k match | Career development opportunities | Donation matching | Flexible spending account | Health and wellness programExecutive-level Full TimeRemote - USA, United States R1d ago
-
Manager, Applied AI Engineer USD 171K-375KAI/ML | ASR | CRM Integration | Go | GuardrailsHybrid work | Remote work | Work-life balanceSenior-level Full TimeRemote (US), United States R1d ago
-
AI/ML Engineering Manager 2 (Travelbank) USD 119K-140KAPI Design | Agile Development | Artificial Intelligence | Backend Development | Code review401k | Adoption Assistance | Healthcare (Medical Dental Vision) | Life insurance | Long-term disabilityMid-level Full TimeMinneapolis, MN, United States1d ago
-
Capacity modeling | Data Governance | Data Integrity | ERP | Forecasting401k match | Dental insurance | Employee assistance program | Employee scholar program | Flexible spending accountsSenior-level Full TimeUS-GA-MIDLAND-MPC ~ 8801/8987 Macon Rd ~ …1d ago
-
Technical Program Manager – AI & Data Platforms USD 119K-140KCloud Migration | Dependency management | Executive reporting | Financial Forecasting | Issue management401k retirement plan | Healthcare (Medical Dental Vision) | Life insurance | Long-term disability | Paid HolidaysMid-level Full TimeHopkins, MN, United States1d ago
-
Artificial Intelligence | Cloud Computing | Content Writing | Generative AI | Language Models401k matching | Employee assistance program | Flexible spending accounts | Flexible work arrangements | Health insuranceSenior-level Full TimeSeattle, Washington, USA1d ago
-
Data Science Manager - Market Origination USD 160K-190KForecasting | Predictive Modeling | Python | R | SQL401k match | Career development | Company equipment | Company events | Disability benefitsMid-level Full TimeAtlanta, GA preferred, Remote R1d ago
-
Data Science Manager - Sports Pricing USD 160K-190KData Modeling | Machine Learning | Optimization | Python | R401k match | Annual performance reviews | Career development | Company equipment | Company eventsMid-level Full TimeAtlanta, GA preferred, Remote R1d ago
-
AWS | Azure | Causal Inference | Cloud platform | DatabricksBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site wellness centersExecutive-level Full TimeNew York, NY, United States1d ago
-
Senior Manager, Marketing Analytics & Operations USD 145K-155KA/B | A/B Testing | Ad Spend | Ads API | Airbyte11 paid holidays | 401(k) plan match | Employee assistance program | Employee discounts | Employee ownership programSenior-level Full TimeRemote United States R1d ago
-
Analytics Solutions Manager - Vice President USD 225K-260KCortex Analyst | Data Governance | Data Modeling | Data Quality | In memory BIBackup childcare | Financial coaching | Health insurance | Mental health support | On-site health and wellness centersExecutive-level Full TimeChicago, IL, United States1d ago
-
Practice Manager - AI & Data USD 160K-190KAWS | Agent systems | Amazon SageMaker | Apache Spark | AzureCareer growth opportunities | Comprehensive benefits | MentorshipSenior-level Full TimeBroomfield, CO. Greensboro, NC. Troy, Michigan1d ago
-
Budget Forecasting | Contract Management | Dashboard Reporting | Data Governance | Data ManagementMid-level Full TimeWesterville, OH, United States1d ago