Technical Program Manager, Gemini Evals, DeepMind
Tasks
- Conduct Loss Analysis
- Coordinate alignment with model teams
- Design and integrate model evals
- Execute model evals
- Identify risks and communicate to stakeholders
- Implement workflow improvements
- Lead multi-disciplinary engineering projects
- Manage Gemini autorater platform
- Manage project schedules
- Plan compute capacity
- Plan requirements with internal customers
- Provide proactive stakeholder status updates
- Track priorities and issues
- Triage autorater issues to modeling team
Perks/Benefits
- N/A
Skills/Tech-stack
Capacity Planning | Compute Capacity Planning | Compute capacity | Data Science | Experimentation | Infrastructure Planning | Language Models | Large Language Models | Loss analysis | Machine Learning | Model Evaluation | Project Management | Risk Management | Stakeholder management
Education
Bachelor of Arts | Bachelor of Engineering | Bachelor of Science
Roles
Regions
Countries
States
Related jobs
-
Data Requirements | Deep learning | KPI Management | Machine Learning | Model Deployment401k matching | Flexible vacation | Health insurance | Home office setup stipend | Paid parental leaveMid-level Full TimeRemote - United States R1d ago
-
Project Manager, AI and Data Science USD 132K-190KAWS | Agile | Change Management | Confluence | Data Ingestion401k matching | EAP mental health support | Floating holiday | Hybrid remote work | Onsite work 1 day per weekMid-level Full TimePipersville, PA, United States1d ago
-
Manager, Analytics and Optimization - Client Care USD 89K-120KAI strategy | Analytics | Automation | Business case | Business case development401k | Employee assistance program | Fertility support | Flexible spending accounts | Healthcare benefitsMid-level Full Time9 WEST OFFICE, United States1d ago
-
AI Office Lead USD 160K-200KAI ethics | Artificial Intelligence | Budget Management | Change Management | Data PrivacySenior-level Full TimeKansas City, KS, United States1d ago
-
C# | C++ | Computer Vision | Data Processing | DebuggingSenior-level Full TimeMountain View, CA, USA2d ago
-
Data Analysis | Deep learning | Direct Preference Optimization | Fine Tuning | Language ModelsSenior-level Full TimeSunnyvale, CA, USA2d ago
-
Analytics | Artificial Intelligence | Campaign orchestration | Cloud Computing | Cost OptimizationSenior-level Full TimeKirkland, WA, USA; Austin, TX, USA2d ago
-
Budgeting | Cross-Functional Collaboration | Cross-functional | Data Analysis | Executive CommunicationSenior-level Full TimeMountain View, CA, USA2d ago
-
Enterprise AI Platform Governance & Operations Lead USD 125K-270KAI Foundry | AI Governance | Access Management | Artificial Intelligence | Audit managementSenior-level Full TimeUS-CA-Fremont (1003)2d ago
-
Product Management, Human Data Platform USD 305K-385KAnnotation | Data Infrastructure | Data Pipelines | Data Quality | Data labelingFlexible working hours | Generous vacation | Optional equity donation matching | Parental leave | Visa sponsorshipMid-level Full TimeSan Francisco, CA | New York …2d ago
-
Automation | Computer Vision | Data Annotation | Data Quality | Data labelingCommuter stipend | Comprehensive health, dental and vision coverage | Generous PTO | Learning and development stipend | Retirement benefitsMid-level Full TimeSt. Louis, MO; Washington, DC2d ago
-
Digital Analytics Manager, CNBC USD 110K-130KAdobe Analytics | Amplitude | Databricks | Experimentation | Google Analytics401k | Daily group classes | Daily specials | Dental insurance | Dry cleaningMid-level Full TimeEnglewood Cliffs, NJ, United States2d ago
-
Delivery Solutions Architect - High-Tech & Manufacturing USD 180K-247KBusiness Analysis | Data Architecture | Distributed Systems | Executive Communication | Program ManagementHybrid work | Travel 30 percentSenior-level Full TimeRemote - Texas R2d ago
-
Delivery Solutions Architect - High-Tech & Manufacturing USD 180K-247KAccount Management | Business Analysis | Data Architecture | Distributed Systems | KPI ReportingAnnual performance bonus | Equity | Hybrid work | Travel up to 30 percentSenior-level Full TimeOklahoma2d ago
-
Data Governance & Enablement Manager USD 115K-150KAI Governance | Accountability | Agile | Alation | Audit ReadinessMid-level Full TimeMiami Lakes, FL, United States2d ago
-
Credit Risk Analytics Manager-Finance USD 115K-169KAttribution | Backtesting | Benchmarking | CECL | Credit RiskMid-level Full TimeMiami Lakes, FL, United States2d ago
-
API configuration | Change Management | Cross-Functional Collaboration | Cross-functional | Data Governance401k savings plan | Dependent Ski Pass | Employee Ski Pass | Flexible spending account | Health, dental, vision insuranceSenior-level Full TimeAspen, Basalt or Denver, Colorado, United …2d ago
-
IT Data Warehouse Leader USD 146K-215KAcceptance Testing | Agile | Data Ingestion | Data Security | Data TransformationHealth benefits | Paid time off | Retirement benefitsSenior-level Full TimeBeachwood, Ohio, USA, 441222d ago
-
AWS | Azure | Data Science | GenAI | Google CloudExecutive-level Full TimeMaryland; Virginia; Washington, D.C.3d ago
-
Program Manager – Data, AI, Databricks | Full-Time | Remote | 12+ years exp. | USC/GC Preferred USD 127K-195KAgile | Artificial Intelligence | Data Engineering | Data platforms | DatabricksOpen across locations | Remote workSenior-level Contract Full TimeDetroit, MI, United States R3d ago
-
AI Agents | Advanced Analytics | Artificial Intelligence | Cross-Functional Collaboration | Cross-functionalTravel On Occasion | Visa sponsorshipExecutive-level Full TimeNew York, NY, United States3d ago
-
Technical Program Manager, MTIA Software USD 177K-251KAI Algorithms | AI hardware | Agent Orchestration | Benchmarking | Bias MitigationMid-level Full TimeMenlo Park, CA3d ago
-
Engineering Manager, SSD Storage USD 207K-301KCapacity Planning | Data Migration | Disk drives | Embedded Systems | Feature PrioritizationSenior-level Full TimeSunnyvale, CA, USA3d ago
-
Product Manager, Gemini API Models, DeepMind USD 217K-237KAPI Design | Application Programming | Application Programming Interfaces | Developer Feedback | Developer Feedback LoopsMid-level Full TimeMountain View, CA, USA3d ago
-
Compliance | Developer experience | Distributed Systems | Enterprise Data | Generative AISenior-level Full TimeKirkland, WA, USA; Sunnyvale, CA, USA3d ago