Manager, Content Engineering — AI Content Understanding
Menlo Park, CA | New York, NY | San Francisco, CA
USD 134K-196K (estimate) Mid-level Full Time
Tasks
- Build repeatable evaluation sprint processes
- Calibrate automated evaluations using human data
- Create and maintain golden datasets
- Define evaluation guidelines and quality rubrics
- Identify and mitigate operational risks in evaluation pipelines
- Manage and develop content engineering team
- Manage workforce planning and performance management
- Own continuous content understanding model evaluations
- Report evaluation results to cross functional leadership
- Train reviewers and run calibration sessions
- Translate evaluation insights into model and prompt improvements
Perks/Benefits
- N/A
Skills/Tech-stack
A/B | A/B Testing | Annotation | B testing | Content labeling | Data Analysis | Evaluation Frameworks | Golden datasets | LLM-as-a-Judge | Machine Learning | Model Benchmarking | Prompt engineering | Python | Quality Rubrics | Recommendation Systems | Red Teaming | SQL | Search & Ranking | Trust and Safety
Education
Regions
Countries
States
Related jobs
-
Head of AI USD 150K-300KAPI Integration | AWS | Artificial Intelligence | Automations | ChatGPT401k with company matching | Dental insurance | Fully remote work environment | Health insurance | Vision insuranceExecutive-level Full TimeAustin, Texas, Austin HQ, Dallas, Texas, … R4h ago
-
Product Manager, AI Integrity (Mandarin Speaker) USD 86K-232KAI Agents | Behavioral Analysis | Customer Insights | Data Analysis | Information RetrievalMid-level Full TimeSan Jose, California, United States6h ago
-
Senior Product Manager, AI Solutions (Mandarin Speaker) USD 161K-350KAnalytics | Cross-functional | Cross-functional leadership | Data Analysis | Functional leadershipSenior-level Full TimeSan Jose, California, United States6h ago
-
Artificial Intelligence | Computer Vision | Data strategy | Latency optimization | Machine LearningSenior-level Full TimeMountain View, CA, USA7h ago
-
AI strategy | API Design | Big Data | Cross-functional | Cross-functional leadershipSenior-level Full TimeNew York, NY, USA; Mountain View, …7h ago
-
Staff AI Research Engineer, Large User Models USD 207K-300KAlgorithms | Artificial Intelligence | Attention Mechanisms | BERT | Data StructuresSenior-level Full TimeMountain View, CA, USA7h ago
-
AI Architect, Partner Co-Innovation, Google Cloud USD 183K-265KAPI performance | Agent systems | AlloyDB | BigQuery | C++Senior-level Full TimeSunnyvale, CA, USA; Atlanta, GA, USA7h ago
-
AI Transformation Lead USD 155K-160KAI Governance | Analytics | Artificial Intelligence | Change Management | Data ScienceSenior-level Full TimeDallas, TX, United States8h ago
-
Mid-level Full TimeSeattle15h ago
-
AI tools | API Integration | Agent Based Workflows | Agent-based | Analytics401k | Dental insurance | Health insurance | Paid time off | Travel opportunitiesSenior-level Full TimeSan Francisco, CA, United States16h ago
-
Business AI Architect - Manufacturing/PLM USD 151K-258KAI Agents | AI Business Services | AI Core | Analytics Cloud | Artificial IntelligenceSenior-level Full TimeNewtown Square, PA, US, 1907316h ago
-
AWS | Agile | CI/CD | Cloud Computing | Cloud platformCareer development support | Flexible working models | Learning resources | Mentoring | Wellbeing prioritizationEntry-level InternshipSan Ramon, CA, US, 945 8316h ago
-
AVP, Lifestyle Analytics - JCrew USD 100K-170KCampaign Optimization | Data Mining | Excel | Forecasting | Marketing AnalyticsOccasional travel for meetings | Work from home flexibilityExecutive-level Full TimeChicago Site, United States18h ago
-
AI Solutions Consultant USD 119K-206KADK | C++ | Chunking | Dialogflow CX | EmbeddingsHybrid work schedule | Up to 10 percent travelMid-level Full Time141753-NC-Three Wells Fargo Center, Charlotte, United …18h ago
-
Data Science Practice Lead USD 140K-285KA/B | A/B Testing | Agentic Workflows | Agile | Artificial IntelligenceComprehensive health care | Educational assistance | Emotional well-being support | Employee matching charitable giving | Hybrid work scheduleSenior-level Full Time245 Summer St, Boston MA, United …18h ago
-
Manager, Sales Analytics USD 120K-135KDashboard Development | Data Integrity | Data Modeling | Data Quality | Data Warehousing401k matching | Comprehensive health insurance | Dental insurance | Employee referral bonus | Paid-time-off policyMid-level Full TimeRemote (United States) R20h ago
-
Manager, Sales Analytics USD 120K-135KAnalytics | Business Intelligence | Dashboard Development | Data Integrity | Data Modeling401k | Dental insurance | Employee referral bonus | Employer matching | Health insuranceMid-level Full TimeRemote (United States) R20h ago
-
AI/ML Scientist – Protein Foundation Models USD 120K-200KAWS | Alphafold | Attention Mechanisms | Diffusion Models | Distributed ComputingOn-site work | Relocation assistanceMid-level Full TimeBoston, MA or San Francisco, CA21h ago
-
AI/ML Scientist USD 120K-200KAutoregressive models | Cloud Computing | Computational Biology | Data Engineering | Deep learningOn-site work | Relocation supportMid-level Full TimeBoston, MA or San Francisco, CA21h ago
-
Machine Learning Engineering Manager - AI Insights USD 139K-304KAnomaly Detection | Azure Data | Azure Data Bricks | C# | C++Mid-level Full TimeNew York, NY, US; Redmond, WA, …21h ago
-
AI Automation Infrastructure Engineer USD 100K-176KAPI Development | APIs | Agent Frameworks | Anomaly Detection | AnsibleMid-level Full TimeUSA - Sandy, UT21h ago
-
Engineering Manager, Data Platform USD 150K-240KAirflow | Amazon Redshift | BigQuery | Data Modeling | Data PipelinesMid-level Full TimeRemote in USA R21h ago
-
Forward Deployment AI Engineer USD 120K-150KAgent Orchestration | Anthropic API | Autogen | Chroma | Embedding pipelinesHybrid work | Travel to client sitesMid-level Full TimeNaperville, IL21h ago
-
Staff AI Researcher USD 160K-210KApache Spark | Benchmarking | Data Processing | Data Transformation | DatabricksSenior-level Full TimeRemote, United States R22h ago
-
Manager of Data & Analytics USD 130K-195KAI Governance | Alation | Artificial Intelligence | Atlan | Azure DataSenior-level Full TimeLouisville, Kentucky, United States22h ago