Product Content Engineer
Menlo Park, CA | New York, NY | San Francisco, CA
USD 129K-187K (estimate) Mid-level Full Time
Tasks
- Build workflows for collecting annotating and analyzing AI outputs
- Collaborate cross functionally to align AI behaviors with user expectations
- Define content quality standards and evaluate AI model performance across products
- Design evaluation frameworks using qualitative and quantitative signals
- Design golden sets taxonomies and guidelines for repeatable content quality assessments
- Evaluate model releases using structured comparisons and documentation
- Synthesize evaluation results into error patterns and actionable recommendations
- Track content quality and model performance over time and flag regressions
Perks/Benefits
- N/A
Skills/Tech-stack
A/B | A/B Testing | Annotation | B testing | Bias Mitigation | Content labeling | Dashboards | Data Analysis | Data Visualization | Ethical AI | Evaluation Framework | Human evaluation | Model as judge | Prompt engineering | Python | Red Teaming | Risk Assessment | SQL | Taxonomy
Education
Roles
Regions
Countries
States
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R3d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R17d ago
-
Senior-level Full TimeUS - Remote; Canada - Remote R10h ago
-
Senior-level Full TimeMountain View, CA12h ago
-
Staff Software Engineer, AI Integrations USD 170K-240KAgile | Alerting | Amazon Web Services | C++ | CSSSenior-level Full TimeMountain View, CA12h ago
-
Senior Software Engineer Remote/Travel- Active Secret -Top Secret clearance(SCI-eligibility) USD 145K-204KAPI Testing | Cypher | Data Processing | DataOps | DevOpsCompetitive benefits | Growth opportunity | Remote work | TravelSenior-level Full TimeReston, VA, United States R22h ago
-
AWS | AWS CloudFormation | Amazon ECS | Automation | CI/CD401-k plan | Company match | Employee stock purchase plan | Healthcare benefits | Life insuranceMid-level Full TimeAtlanta, GA, United States1d ago
-
Senior Staff Generative AI Scientist USD 185K-210KAWS | Apache Spark | Artificial Intelligence | Azure | Cloud platform401k matching | Dental insurance | Disability insurance | Health insurance | Life insuranceSenior-level Full TimeRemote, United States R1d ago
-
Machine Learning Engineer USD 117K-177KAWS | Azure | Cloud Computing | Data Visualization | Google CloudMid-level Full TimeSt. Louis, Missouri, United States1d ago
-
Software Engineer - X Data USD 125K-400KApache Flink | Apache Kafka | Apache Spark | BigQuery | Cause analysis401k retirement plan | Dental insurance | Discounts | Health insurance | Life insuranceMid-level Full TimePalo Alto, CA1d ago
-
Embedded Software Engineer Intern (Fall 2026) USD 108K-108KADC | Antenna Testing | C# | C++ | CANHousing stipend | Overtime pay | Paid sick time | Relocation supportEntry-level InternshipSouth San Francisco, California, USA1d ago
-
AI Engineer – Decision & Optimization Systems USD 80K-210KAMPL | C++ | CPLEX | Constraint Satisfaction | Go401k | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
AI Engineer - Allocation and Packing Systems USD 80K-210KA/B | A/B Testing | Approximation algorithms | Assignment Algorithms | B testing401k | Equity grant | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
AI Engineer – Routing & Network Optimization USD 80K-210KA Star | A Star Algorithm | AMPL | Bellman Ford Algorithm | CPLEX401k | Equity grant | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
AI Software Engineer USD 80K-210KAWS Kinesis | Apache Airflow | Apache Kafka | C# | C++401k | Equity grant | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
Deployment Strategist - North America USD 150K-245KAI Agent | AI Agent Frameworks | API Integration | Agent Frameworks | Agent OrchestrationAnnual company offsite | Annual travel stipend | Co-working stipend | Professional development stipendMid-level Full TimeUnited States R1d ago
-
Applied AI Engineer USD 204K-352KBackend Development | Cloud infrastructure | Deep learning | Docker | Fine TuningAnnual membership | Centralized coffee stipend | Flexible PTO | Health and wellness stipend | Health, dental, and vision insuranceSenior-level Full TimeNew York, NY (HQ)1d ago
-
ML Research Engineer - Hardware Codesign USD 185K-455KC++ | CUDA | Floating point | Floating point numerics | Functional simulationHybrid work schedule | Relocation assistanceSenior-level Full TimeSan Francisco1d ago
-
Staff AI Engineer USD 140K-210KAPI Development | AWS | Access Management | Bedrock AgentCore | CI/CD401k | All-hands meetings | Dental insurance | Employee equity program | Health insuranceSenior-level Full TimeNew York City1d ago
-
Senior Quantum Embedded Engineer USD 142K-175K10G Ethernet | AMD Xilinx | AMD Xilinx toolchain | AMD/Xilinx SoC | BashHybrid work options | Remote work optionsSenior-level Full TimeNew Haven, CT1d ago
-
Associate Quantum Engineer USD 130K-184KCryogenic Systems | Data Analysis | Data acquisition | Instrument Control | Laboratory Instrument ControlInterdisciplinary team | Mentorship | State-of-the-art facilitiesMid-level Full TimeNew Haven, CT1d ago
-
Quantum Engineer (Physicist) USD 136K-187KCircuit-QED | Cryogenics | Data Analysis | Error correction | Low temperature physicsMid-level Full TimeNew Haven, CT1d ago
-
Senior Quantum Applications Engineer - QEC USD 120K-258KCUDA-Q | Decoder algorithms | Docker | End to End | End-to-End TestingSenior-level Full TimeNew Haven, CT1d ago
-
Staff AI engineer USD 170K-250KAI Evaluation | AWS | Agent Orchestration | Caching | Distributed SystemsFlexible working hours | Hybrid work culture | Unlimited time offSenior-level Full TimeSan Francisco1d ago
-
C++ | Deep learning | Imitation Learning | Machine Learning | PlanningHealth insurance | Life insurance | Long-term care insurance | Long-term disability insurance | Paid time offSenior-level Full TimeFoster City, CA1d ago