Product Content Engineer
Menlo Park, CA | New York, NY | San Francisco, CA
USD 129K-187K (estimate) Mid-level Full Time
Tasks
- Build workflows for collecting annotating and analyzing AI outputs
- Collaborate cross functionally to align AI behaviors with user expectations
- Define content quality standards and evaluate AI model performance across products
- Design evaluation frameworks using qualitative and quantitative signals
- Design golden sets taxonomies and guidelines for repeatable content quality assessments
- Evaluate model releases using structured comparisons and documentation
- Synthesize evaluation results into error patterns and actionable recommendations
- Track content quality and model performance over time and flag regressions
Perks/Benefits
- N/A
Skills/Tech-stack
A/B | A/B Testing | Annotation | B testing | Bias Mitigation | Content labeling | Dashboards | Data Analysis | Data Visualization | Ethical AI | Evaluation Framework | Human evaluation | Model as judge | Prompt engineering | Python | Red Teaming | Risk Assessment | SQL | Taxonomy
Education
Roles
Regions
Countries
States
Related jobs
-
Project, Engineer II - Talent Technology and Analytics Support - TS/SCI Poly - Washington, D.C. USD 97K-162KAcceptance Testing | Data Analysis | Data Visualization | Data Warehousing | Microsoft OfficeOn site work 100 percentMid-level Full TimeArlington/Rosslyn, Virginia, United States23h ago
-
Databricks Forward Deployed Engineer - GPS USD 97K-162KAPIs | AWS | Agent Bricks | Apache Airflow | Apache SparkMentorship | Professional development | Travel 50 PercentMid-level Full TimeArlington/Rosslyn, Virginia, United States; Atlanta, Georgia, …23h ago
-
OpenAI Forward Deployed Engineer - GPS USD 97K-163KAPI Integration | AWS | Agents SDK | Airflow | Assistants APIProfessional development | Security clearance support | Travel opportunitiesMid-level Full TimeArlington/Rosslyn, Virginia, United States; Atlanta, Georgia, …23h ago
-
Lead Snowflake Forward Deployed Engineer - GPS USD 102K-174KAPI Integration | AWS | Arctic Embed | CI/CD | Cortex AILeadership opportunities | Mentorship | Professional development | Travel opportunities | US government security clearance supportSenior-level Full TimeArlington/Rosslyn, Virginia, United States; Atlanta, Georgia, …23h ago
-
Data Scientist, Innovation Hub USD 108K-108KAPI Integration | Audit Logging | Azure | CI/CD | Code reviewOn-site childcare | Paid parental leave | Paid time off | Public service loan forgiveness | Retirement programsMid-level Full TimeTexas-Dallas-5323 Harry Hines Blvd1d ago
-
System Safety Engineer, Humanoid Robotics USD 170K-212KC++ | Critical software | FMEA | FTA | Failure Mode and Effects AnalysisSenior-level Full TimeBurlingame, CA1d ago
-
Robotics Verification & Validation Engineer USD 170K-225KAI Risk | AI Risk Assessment | AI model | AI model validation | AI qualitySenior-level Full TimeBurlingame, CA1d ago
-
Technical Lead, Machine Learning, Google Play USD 207K-300KAlgorithms | C++ | Computer Vision | Data Structures | ExperimentationSenior-level Full TimeMountain View, CA, USA1d ago
-
Asynchronous programming | BigQuery | C++ | Concurrency | Distributed DatabasesSenior-level Full TimeKirkland, WA, USA1d ago
-
Staff Data Engineer, Search, Data Science USD 207K-300KC++ | Data Engineering | Data Pipelines | Data Systems | Data systems architectureSenior-level Full TimeMountain View, CA, USA; New York, …1d ago
-
Senior Staff Software Engineer, AI/ML, Security USD 262K-365KAdversarial Machine Learning | Cloud | Data Privacy | Data Processing | Data StructuresSenior-level Full TimeKirkland, WA, USA; Seattle, WA, USA1d ago
-
Applied AI Staff Software Engineer, Looker USD 207K-300KData Processing | Data Visualization | Debugging | Distributed Systems | Fine TuningSenior-level Full TimeKirkland, WA, USA1d ago
-
Staff Software Engineer, AI/ML, Google Public Sector USD 207K-300KAccelerator optimization | C++ | Cloud Object Storage | Deep learning | Distributed SystemsSenior-level Full TimeReston, VA, USA; Washington D.C., DC, …1d ago
-
Data Engineer USD 96K-137KApache Airflow | CI/CD | DBT | GCP | Python15 days PTO | 401k matching | Disability coverage | Life insurance | Medical/Dental/Vision insuranceMid-level Full TimePiscataway, NJ, US1d ago
-
Data Engineer USD 96K-137KApache Airflow | CI/CD | DBT | Python | SQL401k matching | Basic life insurance | Dental insurance | Disability coverage | Medical insuranceMid-level Full TimePiscataway, NJ, US1d ago
-
Data Engineer Lead | $140k-$175k + Hybrid + Equity | Exciting High Growth AI Operational Intelligence Startup A USD 140K-175KApache Airflow | Apache Kafka | DBT | Dagster | Data LineageEquity | Health insurance | Hybrid work | Medical insurance | Paid HolidaysExecutive-level Full TimeWayne, PA, United States R1d ago
-
Corporate Vice President, AI & Data Science USD 147K-211KAWS | Agent Orchestration | Agentic AI | Bedrock | CI/CDExecutive-level Full TimeNew York, NY, US1d ago
-
Applied AI Engineer | $150K-$175K + Hybrid + Equity | High Growth AI Operational Intelligence Startup A USD 150K-175KAI Agents | AI orchestration | APIs | LLM | Model EvaluationEquity | Health medical and vision coverage | Hybrid work | One day per week onsite | Paid HolidaysExecutive-level Full TimeWayne, PA, United States R1d ago
-
Mid-level Full TimeAustin, Texas, United States1d ago
-
Data Engineer USD 70K-80KData Ingestion | Microsoft SQL | Microsoft SQL Server | Python | REmployee assistance program | Health benefits | Retirement plan matching | Student loan assistance | Time offMid-level Full TimeVillanova, PA, United States1d ago
-
Lead Analytics Engineer USD 219K-240KAirflow | Amazon Redshift | Automated testing | BigQuery | CI/CD401k | Dental insurance | Equity | Flexible paid time off | Health insuranceSenior-level Full TimePalo Alto, CA1d ago
-
Mid-level Full TimeEl Segundo, California, United States1d ago
-
Senior-level Full TimeOnsite - Austin, TX1d ago
-
Machine Learning Engineer, Geometry Team USD 175K-215K3D Geometry | C++ | Computer Vision | Distributed Training | Fine Tuning401k match | Baby bonding leave | Dental insurance | Disability insurance | Health insuranceSenior-level Full TimeKirkland, Washington, United States1d ago
-
Member of the Technical Staff - Chatbot Engineer USD 175K-275KAgent Loop | BigQuery | Context Management | LLM | Language ProcessingSenior-level Full TimeSan Francisco HQ1d ago