Machine Learning Engineer, LLM Evals & Observability
Tasks
- Build AI agent observability
- Build evaluation pipelines
- Close loop between eval results and improvements
- Collaborate with engineers on evaluation integration
- Create trace enrichment dashboards
- Design evaluation datasets
- Develop LLM judges
- Evaluate models before release
Perks/Benefits
- 401k contribution
- Education stipend
- Generous time off
- Healthy lunches daily
- Home office improvement stipend
- Hybrid work schedule
- Medical, dental & vision coverage
- Wellness stipend
Skills/Tech-stack
Data Pipelines | Distributed Systems | Go | Human Feedback | LLM Evaluation | Language Models | Language Processing | Large Language Models | Learning from Human Feedback | Machine Learning | Natural Language | Natural Language Processing | Observability | Python | Reinforcement Learning | Reinforcement Learning from Human Feedback
Education
N/A
Regions
Countries
States
Related jobs
-
Aerospace software | C++ | Camera | EKF | Factor GraphsDental insurance | Equity | Health insurance | Paid time off | Vision insuranceSenior-level Full TimeHawthorne, CA3h ago
-
AI Engineer — Learn Engine: Platform & Simulation USD 160K-223KA/B | A/B Testing | B testing | Browser Automation | Computer VisionMid-level Full TimeSan Francisco HQ3h ago
-
Machine Learning Engineer, Depot Automation USD 175K-215KFoundation Models | Industrial robotics | Machine Learning | Model Evaluation | Production DeploymentAnnual bonus | Company benefits program | Equity incentive plan | Hybrid work scheduleMid-level Full TimeMountain View, CA, US4h ago
-
Automated testing | C# | C++ | CAN | CI/CD401k retirement | Dental insurance | Flexible Spending Account (FSA) | Health Savings Account (HSA) | Health insuranceSenior-level Full TimeHenderson, CO6h ago
-
Robotics Technician USD 100K-300KCAD | Cause analysis | Debugging | Electromechanical | Embedded SystemsSenior-level Full TimeSan Mateo6h ago
-
AWS | Analytics | Data Mining | Generative AI | Machine LearningMentorship | Training | Work-life balanceSenior-level Full TimeArlington, Virginia, USA8h ago
-
Senior Perception Engineer USD 154K-210KAWS | Computer Vision | Containerization | Data Annotation | Data AugmentationOnsite work five days per week | Startup environment high ownershipSenior-level Full TimeSan Francisco8h ago
-
Senior Software Engineer, Robotics Platform USD 140K-190KBuild systems | Cloud infrastructure | Data logging | Debugging | Device provisioningOnsite work five days a weekSenior-level Full TimeSan Francisco8h ago
-
Energy Data Engineer/Analyst USD 110K-130KApache Airflow | Data Modeling | Data Warehousing | Databricks | PythonMid-level Full TimeFoster City, California, United States9h ago
-
Robotics Software Test Engineer USD 130K-175KAutomated testing | CI/CD | Debugging | Docker | GazeboHigh-ownership environment | Onsite work 5 days per weekMid-level Full TimeSan Francisco9h ago
-
API Integration | AWS | Autogen | Azure | Cloud platformHybrid work environmentSenior-level Contract Full TimeChicago, Illinois, United States9h ago
-
Senior/Staff Software Engineer - Perception & Sensing USD 195K-280K3D Object Detection | 3D segmentation | A/B | A/B Testing | B testingSenior-level Full TimeFoster City, CA9h ago
-
AI Engineer USD 103K-140KAI Agents | AI Studio | Access Control | Anthropic Claude | AuthenticationBonus eligibleSenior-level Full TimeDenver, CO, United States10h ago
-
Staff Software Engineer, GenAI Platform USD 208K-250KAPI | AWS EKS | Access Control | Agent Orchestration | Audit LoggingCatered lunches | Cultural and team offsites | Employee giving match | Flexible work schedule | Generous vacation policySenior-level Full TimeSan Francisco, CA, United States10h ago
-
Analytics Engineer USD 95K-115KAirflow | DBT | Dagster | Data Governance | Data IntegrityBackground check compliant | Hybrid work | In office once every 2 weeks | Industry complianceMid-level Full TimeChicago, Illinois, United States13h ago
-
Robotic Orchestration Platform Software Engineer USD 125K-250KAWS | Agile | Azure | C# | CI/CDAgile environment | Remote supportSenior-level Full TimeSan Francisco, California13h ago
-
A/B | A/B Testing | B testing | Data Pipelines | Fine Tuning401k retirement plan | Health insurance | Meal allowance | Paid flexible holidays | Paid parental leaveSenior-level Full TimeNew York, NY14h ago
-
Senior Data Engineer USD 111K-124KAccess Control | Agile | Azure | CI/CD | Data Governance401k contributions | Education assistance | Life and disability coverage | Medical, dental, and vision coverage | Paid sabbaticalSenior-level Full TimeAtlanta, Georgia or Gainesville, FL14h ago
-
Senior-level Full TimeBoston, Massachusetts, United States14h ago
-
Data Analytics & Engineering Opportunities USD 65K-105KHive | Microstrategy | MySQL | Oracle | Python401k with firm profit share | Dental insurance | Disability insurance | Firm paid holidays | Flexible spending accountEntry-level Full TimeWashington, DC, United States14h ago
-
Analytics Engineer USD 60K-75KDBT | Data Architecture | Data Modeling | Data Pipelines | Data Quality401k employer match | Medical/Dental/Vision | Paid Holidays | Paid parental leave | SabbaticalMid-level Full TimeBoulder, Colorado, United States15h ago
-
Mid-Level Data Engineer USD 90K-98KAPI Development | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageRemote workMid-level Full TimeWork from home, VA, United States R15h ago
-
Senior Data Engineer USD 165K-180KAPIs | Anomaly Detection | Azure | Azure Data | Azure Data FactorySenior-level Full TimeWork from home, VA, United States R15h ago
-
Analytics Engineer USD 110K-120KDBT | Data Modeling | Data Warehousing | Documentation | GitCareer growth | MentorshipMid-level Full TimeLehi, Utah16h ago
-
Sr. Machine Learning Engineer USD 100K-160KCI/CD | Data Fusion | Data analytics | Deep learning | DockerHybrid work environmentSenior-level Full TimeCocoa Beach, Florida, United States17h ago