Sr. Staff Machine Learning Systems Engineer
Tasks
- Build and scale data pipelines for ingestion transformation and dataset versioning
- Design adversarial and red team evaluation programs
- Design statistical regression testing methodology
- Implement labeling and calibration workflows
- Lead multi team initiatives for automated evaluation gates
- Mentor engineers and raise technical and statistical bar
- Own evaluation systems end to end
- Rearchitect evaluation and data pipeline systems for reliability
- Resolve cross team technical disagreements and drive alignment
- Set technical direction for evaluation metrics judge and scorer design
- Track and categorize evaluation failures over time
Perks/Benefits
- 401k employer matching
- Company holidays
- Comprehensive health benefits
- Employee stock purchase program
- Offsite team retreats
- Parental leave
- Quarterly mental health days
- Unlimited PTO
Skills/Tech-stack
Adversarial Testing | Calibration | Data Engineering | Data Ingestion | Data Pipelines | Data Science | Data Transformation | Dataset versioning | Evaluation systems | High Throughput | High-Throughput Systems | LLM Evaluation | Labeling workflows | Language Models | Large Language Models | Machine Learning | Multiple Comparisons | Multiple Comparisons Correction | Python | Red team | Red team testing | Regression testing | Significance Testing | Statistical Testing
Education
N/A
Related jobs
-
Senior Machine Learning Engineer, Content Platform USD 141K-360KApache Spark | Cloud Computing | Generative AI | Language Models | Large Language ModelsDisability benefits | Equity awards | Financial wellness support | Health insurance | Life insuranceSenior-level Full TimeSan Jose, California R6h ago
-
Staff Machine Learning Engineer, Ad Serving USD 195K-510KA/B | A/B Testing | B testing | Continuous Improvement | Feature EngineeringCommuter benefits | Disability benefits | Financial wellness support | Health insurance | Life insuranceSenior-level Full TimeNew York, New York R6h ago
-
Staff Machine Learning Engineer, Ad Serving USD 195K-435KDistributed Systems | Experimentation | Feature Engineering | Java | Machine LearningDisability benefits | Equity awards | Health insurance | Life insurance | Paid time offSenior-level Full TimeBoston, Massachusetts R6h ago
-
Machine Learning Engineer, Ads USD 165K-230KA/B | A/B Testing | B testing | Bidding | Conversion PredictionDirect impact | Hybrid work model | International team collaboration | Professional growth opportunities | Remote work flexibilityMid-level Full TimeSan Francisco Bay Area, USA R7h ago
-
Sr. Analytics Engineer USD 160K-185KAirflow | CI/CD | DBT | Dagster | Data ModelingCross-functional collaboration | High autonomy | US Pacific overlapSenior-level Full TimeRemote - US R11h ago
-
AWS | Cloud infrastructure | Compute | Data Curation | Distributed Systems401k match | Daily lunches and snacks | Health insurance covered | Hybrid work schedule | Learning and development stipendSenior-level Full TimeRedwood City R11h ago
-
Staff Robotics Engineer USD 190K-235KAI Agents | AI Libraries | C++ | Critical Systems | FDIRComprehensive employee benefits | Equity | On call rotation for continuous campaignsSenior-level Full TimeRichmond, CA R12h ago
-
Senior Data Engineer USD 130K-170KAmazon Kinesis | Amazon Redshift | Apache Airflow | BigQuery | DBT401k match | Education stipend | Flexible time off | Health insurance | IKEA discountsSenior-level Full TimeSan Francisco, California, United States R12h ago
-
Staff Backend AI Engineer, Remote USD 164K-219KAWS | AWS Cloud | AWS Cloud Development Kit | AWS Lambda | AWS cloud development401k matching | Dental insurance | Flexible time off | Medical insurance | Paid HolidaysSenior-level Full TimeUnited States, UNITED STATES, United States R13h ago
-
Junior AI Engineer (Open to remote) USD 110K-135KAPIs | Agent SDK | Agentic coding | Arize | Google ADK401k | Commuter benefits | Dental insurance | Educational assistance | Medical insuranceExecutive-level Full TimeNew York, United States R13h ago
-
Principal Machine Learning Engineer, Geometric Vision USD 407K-460K3D Computer Vision | 3D representations | Auto-labeling | C++ | CalibrationSenior-level Full TimeSunnyvale R14h ago
-
Media Software Engineer, Speech (Senior-Staff Levels) USD 180K-270KAndroid | Asynchronous I/O | Audio streaming | C# | C++401k retirement savings plan | Company holidays | Complimentary lunch and snacks | Fertility support | Lifestyle spending accountSenior-level Full TimeSunnyvale R14h ago
-
AI/ML Engineer USD 100K-150KAPI Design | Asynchronous programming | Claude Code | Cloud platform | Cursor401k | Employee assistance program | Health spending account | Life insurance | Long-term disabilityMid-level ContractDearborn, Michigan, USA R14h ago
-
Machine Learning Engineer, Detection and Tracking USD 160K-210KByteTrack | DETR | Data Augmentation | DeepSORT | Faster R-CNNFlexible paid time off | Hybrid work | Medical insurance | Paid Holidays | Remote workMid-level Full TimeWashington, DC R15h ago
-
Senior Analytics Engineer II CAD 148K-203KAirbyte | Airflow | Alerting | Amazon Redshift | AutomationHealth and dental benefits | Learning and development opportunities | Unlimited paid time off | Work from anywhereSenior-level Full TimeRemote - US & Canada R15h ago
-
Applied AI Architect, Public Sector (National Security) USD 275K-315KCloud Architecture | Data Science | Enterprise Systems Integration | Enterprise systems | LLM EvaluationFlexible working hours | Generous vacation | Optional equity donation matching | Parental leave | Visa sponsorshipSenior-level Full TimeWashington, DC R17h ago
-
Senior Data & Analytics Specialist USD 108K-130KAzure | CI/CD | Data Governance | Data Lake | Data ModelingHybrid work environment | Training and professional developmentSenior-level Full TimePhoenix, AZ R17h ago
-
Senior Data & Analytics Specialist USD 123K-150KAzure | CI/CD | Data Architecture | Data Governance | Data LakesHybrid work | Reasonable notice of schedule changes | Training and developmentSenior-level Full TimeWestlake Village, CA R17h ago
-
Software Engineer 2 - (Python, LLM, Machine Learning) USD 201K-246KAPI Integration | AWS | Agentic Workflows | Azure | CrewAI401k match | Educational assistance | Federal Holidays | Healthcare coverage | Paid time offMid-level Full TimeAnnapolis Junction, Maryland R18h ago
-
AI/ML | AKS | Audit Logs | Azure DevOps | Azure MonitoringLocal only candidates required | Onsite 2 days per week | Remote 3 days per weekSenior-level Full TimeAustin, TX R18h ago
-
Sr. DevOps Engineer - Snowflake USD 165K-216KADLS | AWS | Azure | Azure Data | Azure Data FactoryAI assisted troubleshooting | Mentorship | Ownership and autonomy | Remote-first cultureSenior-level Full TimeUS - Remote R18h ago
-
Senior Solution Engineer, Manufacturing USD 165K-216KAI code generation | Analytics | Cloud Computing | Code generation | Data ArchitectureSenior-level Full TimeUS-IL-Chicago-MSO R19h ago
-
Senior Machine Learning Operations Engineer USD 144K-181KAPI Development | AWS SageMaker | Agentic AI | Classification | Computer VisionCareer Development Programs | Commuting cost coverage | Daily free lunch | Employee equity | Employee resource groupsSenior-level Full TimeBoston, Massachusetts, United States R21h ago
-
Desktop UI Developer (AI Software) USD 120K-185KAnimation | Asynchronous programming | C++ | Component design | GraphicsAccess to cutting-edge AI tools | Flexible working hours | Remote workMid-level Full TimeGeorgia R22h ago
-
Senior Software Engineer, Perception USD 184K-287K2D Object Detection | 3D Object Detection | C++ | CUDA | CameraSenior-level Full TimeAtlanta - Onsite R1d ago