Machine Learning Engineering Manager - Evaluations

Tasks

• Build automated evaluation metrics
• Build evaluation infrastructure
• Coach machine learning team
• Connect user signals to model improvements
• Create quality metrics
• Define technical strategy
• Design human evaluation pipelines
• Ensure ML systems reliability
• Guide model deployment
• Maintain evaluation systems
• Monitor safety systems
• Partner with cross-functional teams
• Perform competitive benchmarking
• Run red teaming

Perks/Benefits

• Equity packages
• Flexible leave options
• Home office setup support
• Inclusive parental leave
• Vibe and Thrive allowance

Skills/Tech-stack

Benchmarking | Data Pipelines | Diffusion Models | Evaluation | GAN | Generative Models | Human-in-the-loop | LLM | MLOps | Machine Learning | Machine Learning Infrastructure | Model Serving | Monitoring | Red Teaming | Safety monitoring | Systems Design | The Loop | Variational Autoencoder