Staff Software Engineer, Gemini Evals, GenAI, DeepMind
Mountain View, CA, USA; New York, NY, USA
USD 207K-300K Senior-level Full Time
Tasks
- Advocate system design practices
- Advocate testing practices
- Build LLM agent evaluation abstractions
- Create observability dashboards
- Design distributed evaluation execution engines
- Design error classification
- Develop LLM as a judge rating systems
- Implement automated retry policies
- Maintain SLOs for evaluation pipelines
- Mentor engineers
- Optimize inference orchestration
- Partner with research scientists and data science teams
- Set code quality standards
Perks/Benefits
- N/A
Skills/Tech-stack
Agent evaluation | Automated Retry | Distributed Systems | Error classification | Inference | LLM Agent | LLM Agent Evaluation | LLM-as-a-Judge | Observability | Python | Service Level | Service-Level Objectives | Test-Driven | Test-Driven Development
Education
Roles
Regions
Countries
States
Related jobs
-
Production Engineer (University Grad) USD 182K-200KAI tools | API | Agent Orchestration | C++ | CDNTraining and development | Work authorization sponsorshipSenior-level Full TimeSunnyvale, CA | New York, NY3h ago
-
Software Engineer, SystemML - AI Networking USD 170K-251KC# | C++ | CUDA | Data-parallel | Deep learningMid-level Full TimeMenlo Park, CA3h ago
-
Senior Software Engineer, Applied AI, Commerce USD 174K-252KC++ | Deterministic testing | Gemini | Language Models | Language ProcessingSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA3h ago
-
Code Safety | Data Processing | Debugging | Distributed Systems | Fine TuningSenior-level Full TimeMountain View, CA, USA3h ago
-
Senior Software Engineer, Generative AI, Core ML USD 174K-252KAgentic Applications | Automated testing | Computer Vision | Data Processing | DebuggingSenior-level Full TimeMountain View, CA, USA3h ago
-
APIs | Agent systems | Cloud platform | Cost Per Request | CrewAIEmployee discounts | Health insurance | Paid time off | Retirement plansMid-level Full TimeSan Francisco, CA, USA; Atlanta, GA, …3h ago
-
Forward Deployed Engineer III, Google Cloud, Applied AI USD 174K-252KAPIs | Agent systems | Cloud platform | Conversational AI | DebuggingTravel opportunitiesSenior-level Full TimeSan Francisco, CA, USA; Atlanta, GA, …3h ago
-
Artificial Intelligence | Automated Evaluation | Behavior analytics | C# | C++Senior-level Full TimeMountain View, CA, USA3h ago
-
Mechanical Engineer, Data Center Technology Systems USD 171K-248KASME B31 | C++ | Fire Protection | Fuel Systems | HVACSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Software Engineer III, Infrastructure/Cloud Storage USD 147K-211KC++ | Cloud Computing | Code review | Debugging | Distributed SystemsSenior-level Full TimeSeattle, WA, USA3h ago
-
Artificial Intelligence | Documentation | Gemini | Hugging Face | JAXMid-level Full TimeMountain View, CA, USA3h ago
-
Forward Deployed Engineer II, Applied AI, Google Cloud USD 147K-211KAPI Integration | APIs | Agent systems | Cloud platform | Evaluation PipelinesTravel opportunitiesMid-level Full TimeSan Francisco, CA, USA; Atlanta, GA, …3h ago
-
Senior Software Engineer, BigQuery Managed Storage USD 174K-252KBigQuery | C++ | Change Data Capture | Cloud platform | Data CaptureSenior-level Full TimeKirkland, WA, USA3h ago
-
Junior AI Engineer (Open to remote) USD 110K-135KAPI Development | Language Model | Language Model Evaluation | Language Models | Language Processing401k | Dental insurance | Health savings account | Medical insurance | Paid time offEntry-level Full TimeNew York, NY, US, NY 10019 R7h ago
-
Senior Data Platform Engineer, Remote USD 135K-180KAWS | AWS Lambda | Access Control | Amazon Aurora | Amazon CloudWatchSenior-level Full TimeUnited States, UNITED STATES, United States R9h ago
-
Robotics Application Engineer USD 100K-300KAutonomous Systems | C++ | Computer Vision | Machine Learning | PythonMid-level Full TimePittsburgh, San Mateo9h ago
-
AI Software Engineer USD 181K-270KAWS | CI/CD | Docker | Edge Functions | GitHub CopilotComprehensive benefits | Equity | Learning stipend | Remote-first cultureSenior-level Full TimeUnited States or Canada R13h ago
-
Senior-level Full TimeLos Angeles, CA13h ago
-
Machine Learning Intern USD 80K-90KAWS | Apache Hive | Apache Spark | C++ | Cloud platformFinal evaluation | Mentorship | Performance feedbackEntry-level InternshipMountain View, USA13h ago
-
Forward Deployment AI Engineer USD 120K-150KAgent Orchestration | Autogen | Automation | Cloud services | Data extractionClient site travel | Client-facing role | Hybrid workMid-level Full TimeCleveland, OH or Columbus, OH14h ago
-
Senior-level Full TimeUT, US, 8404314h ago
-
Prompt Engineering Architect USD 100K-150KAgentic Workflows | Embeddings | Evaluation Frameworks | Fine Tuning | Language ModelsSenior-level Full TimeUnited States - Remote R14h ago
-
Robotics Software Engineer USD 100K-150KBehavior Trees | C++ | Concurrent Systems | Embedded Systems | Fault detectionRemote workMid-level Full TimeUnited States - Remote R14h ago
-
Agentic Workflows | Caching Strategies | ES6 | Evaluation Pipelines | Hallucination detectionSenior-level Full TimeSan Francisco, United States R14h ago
-
Senior Data Engineer - Remote - Multiple Levels USD 85K-141KAWS Data | AWS Data Migration | AWS Data Migration Service | AWS Lambda | Airflow401k retirement plan | Dental insurance | Health insurance | Paid Holidays | Parental leaveSenior-level Full TimeHome Office: Tysons, VA, United States R14h ago