Staff Software Engineer, Vertex GenAI Evaluation Service
Tasks
- Build scalable systems for evaluation workflow
- Collaborate with model and agent quality teams
- Design evaluation infrastructure for model and agent quality
- Develop strategies to measure quality at scale
- Investigate usability gaps and customer pain points
- Lead evaluation frameworks and tools for GenAI developers
Perks/Benefits
- N/A
Skills/Tech-stack
Cloud infrastructure | Computer Vision | Data Processing | Debugging | Distributed Systems | Enterprise Security | Fine Tuning | GenAI | Information Retrieval | LLM Evaluation | Language Models | Language Processing | Large Language Models | Machine Learning | Model Deployment | Model Evaluation | Natural Language | Natural Language Processing | Production Systems | Scalable Production Systems | Security | Software Architecture
Education
Roles
Regions
Countries
States
Cities
Related jobs
-
Applied AI Lead USD 140K-160KAI Agents | AWS | AWS Bedrock | Anthropic Claude | Automation401k match | Cell phone stipend | College savings | EAP | FSA optionsSenior-level Full TimePhiladelphia, PA, United States6h ago
-
AI Research Engineer – Agentic AI USD 165K-180KAgent systems | Evaluation | Experiment design | Language Models | Large Language Models401k matching | Disability insurance | Health insurance | Life insurance | Paid time offMid-level Full TimeSunnyvale, CA, United States7h ago
-
AI/Machine Learning Engineer - Python - Loops USD 130K-150KAPI Development | CI/CD | Data Pipelines | Debugging | Embeddings401k | Community volunteering events | Dental insurance | Disability benefits | Flexible paid time offEntry-level Full TimePalo Alto, California, United States7h ago
-
AI/ML Engineer USD 100K-150KAPI Design | Asynchronous programming | Claude Code | Cloud platform | Cursor401k | Employee assistance program | Health spending account | Life insurance | Long-term disabilityMid-level ContractDearborn, Michigan, USA R9h ago
-
Cloud Software Engineer III USD 180K-250KAnsible | Apache Superset | CentOS | Elasticsearch | GrafanaSenior-level Full TimeAnnapolis Junction, MD; Laurel, MD; Annapolis …12h ago
-
Software Engineer 2 - (Python, LLM, Machine Learning) USD 201K-246KAPI Integration | AWS | Agentic Workflows | Azure | CrewAI401k match | Educational assistance | Federal Holidays | Healthcare coverage | Paid time offMid-level Full TimeAnnapolis Junction, Maryland R13h ago
-
Code-Review and the Applicability of LLM Graduate Assistant (Software Eng. and Data Mining with Dr. Rahman) USD 38K-46KC++ | Data Mining | GitHub | Java | Machine LearningEntry-level Part TimeErie, Pa15h ago
-
Analytics and Insights Engineer II USD 84K-140KAlgorithm Development | DAX | Databricks | Entity Resolution | Language ProcessingEntry-level Full TimeRosslyn, Virginia, United States16h ago
-
Palantir AI and Data Science Engineer USD 84K-121KAgile | Cloud Computing | Data Engineering | Data Modeling | Data VisualizationEntry-level Full TimeHuntsville, Alabama, United States16h ago
-
Apache Flink | Apache Hive | Apache Iceberg | Apache Spark | Batch ProcessingDevelopment workshops | Hands-on experience | Industry exposure | Internship program | Social eventsEntry-level InternshipSan Jose, California, United States17h ago
-
Apache Flink | Apache Hive | Apache Spark | Batch Processing | C++Development workshops | Hands-on experience | Industry exposure | Learning programs | Social eventsEntry-level InternshipSan Jose, California, United States17h ago
-
Senior Software Engineer, AI/ML, YouTube USD 174K-252KC++ | Data Processing | Data Structures | Data Structures and Algorithms | DebuggingSenior-level Full TimeMountain View, CA, USA17h ago
-
Senior Software Engineer, AI/ML, Gen AI, Geo USD 174K-252KComputer Vision | Data Processing | Data Storage | Data Structures | Data structures algorithmsSenior-level Full TimeMountain View, CA, USA17h ago
-
Senior Software Engineer, Agentic AI, AIR USD 174K-252KComputer Vision | Data Processing | Data Structures | Data structures algorithms | DebuggingSenior-level Full TimeMountain View, CA, USA; New York, …17h ago
-
Cause analysis | Computer Vision | Data Processing | Debugging | Distributed SystemsSenior-level Full TimePittsburgh, PA, USA; Mountain View, CA, …17h ago
-
Algorithms | C# | C++ | Code review | Computer VisionSenior-level Full TimeSan Jose, CA, USA; San Francisco, …17h ago
-
C++ | Computer Vision | Data Processing | Debugging | Distributed ComputingSenior-level Full TimeBoulder, CO, USA18h ago
-
ATA | Algorithms | C# | C++ | Data StructuresSenior-level Full TimeSunnyvale, CA, USA18h ago
-
Staff Software Engineer, AI/ML, Search Ads USD 207K-300KAgentic AI | C++ | Data Processing | Data Quality | Deep learningSenior-level Full TimeMountain View, CA, USA18h ago
-
Agent systems | Apache Beam | Artificial Intelligence | Autonomous Agents | Autonomous ReasoningMid-level Full TimeLondon, UK; Mountain View, CA, USA18h ago
-
Software Engineer, AI/ML, Full Stack, Search Ads USD 147K-210KA/B | A/B Testing | Agentic AI | B testing | C++Mid-level Full TimeMountain View, CA, USA18h ago
-
Staff Software Engineer, Network Health USD 207K-300KAnomaly Detection | Automated remediation | Data Processing | Debugging | Distributed ComputingSenior-level Full TimeSunnyvale, CA, USA18h ago
-
Customer Engineer, Databases, Google Public Sector USD 152K-221KAnalytical workloads | Cloud Computing | Cloud Native | Cloud Native Architecture | Cloud platformSenior-level Full TimeReston, VA, USA; Washington D.C., DC, …18h ago
-
Novel AI Lead Methodologist USD 171K-247KAutomation | Data Analysis | Data Manipulation | Experiment design | Language ProcessingSenior-level Full TimeWashington D.C., DC, USA; Atlanta, GA, …18h ago
-
AI machine learning | Algorithms | C++ | Data Analysis | Data StructuresSenior-level Full TimeRaleigh, NC, USA; Durham, NC, USA18h ago