Staff Software Engineer, Vertex GenAI Evaluation Service
Tasks
- Build scalable systems for evaluation workflow
- Collaborate with model and agent quality teams
- Design evaluation infrastructure for model and agent quality
- Develop strategies to measure quality at scale
- Investigate usability gaps and customer pain points
- Lead evaluation frameworks and tools for GenAI developers
Perks/Benefits
- N/A
Skills/Tech-stack
Cloud infrastructure | Computer Vision | Data Processing | Debugging | Distributed Systems | Enterprise Security | Fine Tuning | GenAI | Information Retrieval | LLM Evaluation | Language Models | Language Processing | Large Language Models | Machine Learning | Model Deployment | Model Evaluation | Natural Language | Natural Language Processing | Production Systems | Scalable Production Systems | Security | Software Architecture
Education
Roles
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R13d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R27d ago
-
Senior Staff Software Engineer, App Hub USD 262K-365KAlgorithms | Artificial Intelligence | Back-end systems | Big Data | C++Senior-level Full TimeSunnyvale, CA, USA3h ago
-
Senior Staff Software Engineer, AI/ML USD 262K-365KAccelerated inference | Android | Android on device ML | Artificial Intelligence | C++Benefits | Bonus | EquitySenior-level Full TimeSunnyvale, CA, USA3h ago
-
Smart NIC Software Engineer, Embedded USD 147K-211KAI machine learning | C# | C++ | Data center | Data center networkingBonus target | Equity | Health insurance | Paid time off | Retirement benefitsMid-level Full TimeDurham, NC, USA; Raleigh, NC, USA3h ago
-
Computer Vision | Data Processing | Data Structures | Data structures algorithms | Distributed Computing401k match | Health insurance | Paid time offSenior-level Full TimeMountain View, CA, USA3h ago
-
Senior Embedded Software Engineer, Microcontroller OS USD 174K-253KBSP | BSP Development | Bazel | C++ | CMakeSenior-level Full TimeSan Francisco, CA, USA; San Jose, …3h ago
-
Staff Software Developer, AI/ML, Safety and Security USD 207K-301KComputer Vision | Data Processing | Debugging | Fine Tuning | Language ModelsSenior-level Full TimeWaterloo, ON, Canada; New York, NY, …3h ago
-
Data Engineer, GCS Data Science USD 106K-151KBig Data | C plus plus | Data Documentation | Data Modeling | Data ValidationEntry-level Full TimeNew York, NY, USA; Mountain View, …3h ago
-
AOSP | Abstraction layer | Algorithms | Android Hardware Abstraction Layer | Android hardwareSenior-level Full TimeMountain View, CA, USA3h ago
-
Agent Engine | Agentic Development Kit | Agentic development | Cloud platform | Computer VisionSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Senior Software Engineer, AI/ML GenAI, Google Cloud USD 174K-253KC++ | Computer Vision | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Cloud platform | Computer Vision | Data Processing | Debugging | Distributed ComputingSenior-level Full TimeSunnyvale, CA, USA; New York, NY, …3h ago
-
Senior Data Engineer USD 170K-200KApache Airflow | Apache Spark | ClickHouse | Data Contracts | Data Modeling401k | Catered lunches | Commuter benefits | Continued education benefit | Equity compensationSenior-level Full TimeNew York, New York, United States8h ago
-
Senior Data Engineer USD 170K-200KApache Airflow | Apache Spark | Data Contracts | Data Modeling | Data Quality401k | Catered lunches | Commuter benefits | Company wellness day off | Continued education benefitSenior-level Full TimeLos Angeles, California, United States8h ago
-
Senior Data Engineer USD 170K-200KApache Airflow | Apache Spark | Data Contracts | Data Modeling | Data Quality401k | Catered lunches | Commuter benefits | Continued education benefit | FSASenior-level Full TimeSan Francisco, California, United States8h ago
-
AI Engineer USD 113K-180KAngular | Apache Kafka | CI/CD | Context Management | DevOps401k matching | Charitable matching | Community outreach groups | Flexible paid time off | Life insuranceMid-level Full TimeLeawood, KS, US R9h ago
-
Sr. Embedded Software Development Engineer, Ring Team USD 193K-261KC# | C++ | Debugging | Diagnostic tooling | Embedded Linux401k matching | Employee assistance program | Flexible spending accounts | Health insurance | Paid time offSenior-level Full TimeSunnyvale, California, USA14h ago
-
Senior Data Science Engineer USD 116K-198KApache Airflow | CI/CD | DBT | Databricks | Databricks AutoloaderFlexible work optionsSenior-level Full TimeRemote (United States) R14h ago
-
ML Data - Research Scientist USD 143K-264KAudio signal processing | Computer Vision | Data Annotation | Experiment design | Inter-rater reliabilityMid-level Full TimeCupertino15h ago
-
APIs | Behavior Trees | C++ | Dashboards | Data PipelinesSenior-level Full TimeIrvine, CA15h ago
-
Machine Learning Engineer, Local Search & Marketplace USD 190K-300KA/B | A/B Testing | Apache Spark | B testing | C++Senior-level Full TimeMountain View, California, United States15h ago
-
Machine Learning Engineer, Local Search & Marketplace USD 190K-300KA/B | A/B Testing | Algorithms | B testing | C++Senior-level Full TimeMountain View, California, United States15h ago
-
Member of Technical Staff, Applied AI Research USD 203K-299KAgentic technology | Apache Cassandra | Apache Kafka | Caching | Capacity PlanningSenior-level Full TimeSan Francisco, CA; Sunnyvale, CA15h ago
-
AI Data Engineer USD 133K-258KAWS | Agile | Apache Kafka | Apache Spark | AzureAnnual bonus eligibility | Health and wellness benefits | Hybrid work schedule | Paid time off | Retirement planSenior-level Full TimeBaltimore, Maryland, United States16h ago