Engineering Manager, AI Observability
Tasks
- Architect AI observability platform
- Communicate progress to stakeholders
- Define prompt instrumentation
- Design data collection for ML inputs features predictions
- Develop LLM evaluation frameworks
- Drive AI workload observability strategy
- Evaluate grounding correctness
- Hire grow and mentor engineering team
- Implement human in the loop scoring
- Implement observability by default
- Manage platform roadmap and adoption
- Measure response quality
- Scale solutions for model training online inference agent orchestration
- Track hallucination rates
Perks/Benefits
Skills/Tech-stack
Amazon SageMaker | Artificial Intelligence | Cloud Computing | Data Quality | Data quality monitoring | Distributed Systems | Drift Detection | Human-in-the-loop | Incident Response | LLM Evaluation | Latency monitoring | Machine Learning | Model Evaluation | Model Monitoring | Observability | Prompt Instrumentation | Quality monitoring | Telemetry | The Loop | Vertex AI | Weights and Biases
Education
N/A
Roles
AI | AI Infrastructure Engineer | Engineer | Engineering | Engineering Manager | Infrastructure Engineer | Manager
Regions
Countries
States
Cities
Related jobs
-
Mid-level Full TimeKing George, VA, United States3h ago
-
Data Engineer USD 130K-145KApache Spark | CI/CD | Cloud platform | Containerization | Data GovernancePublic trust clearance support | Remote workSenior-level Full TimeWork from home, VA, United States R3h ago
-
Artificial Intelligence | Competency Mapping | Content Review | Curriculum Development | Data ScienceCross-cultural work experience | Free in-house training | Networking opportunities | Opportunities to develop as a public expert | Remote workSenior-level Part TimeBoston, US7h ago
-
Artificial Intelligence | Data Science | Language Models | Large Language Models | Machine LearningFree training | Networking | Professional development opportunities | Remote workSenior-level Part Timegeorgia, georgia, GE7h ago
-
Research Scientist – TikTok Recommendation(NextGen LLM) - Global Frontier Tech Recruitment Program - 2027 Start (PhD) USD 136K-237KAgentic AI | Computer Vision | Deep learning | Language Models | Language ProcessingEntry-level Full TimeSan Jose, California, United States7h ago
-
Senior Engineer, Big Data USD 90K-140KAmbari | Apache HBase | Apache Hive | Apache Impala | Apache KafkaSenior-level Full TimeUnited States8h ago
-
Senior Software Engineer, Generative AI, Ads Safety USD 174K-252KAI Agents | Algorithms | C++ | Computer Vision | Context learningSenior-level Full TimeMountain View, CA, USA8h ago
-
Senior Staff Software Engineer, AI/ML, Security USD 262K-365KAdversarial Machine Learning | Artificial Intelligence | Cloud Architecture | Cloud Computing | Data PrivacySenior-level Full TimeKirkland, WA, USA; Seattle, WA, USA8h ago
-
Senior Software Engineer, Google.org, Data Analytics USD 174K-252KArtificial Intelligence | Data Structures | Data Structures and Algorithms | Distributed Systems | Integration TestingSenior-level Full TimeSeattle, WA, USA8h ago
-
C++ | Compute Optimization | Deep learning | GPU | JAXSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA8h ago
-
Software Engineer III, Storage for Analytics USD 147K-211KC++ | Data analytics | Databases | Distributed Systems | ExperimentationSenior-level Full TimeKirkland, WA, USA8h ago
-
Software Engineer, ML Fleet Intelligence USD 207K-300KAnomaly Detection | Artificial Intelligence | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA8h ago
-
Product Manager, Local Ads Data Platform USD 163K-237KArtificial Intelligence | Data Analysis | Data Infrastructure | Data Quality | Machine LearningMid-level Full TimeMountain View, CA, USA; Kirkland, WA, …8h ago
-
AI Governance | Agent systems | Architecture | Context engineering | Data SovereigntySenior-level Full TimeChicago, IL, USA; Atlanta, GA, USA8h ago
-
Forward Deployed Engineer II, Applied AI, Cloud USD 127K-183KAPIs | Agent systems | Agentic Workflows | Conversational AI | DebuggingTravel 50% timeSenior-level Full TimeNew York, NY, USA; Atlanta, GA, …8h ago
-
Head of AI GTM, GenMedia, Google Cloud USD 224K-312KAPI Integration | Advertising Technology | Artificial Intelligence | Audio AI | Business DevelopmentExecutive-level Full TimeChicago, IL, USA; Atlanta, GA, USA8h ago
-
Data Engineer, Geo 3P and Developer Data Operations USD 106K-151KArtificial Intelligence | BigQuery | C++ | Cloud Dataflow | Data MiningEntry-level Full TimeMountain View, CA, USA8h ago
-
Senior Software Engineer, Google Cloud Storage USD 174K-252KC++ | Data Structures | Data Structures and Algorithms | Distributed Systems | File systemsSenior-level Full TimeSeattle, WA, USA8h ago
-
Senior Robotics Engineer, Maritime USD 220K-292KAnomaly Detection | C++ | Cameras | Computer Vision | Data AnalysisCommuter benefits | Generous time off | Healthcare benefits | Income protection | Mental health resourcesSenior-level Full TimeCosta Mesa, California, United States17h ago
-
Senior Robotics Engineer, Maritime USD 220K-292KAnomaly Detection | C++ | Cameras | Computer Vision | GazeboCaregiver leave | Commuter benefits | Dental benefits | Fertility coverage | Healthcare benefitsSenior-level Full TimeQuincy, Massachusetts, United States18h ago
-
Director, Head of Data Engineering USD 170K-190KAWS | Apache Airflow | Business Intelligence | DBT | Data Architecture401k match | Catered lunches | Corporate discounts | Dental insurance | Gym reimbursementExecutive-level Full TimeUnion, NJ, US18h ago
-
Software Engineer USD 100K-150KAmazon Simple Queue Service | Amazon Web Services | Bedrock | Data Pipelines | Distributed Systems401k retirement plan | Dental insurance | Health insurance | Unlimited vacation | Vision insuranceSenior-level Full TimeLos Angeles, CA, US19h ago
-
Data Engineer / Azure Fabric Engineer Consultant USD 119K-193KAI Search | Access Control | Automated Monitoring | Azure AI | Azure AI SearchSenior-level Full TimeMason, OH, US19h ago
-
Principal Engineer, Data & ML Platform USD 120K-180KAPIs | Automated testing | Batch Processing | Cloud platform | Data ModelingSenior-level Full TimeScottsdale, AZ19h ago
-
Principle Data Engineer USD 220K-235KAWS | Airflow | BigQuery | Capacity Planning | Compliance401k | Equity | Essential equipment | Flexible PTO | Fully remoteSenior-level Full TimeCleveland, OH R19h ago