Engineering Manager, AI Observability
Tasks
- Architect AI observability platform
- Communicate progress to stakeholders
- Define prompt instrumentation
- Design data collection for ML inputs features predictions
- Develop LLM evaluation frameworks
- Drive AI workload observability strategy
- Evaluate grounding correctness
- Hire grow and mentor engineering team
- Implement human in the loop scoring
- Implement observability by default
- Manage platform roadmap and adoption
- Measure response quality
- Scale solutions for model training online inference agent orchestration
- Track hallucination rates
Perks/Benefits
Skills/Tech-stack
Amazon SageMaker | Artificial Intelligence | Cloud Computing | Data Quality | Data quality monitoring | Distributed Systems | Drift Detection | Human-in-the-loop | Incident Response | LLM Evaluation | Latency monitoring | Machine Learning | Model Evaluation | Model Monitoring | Observability | Prompt Instrumentation | Quality monitoring | Telemetry | The Loop | Vertex AI | Weights and Biases
Education
N/A
Roles
AI | AI Infrastructure Engineer | Engineer | Engineering | Engineering Manager | Infrastructure Engineer | Manager
Regions
Countries
States
Cities
Related jobs
-
Credit Risk Analytics Manager-Finance USD 115K-169KBacktesting | Benchmarking | CECL | Credit Risk | Credit risk modelingMid-level Full TimeMiami Lakes, FL, United States4h ago
-
Senior Data Engineer USD 120K-135KAPI Gateway | AWS Big Data | AWS Big Data Services | AWS Glue | AWS LambdaSenior-level Full TimeFort Lauderdale, FL, United States4h ago
-
Manager, Master Data Governance USD 120K-145KCoding Structures | Data Cleansing | Data Conversion | Data Governance | Data Management401k | Career opportunities | Dental insurance | Disability insurance | Health insuranceMid-level Full TimeLong Island City, NY, United States5h ago
-
Apptio Target Process | Artificial Intelligence | Automation | Clarity | ConfluenceCareer development and training opportunities | Company retirement match | Comprehensive medical dental vision life insurance disability benefits | Flexible working model | Free counseling supportSenior-level Full TimeUS-Florida-Sunrise8h ago
-
Sr. Data Scientist USD 90K-132KClustering | Component analysis | Dashboards | Data Quality | Data Visualization401k matching | Dental insurance | Disability insurance | Educational growth | Employee discount programSenior-level Full TimeTexas-The Woodlands8h ago
-
AI Machine Learning Engineer USD 88K-176KCNN | Deep learning | LLM | Machine Learning | Machine Learning PipelinesMid-level Full TimeUSA-VA-Chantilly8h ago
-
Computer Vision Scientist, EIARC USD 86K-120KData Visualization | Deep learning | Image Processing | Keras | Machine LearningConference outreach activities | Hybrid work | Mentorship opportunitiesNone Full TimeUnited States-Ohio-Cleveland9h ago
-
Project Manager - Knowledge Graph Group USD 110K-140KArtificial Intelligence | BQL | Data Governance | Data Management | Data ModelingMid-level Full TimeNew York9h ago
-
Project Manager - Knowledge Graph Group USD 110K-140KAI | Automation | Data Governance | Data Modeling | Data integration401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceMid-level Full TimePrinceton9h ago
-
Automation | Distributed Systems | Linux | Machine Learning | Performance AnalysisSenior-level Full TimeSan Jose, California, United States9h ago
-
Machine Learning Engineer Graduate (Global E-commerce Recommendation/Search) - 2026 Start (PhD) USD 118K-187KData Mining | Language Processing | Machine Learning | Natural Language | Natural Language ProcessingEntry-level Full TimeSan Jose, California, United States9h ago
-
(General Hire) Machine Learning Engineer Graduate (Commerce& Content Service & Search Ads) - 2026 Start (BS/MS) USD 113K-250KData Science | Machine Learning | PyTorch | Python | SearchEntry-level Full TimeSan Jose, California, United States9h ago
-
AIGC | Computer Vision | Deepfake Synthesis | Deepfake detection | Ethical AIFlexible work culture | Opportunities for career growthSenior-level Full TimeSan Jose, California, United States9h ago
-
Research Engineer - Ads Integrity USD 136K-205KAI Safety | AIGC | Deepfake Synthesis | Deepfake detection | Generative AICareer growth opportunities | Flexible work culture | Opportunities for open-source contributions | Opportunities for publicationMid-level Full TimeSan Jose, California, United States9h ago
-
Machine Learning Engineer, Commerce Ads USD 177K-266KA/B | A/B Testing | Ads Ranking | B testing | Click Through RateSenior-level Full TimeSan Jose, California, United States9h ago
-
Software Engineer, Ads Machine Learning Data Platform USD 212K-389KData Engineering | Feature Engineering | Feature Store | Machine Learning | PrivacySenior-level Full TimeSan Jose, California, United States9h ago
-
Software Engineer, Ads ML Infrastructure USD 212K-389KData Engineering | Feature Engineering | Feature Store | Infrastructure Engineering | Machine LearningSenior-level Full TimeSan Jose, California, United States9h ago
-
AIGC | Audio Processing | Computer Vision | Deep learning | Information RetrievalMid-level Full TimeSan Jose, California, United States9h ago
-
Debugging | Deep learning | Distributed Systems | Fault Tolerance | GPUEntry-level Full TimeSan Jose, California, United States9h ago
-
Machine Learning Engineer - Inference USD 156K-387KDistributed Systems | Machine Learning | Model Inference | Monitoring | Online InferenceMid-level Full TimeSan Jose, California, United States9h ago
-
GenAI Product Policy Manager USD 158K-228KAI Policy | Adversarial Simulation | Agent Orchestration | Artificial Intelligence | Bias MitigationMid-level Full TimeAustin, TX | Washington, DC10h ago
-
Senior Product Engineer, Machine Learning Accelerators USD 144K-209KBash | Bench testing | Board assembly | Bring-up | Cause analysisSenior-level Full TimeSunnyvale, CA, USA10h ago
-
Software Engineer, Storage Everywhere, Google Cloud USD 147K-211KBack-end | Back-end development | C++ | Cloud Platforms | Data StructuresMid-level Full TimeSunnyvale, CA, USA10h ago
-
Senior Staff Software Engineer, Torch TPU USD 262K-365KAlgorithms | C++ | Cloud Computing | Computer Vision | Data ProcessingSenior-level Full TimeSunnyvale, CA, USA10h ago
-
Senior Engineering Analyst, Trust and Safety, Cloud USD 159K-231KAI Agents | API | Anomaly Detection | Artificial Intelligence | BenchmarkingSenior-level Full TimeWashington D.C., DC, USA10h ago