LLMOps Engineer
Tasks
- Build and run eval suites
- Build evals from instrumented data
- Decompose products into success and failure metrics
- Maintain cost effective evals
- Measure AI systems honestly
- Own reporting loop
- Run scheduled regression testing
- Steward core datasets
- Surface model failures before learners
Perks/Benefits
Skills/Tech-stack
Classification | Data Analysis | Data Instrumentation | Dataset Management | Dspy | Langfuse | Language Processing | Machine Learning | Natural Language | Natural Language Processing | Python | Ragas | Regression testing | Retrieval-Augmented Generation
Education
N/A
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R25d ago
-
Consultant Robotics & Embodied AI (m/f/d) EUR 66K-80KC++ | Cloud Platforms | Computer Vision | Data analytics | DevOpsCompany pension scheme | Employee discounts | Fitness offerings | Flexible working hours | Freely choose place of residence anywhere in GermanyEntry-level Full TimeBerlin, Deutschlandweit, Düsseldorf, Frankfurt, Hamburg, München, … R4h ago
-
Senior AI Engineer-1 PLN 242K-242KAWS | Agentic Systems | Agile | Amazon SageMaker | AzureCompany car provided | Employer supported pension plan | Extra paid holidays | Flexible benefits | Home-office allowanceSenior-level Full TimeWarszawa, Mazowieckie, PL R4h ago
-
Senior Machine Learning Engineer EUR 66K-88KAWS | Data Pipelines | Databricks | Distributed Computing | Ensemble ModelsCommunity days | Complete laptop workstation | Global Lifestyle Reimbursement Account | Life/risk assurance | Meal allowanceSenior-level Full TimeHybrid - Lisbon, Portugal R7h ago
-
Senior-level Full TimeNoida, India R14h ago
-
Senior Analytics Engineer USD 66K-66KAirflow | Business Intelligence | Code review | DBT | DagsterEquity | Flexible time off | MacBook equipment | Paid Holidays | Remote-friendly workSenior-level Full TimeBrazil R15h ago
-
AWS | Azure | Cloud platform | Direct Preference Optimization | Fine TuningComprehensive benefits package | Remote work | Travel to enterprise customers | Visa sponsorship possibleMid-level Full TimeSan Mateo R17h ago
-
Lead AI Engineer- GTM Systems USD 135K-175KAWS Bedrock | AWS Lambda | AWS SNS | AWS SQS | Amazon EventBridgeRSU grant eligible | Remote work opportunitySenior-level Full TimeRemote - Canada R17h ago
-
Lead AI Engineer- GTM Systems USD 131K-176KAWS Bedrock | AWS Lambda | Agent coordination | Async event-driven architecture | Async/event-drivenNo relocation assistance not provided | Remote workSenior-level Full TimeRemote - US R17h ago
-
Lead AI Engineer CAD 131K-211KAgentic Frameworks | CI/CD | Cloud Platforms | Containers | Fine TuningSenior-level Full TimeToronto, Canada R17h ago
-
Lead AI Engineer USD 198K-261KAgentic Frameworks | CI/CD | Cloud Computing | Containers | Fine TuningSenior-level Full TimeUSA, USA R17h ago
-
A/B | A/B Testing | AWS | AWS SageMaker | Apache Spark401k employer match | Commuter benefit | Dental insurance | Dependent Care Flexible Spending Account FSA | Flexible time offSenior-level Full TimeLos Angeles, California, United States; San … R19h ago
-
Senior Data Engineer USD 145K-200KAWS | Apache Airflow | Cloud platform | Data Engineering | Data Infrastructure100 percent employer paid medical premiums | Self-managed PTOSenior-level Full TimeRemote R19h ago
-
Senior Machine Learning Engineer USD 145K-250KDeep learning | Feature Engineering | Machine Learning | Model Training | Neural NetworksEmployer-paid medical premiums | Remote work | Self-managed PTOSenior-level Full TimeRemote R19h ago
-
Agent Orchestration | Agentic AI | Artificial Intelligence | Autogen | Autonomous ReasoningSenior-level Full TimeRemote-USA R19h ago
-
Senior Data Scientist, Applied ML USD 154K-200KAWS | Airflow | Apache Spark | DVC | Data Transformation401k employer contribution | Employer-paid life insurance | Flexible Remote Friendly Work Options | Generous paid holidays | Health Savings Account employer contributionSenior-level Full TimeAustin, Texas | Remote R19h ago
-
Ego-Centric视觉感知算法工程师 CNY 180K-300K2D Segmentation | 3D Gaussian Splatting | 3D Hand Pose | 3D Reconstruction | 6D PoseMid-level Full Time上海、北京 R19h ago
-
Senior AI/ML Engineer USD 130K-210KBayesian Knowledge Tracing | Causal Inference | Data Pipelines | Experiment tracking | Knowledge Space TheoryCareer development | Certification support | Collaborative team environment | Remote work flexibilitySenior-level Full TimeRemote R19h ago
-
AI Engineer EUR 35K-46KAgent Frameworks | Application development | Context engineering | Eval Loops | FastAPICertification developmentEntry-level Full TimeRemote R19h ago
-
Senior AI Engineer COP 95420K-101090KAWS | Amazon Textract | Confidence scoring | Data Validation | Document AISenior-level Full TimeColombia R19h ago
-
Staff Engineer Tech Lead Manager, ML Acceleration USD 172K-229KAWS | Azure | C++ | Cloud Computing | Distributed Systems401k match | Dental insurance | Health savings account | Life insurance | Medical insuranceSenior-level Full TimeRemote U.S. R20h ago
-
AWS ECS | AWS ECS Fargate | Agentic tools | Anthropic Claude | Async pipelinesAsync communication | Continuous learning | High ownership | Remote workMid-level Full TimeBrazil (Remote) R21h ago
-
Senior-level Full TimeKitchener, Canada R21h ago
-
Sr Machine Learning Engineer I USD 150K-241KAutomated retraining | Bayesian Filter | CI/CD | Data Association | Extended KalmanDiscretionary paid time off | Emotional and mental wellness support | Employee resource groups | Fitness programs | Learning and development programsSenior-level Full TimeSeattle, Washington, United States R22h ago
-
AWS | Apache Kafka | Apache Spark | Azure | Cloud platformSenior-level Full TimeFlorida; Remote - Massachusetts; Remote - … R22h ago