Machine Learning Engineer - Evaluation
Tasks
- Audit evaluation for bias
- Build LLM powered evaluation pipelines
- Build RAG pipelines
- Build model fine tuning workflows
- Define evaluation benchmarking infrastructure
- Design and run evaluation experiments
- Detect evaluation drift and regressions
- Own evaluation methodology end to end
- Translate model behavior into product outcomes
Perks/Benefits
Skills/Tech-stack
Benchmarking | Bias detection | Evaluation methodology | Experiment design | Fine Tuning | LLM Evaluation | Language Models | Large Language Models | Machine Learning | Prompt engineering | RAG
Education
N/A
Regions
Countries
States
Cities
Related jobs
-
Senior Data Engineer USD 165K-180KAPIs | Anomaly Detection | Azure | Azure Data | Azure Data FactorySenior-level Full TimeWork from home, VA, United States R15h ago
-
Senior AI Engineer | Sage Home Loans USD 150K-220KAgent Orchestration | Automated Regression | Automated regression testing | Cost Optimization | DPO401k match | Disability insurance | Employee assistance program | Flexible paid time off | Flexible spending accountsSenior-level Full TimeCharlotte, NC R16h ago
-
Solutions Engineer USD 150K-180KAI | Apache Flink | Apache NiFi | Apache Spark | Applied ScienceContinued Career Development | Employee resource groups | Flexible work from home | Generous paid time off | Paid volunteer timeMid-level Full TimeUS-California-Remote, United States R1d ago
-
Digital Technical Specialist (Associate/Sr Associate) - Heathcare Data, Analytics & Automation - Remote USD 120K-171KAI | Automation | Data Transformation | Data integration | EHRDental insurance | Healthcare coverage | Remote work | Travel opportunity | Vision insuranceSenior-level Full TimeChicago - 550 Van Buren, United … R1d ago
-
AI Solutions Engineer, East USD 125K-175KAWS | Azure | Cloud platform | Dspy | Generative AI401k plan | Dental insurance | Medical insurance | Mental wellness support | Parental leaveMid-level Full TimeRemote (New York) R1d ago
-
Senior Data Engineer, Sentinel (Pacific Time Zone) USD 153K-210KAWS | Airflow | Alerting | CI/CD | DatabricksSenior-level Full TimeUnited States R1d ago
-
Senior Data Engineer - Agentic AI Engineering USD 138K-173KAWS | Access Control | Airflow | Azure | DBTSenior-level Full TimeUnited States of America R1d ago
-
Principal Data Engineer USD 152K-190KApache Spark | Artificial Intelligence | CI/CD | Cloud Platforms | Code Coverage401k company match | Dental insurance | Flexible paid time off | Life insurance | Long-term disabilitySenior-level Full TimeDallas, TX - Hybrid (3x in … R1d ago
-
Staff Machine Learning Engineer USD 205K-272KAWS | Active Learning | Azure | CI/CD | Cloud Computing401k match | Dental insurance | Health savings account | Life insurance | Medical insuranceSenior-level Full TimeRemote U.S. R1d ago
-
Staff Machine Learning Engineer USD 205K-272KAWS | Active Learning | Azure | CI/CD | CPU401k match | Dental insurance | Health savings account | Life insurance | Medical insuranceSenior-level Full TimeLas Vegas, Nevada, United States R1d ago
-
Staff Machine Learning Engineer USD 205K-272KAWS | Active Learning | Azure | CI/CD | Cloud Computing401k match | Dental insurance | Health benefits | Health savings account | Life insuranceSenior-level Full TimePittsburgh, Pennsylvania, United States R1d ago
-
AI Platform Engineer USD 119K-258KAI orchestration | API Integration | Azure | Azure Data | Azure Data FactoryOccasional travel | Remote workSenior-level Full TimeBaltimore, Maryland, United States R1d ago
-
Machine Learning Engineer USD 80K-90KAblation Studies | Benchmarking | Deep learning | Evaluation metrics | Experiment designBenefits | On-site roleSenior-level Full TimeFremont, California R2d ago
-
Machine Learning Engineer USD 80K-90KDeep learning | Evaluation metrics | Generalization | Language Models | Large Language ModelsSenior-level Full TimeManteno, Illinois R2d ago
-
Analytics engineering | Artificial Intelligence | Automation | CI/CD | Cloud NativeConnectivity reimbursement | Professional growth opportunities | Technology setup provided | Work from home supportSenior-level Full TimeNew York R2d ago
-
Generative AI Scientist - (Model Risk & Validation) USD 110K-130KAI Platform | AWS | Amazon SageMaker | Apache Spark | Azure401k matching | Insurance | Paid Holidays | Paid family leave | Paid time offEntry-level Full TimeRemote, United States R2d ago
-
Principal Machine Learning USD 120K-220KAI Observability | AWS | AWS Bedrock | Agentic AI | Amazon SageMakerSenior-level Full TimeLivonia, MI, United States R2d ago
-
Technical Delivery Lead USD 94K-164KAgentic AI | Apache Kafka | CI/CD | Cloud Native | Cloud Native ArchitectureBonus eligibility tied to performance | Comprehensive total rewards packageSenior-level Full TimeDayton WFH, United States R2d ago
-
Senior Data Engineer USD 144K-165KAccess Control | Agentic Frameworks | Agile | Automated Deployment | Configuration as Code401k match | Flexible time off | Hybrid work | Lifestyle spending account | Medical/Dental/VisionSenior-level Full TimeHybrid - Denver, United States R2d ago
-
IT Data Scientist USD 102K-152KAPI | Bayesian Inference | CI/CD | Causal modeling | ClassificationAdoption Assistance | Dental benefits | Educational assistance program | Flexible spending accounts | Fully remoteMid-level Full TimeAAO Oak Brook - 2025 Windsor … R2d ago
-
Principal Data Engineer USD 170K-210KAWS | Apache Spark | Batch Processing | Cloud Data | Cloud Data InfrastructureDental insurance | Employer-matched 401k | Flexible paid time off | Health insurance | Remote workSenior-level Full TimeUnited States - Remote R2d ago
-
Algorithms | Analytics | Apache Spark | Data Mining | Distributed Computing401k | Cold Brew | Dental insurance | Disability insurance | EspressoSenior-level Full TimeBoston, MA R2d ago
-
Amazon Redshift | Apache Airflow | DBT | Databricks | Great ExpectationsCollaborative team environment | Remote workSenior-level Full TimeBoston, MA R2d ago
-
Senior AI Engineer USD 187K-215KAgent systems | Agno | AugmentCode | Benchmarks | CI/CDCollaborative work environment | Comprehensive dental insurance | Comprehensive medical insurance | Comprehensive vision insurance | Equity participationSenior-level Full TimeCalifornia R3d ago
-
AI Data Engineer USD 80K-85KAWS | CI/CD | Data Modeling | Data Quality | Data Warehousing401k matching | Employee assistance program | Employee development programs | Flexible work environment | Full remote optionMid-level Full TimeRemote - United States R3d ago