Senior Applied Scientist
Tasks
- Analyze failure modes and conduct error analysis
- Assess dataset quality and provenance
- Build evaluation pipelines and test suites
- Calibrate evaluators against human judgments
- Collaborate with engineering, product, and data teams
- Create reusable evaluation protocols and infrastructure
- Design and implement evaluation experiments
- Design human evaluation and annotation workflows
- Develop automated evaluators
- Develop benchmarks and evaluation methods
- Own end to end model evaluation
- Produce decision ready evaluation reports
- Review technical work and mentor team
- Stay current with machine learning research
- Translate needs into measurable hypotheses
- Validate data and metrics
Perks/Benefits
- Commuter benefits
- Disability insurance
- Employee stock purchase plan
- Flexible spending accounts
- Health insurance
- Life insurance
- Paid parental leave
- Paid sick leave
- Paid time off
- Retirement plan match
- Tuition N/A
- Volunteer program
Skills/Tech-stack
Automated Evaluation | Benchmark development | Data Quality | Data provenance | Data quality assessment | Error Analysis | Evaluation methodology | Experiment design | Generative AI | LLM-as-a-Judge | Language Models | Language Processing | Large Language Models | Machine Learning | Multimodal Understanding | Natural Language | Natural Language Processing | Nl2sql | Python | Quality Assessment | Retrieval-Augmented Generation | Statistical Analysis | Vision Language Models | Vision-language
Education
Related jobs
-
Data Scientist Associate USD 77K-80KAPIs | Access Control | Artificial Intelligence | Audit Logging | BenchmarkingOnsite work locationMid-level Full TimeUniversity1d ago
-
Cyber Data Scientist, Analytics and Insights USD 102K-171KCloud Computing | Data Modeling | Data Pipelines | Jira | Machine LearningMid-level Full TimeHuntsville, Alabama, United States1d ago
-
AI Research Scientist, Modeling & ToolCalling - MSL USD 150K-260KData Curation | LLM post training | Language Models | Language Processing | Large Language ModelsEntry-level Full TimeMenlo Park, CA1d ago
-
Senior Specialist, Data Science USD 120K-140KAWS | Advanced SQL | Amazon Web Services | Automation | Data GovernanceSenior-level Full TimeKansas City 1, MO, US, 641051d ago
-
Data Management | Data Quality | Databases | Knowledge graphs | Machine LearningSenior-level Full TimeGarching bei München (Munich), DE, 857481d ago
-
Applied Scientist, Amazon Connect Talent USD 142K-223KAlgorithms | C++ | Data Mining | Data Structures | Distributed ComputingFlexible work hours | Paid time off | Parental leaveMid-level Full TimeSeattle, Washington, USA1d ago
-
Applied Scientist, Alexa Connected Devices USD 142K-193KAlgorithms | C++ | Data Mining | Data Structures | Distributed Computing401k matching | Dental insurance | EAP mental health support | Health insurance | Paid time offMid-level Full TimeSeattle, Washington, USA1d ago
-
AI & Machine Learning Vice President USD 120K-220KAWS | Alteryx | Anomaly Detection | Azure | C#401k | Dental insurance | Disability insurance | Health insurance | Hybrid work modelExecutive-level Full TimeUnited States R1d ago
-
Senior Data Scientist USD 159K-201KAnyLogic | Arena | Capacity Planning | Data Validation | Digital Twin401k | Dental insurance | Flexible spending accounts | Health savings account | Medical insuranceSenior-level Full TimePittsburgh, PA1d ago
-
Senior-level Full TimeClearwater, Florida, United States1d ago
-
Data Scientist II USD 159K-222KA B Test | A B test power calculations | A/B | A/B Testing | AirflowSenior-level Full TimeClearwater, Florida, United States1d ago
-
Senior Bioinformatics Scientist USD 118K-148KAWS | Algorithm Design | Computational Complexity | Data Structures | Fusion detection401k benefits | Commuter benefits | Comprehensive medical insurance | Dental insurance | Disability insuranceSenior-level Full TimeUS Remote R1d ago
-
Senior-level Full TimeUnited States1d ago
-
Senior Machine Learning Data Scientist USD 135K-165KAWS | AWS SageMaker | Data Drift | Feature Engineering | Gradient boosting401k | Dental insurance | Health insurance | Paid time off | Vision insuranceSenior-level Full TimeRemote, US R1d ago
-
Consultant – Compliance Data Science & AI USD 98K-122KAPI | AWS | Anthropic | Azure | Dashboards401k match | Cell phone stipend | College savings | EAP | HSA & FSA optionsMid-level Full TimeSan Francisco, CA, United States1d ago
-
Data Scientist - Fraud Risk USD 175K-220KA/B | A/B Testing | B testing | Dashboarding | Machine Learning401k match | Dental insurance | Equity plan | Flexible working | Health and wellness spendingSenior-level Full TimeNew York, NY1d ago
-
Sr Machine Learning Engineer I USD 150K-241KAutomated retraining | Bayesian Filter | CI/CD | Cloud Computing | Data Association401k match | Dental insurance | Emotional & mental wellness support | Employee resource groups | Fitness programsSenior-level Full TimeSeattle, Washington, United States R1d ago
-
AI Engineer - FDE (Forward Deployed Engineer) USD 152K-210KAWS | Agent systems | Apache Spark | Azure | Databricks Intelligence PlatformRemote work | Travel opportunitiesSenior-level Full TimeUnited States1d ago
-
Senior Data Scientist USD 160K-261KData Integrity | Data Modeling | Data Preparation | Data debugging | Machine LearningSenior-level Full TimeUnited States1d ago
-
Senior Data Scientist USD 192K-288KCausal Inference | Data Mining | Data Visualization | Experimental Design | Forecasting401k plan | Domestic travel | Health benefits | Telecommuting | Wellness stipendsSenior-level Full TimeSeattle, WA R1d ago
-
Principal Data Scientist USD 175K-200KAgentic Workflows | Causal Inference | Data Analysis | Data Science | DatabricksSenior-level Full TimeNew York, NY1d ago
-
Principal Data Scientist USD 175K-200KBI Visualization | Causal Inference | Data Analysis | Databricks | Delta LakeSenior-level Full TimeNew York, NY1d ago
-
Senior Data Scientist USD 186K-219KBayesian statistics | Calibration | Claims data | Data diagnostics | Estimation401k company match | Dental insurance | Disability insurance | Flexible time off | Health insuranceSenior-level Full TimeUnited States R1d ago
-
A/B | A/B Testing | Adobe Analytics | Alteryx | Artificial IntelligenceSenior-level Full TimeOH, United States2d ago
-
AWS | Big Data | Data Analysis | Data Modeling | Data ScienceSenior-level Full TimeVienna, VA, United States2d ago