Senior AI Scientist
Durham, North Carolina, United States of America
USD 108K-270K Senior-level Full Time
Tasks
- Assess model bias fairness safety explainability
- Build automated evaluation pipelines
- Build human in the loop evaluation
- Build simulation based evaluation environments
- Collaborate with cross-functional teams
- Conduct end to end pipeline evaluation
- Create audit ready evaluation frameworks
- Define evaluation metrics
- Design LLM evaluation frameworks
- Develop agent evaluation strategies
- Develop prompt and version tracking systems
- Enable observability and traceability
- Ensure responsible AI compliance
- Establish evaluation standards
- Evaluate RAG systems
- Evaluate multi agent task success
- Implement LLM benchmarking
- Implement offline online evaluation
- Implement production monitoring
- Integrate evaluation with enterprise AI platforms
- Measure tool usage accuracy
- Mentor junior team members
- Mitigate privacy and data leakage risks
- Monitor model performance and drift
- Run A/B testing
- Set up canary releases
Perks/Benefits
- N/A
Skills/Tech-stack
A/B | A/B Testing | Agent systems | Agentic AI | Artificial Intelligence | B testing | Bias Mitigation | Canary Releases | Data Privacy | Drift Detection | Embeddings | Evaluation metrics | Experiment Management | Experimental Design | Fairness | Knowledge graphs | LLM Evaluation | Langchain | Langgraph | Langsmith | Language Models | Language Processing | Large Language Models | Machine Learning | Model Drift | Model Drift Detection | Model Monitoring | Multi-Agent | Multi-Agent Systems | Natural Language | Natural Language Processing | OpenTelemetry | Privacy Protection | Prompt engineering | PyTorch | Python | RAG | Responsible AI | Retrieval-Augmented Generation | Semantic Kernel | TensorFlow | Tool use | Vector Databases
Education
Roles
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Data Scientist USD 80K-157KAWS | Airflow | Annotation | Azure | ClassificationHybrid work mode | Professional development opportunitiesMid-levelGeorgetown University: Main Campus: Walsh School …20d ago
-
Manager of Applied AI Architecture, Startups USD 315K-380KAI architecture | API Integration | Agentic Architectures | Context engineering | Developer ToolingSenior-level Full TimeSan Francisco, CA9h ago
-
AWS Amazon Connect Agentic AI Engineer USD 153K-227KAWS CDK | AWS Lambda | Amazon Bedrock | Amazon CloudWatch | Amazon ConnectMid-level Full TimeUnited States - Remote R9h ago
-
Sr. Data Scientist, Product USD 145K-203KAmplitude | Apache Airflow | DBT | Data Integrity | Data Quality401k match | Family-forming benefits | Financial Planning Sessions | Flexible time off | Free snacks and refreshmentsSenior-level Full TimeAustin, Texas, United States10h ago
-
AI Research Scientist- World Model USD 165K-185K3D Reconstruction | Auto-labeling | C plus plus | Computer Vision | Data Curation401k matching | Health insurance | Life and disability protection | Paid time off | Parental leaveMid-level Full TimeSunnyvale, CA, United States11h ago
-
Senior Data Scientist USD 137K-237KAnomaly Detection | AutoEncoder | Betweenness centrality | Boosted Trees | Cluster detection401k employer contribution | Bereavement leave | Company laptop | Flexible work hours | HSA and FSASenior-level Full TimeMaryland11h ago
-
Senior-level Full TimeLATAM (Remote), US (Remote) R11h ago
-
Data Scientist USD 200K-330KAPI Integration | Causal Inference | Data Modeling | Data Pipelines | Data Visualization401k | Commuter benefits | Employee assistance program | Free Lunches | Medical, dental, and vision insuranceMid-level Full TimeSan Francisco, CA12h ago
-
Data Scientist, Marketing USD 141K-202KAirflow | Attribution Modeling | BI Dashboards | BigQuery | Cloud DataFree days | Healthcare benefits | Paid Holidays | Paid time off | Pet insuranceMid-level Full TimeUS - San Francisco, United States; …12h ago
-
Data Insights – Tech Manager – IC4 USD 138K-225KAirflow | Amazon SageMaker | Business Intelligence | Customer Health | DBTHybrid work option | Inclusion and fun culture | Mentoring and growth opportunitiesMid-level Full TimeSan Francisco, CA, United States13h ago
-
Mid-level Full TimeDallas, TX, United States13h ago
-
Senior-level Full TimeWashington, D.C.13h ago
-
Lead AI Engineer USD 180K-280KAWS SageMaker | Azure ML | CI/CD | ColBERTv2 | Docker401k match | Childcare benefits | Dental insurance | Disability coverage | Health insuranceSenior-level Full TimeQuincy, MA, United States15h ago
-
AI Engineer USD 128K-170KAgent systems | Agentic Architectures | Amazon Web Services | Azure | DatabricksMid-level Full TimeLATAM (Remote), US (Remote) R16h ago
-
Senior People Data Scientist USD 161K-204KAccess Control | Business Intelligence | Causal Inference | Cohort Analysis | Data ArchitectureSenior-level Full TimeUnited States - Remote R16h ago
-
Senior Data Scientist/Data Analytics - R01565538 USD 108K-158KARIMA | Anomaly Detection | Azure | Backtesting | BentoMLPST time zone availability | Remote workSenior-level Full TimeSeattle, Washington, United States16h ago
-
AI Solutions Specialist USD 128K-200KAI Search | Artificial Intelligence | Azure AI | Azure AI Search | Azure OpenAIHybrid work | Training supportMid-level Full TimeKansas City, MO, United States16h ago
-
Machine Learning Scientist USD 121K-131KDeep learning | Embeddings | Fine Tuning | Language Models | Language Processing401k matching | Flexible spending accounts | Medical/Dental/Vision insurance | Paid parental leave | Paid sick daysMid-level Full TimeNew York, NY17h ago
-
Head of Global Workplace GenAI - Remote USD 163K-237KAI Governance | Anthropic Claude | Change Management | ChatGPT enterprise | Enterprise RoadmapGlobal travel | Remote workExecutive-level Full TimeCosta Mesa, CA, United States R17h ago
-
AI Engineer 1 USD 121K-143K3D simulation | AI Assisted Development | AWS | C# | C++401k match | Dental insurance | Holidays | Medical insurance | Paid time offEntry-level Full TimeWashington, DC17h ago
-
AI Architect - Business Applications USD 138K-225KA/B | A/B Testing | Access Control | Agent Frameworks | B testingCareer growth opportunities | Hybrid work optionSenior-level Full TimeSan Francisco, CA, United States17h ago
-
Senior AI Engineer - AI & Credit Analytics USD 135K-215KAWS | CI/CD | Cloud Computing | Data Governance | Generative AIRemote workSenior-level Full Time,, ,, United States R17h ago
-
Senior Applied Scientist USD 136K-165KAudio Processing | C++ | Cloud Computing | Continuous Deployment | Dialog Systems401k retirement plan | Dental insurance | Diversity and inclusion programs | Employee referral bonus | Employee resource groupsSenior-level Full TimeSeattle, WA17h ago
-
Data Scientist USD 86K-176KAI Governance | Cybersecurity | Data Engineering | DevSecOps | Entity ResolutionSenior-level Full TimeArlington, VA18h ago
-
Data Scientist USD 116K-243KAI Governance | DevSecOps | Entity Resolution | Explainable AI | Geospatial AnalyticsSenior-level Full TimeArlington, VA18h ago