Senior Automation Specialist, AI Operations
Tasks
- Build dataset creation automations
- Build evaluation pipelines
- Collaborate cross-functionally
- Create lightweight operational processes
- Define and track performance metrics
- Define evaluation approaches
- Design gold datasets
- Develop automated evals
- Develop manual evals
- Ensure real world scenario coverage
- Identify next steps and unblock progress
- Implement LLM as judge evaluations
- Improve evaluation data workflows
- Refine datasets evaluations and workflows
- Translate ambiguous problems into evaluation frameworks
Perks/Benefits
- N/A
Skills/Tech-stack
Benchmarking | Data Pipelines | Data labeling | Dataset QA | LLM Evaluation | Language Models | Large Language Models | Machine Learning | Prompt engineering | Python | SQL
Education
N/A
Related jobs
-
Featured Feat. AI Engineer (MTS) USD 160K-300KAPI Development | AWS | Amazon Web Services | Deep learning | FastAPIMentoring | Open source contributions | Remote workMid-levelRemote R11d ago
-
Mid-level Full Time北京 R5h ago
-
Miclaw-端云协同调度专家 (Hybrid AI Architect) CNY 240K-480K5G | API Integration | Classifier Training | Claude 3 | Claude 3 5 APIHybrid workSenior-level Full Time北京 R5h ago
-
AI / LLM Integration Engineer (Remote) INR 1800K-2800KAPI Integration | Anthropic SDK | Async APIs | Benchmarking | Confidence scoring100 percent remote | Collaborative culture | Inclusive team culture | Professional growthMid-level Full TimeMaharashtra, Pune, India R9h ago
-
AI Solution Architect GBP 75K-100KAI Services | API Development | API Gateway | AWS | AWS BedrockBusiness travel | Free eye tests | Glasses Support | Hybrid work schedule | Incentivized certifications and accreditationsSenior-level Full TimeLondon, Birmingham, Manchester, Newcastle upon Tyne, … R11h ago
-
Anthropic Claude | Async Programming | ChatGPT | Claude Code | CodexDirect user impact | Flexible work schedule | Occasional international travel | Work from home optionMid-level Full TimeSlovenia / Remote R17h ago
-
APIs | Anomaly Detection | Data Modeling | Data Pipelines | Docker100% remote work | Career growth opportunities | Flexible work environmentMid-level Full TimeEstonia R19h ago
-
APIs | Anomaly Detection | Data Modeling | Data Pipelines | Docker100 percent remote work | Autonomous work environment | Career growth | Flexible work environment | International team cultureMid-level Full TimeHungary R19h ago
-
Anomaly Detection | Data Modeling | Data Pipelines | Docker | ForecastingCareer growth | Flexible work environment | Remote workMid-level Full TimeFinland R19h ago
-
Anomaly Detection | Data Modeling | Data Pipelines | Docker | JavaScript100% remote work | Career growth opportunities | Flexible work environmentMid-level Full TimeCzechia R19h ago
-
APIs | Anomaly Detection | Data Modeling | Data Pipelines | Docker100% remote work | Career growth opportunities | Flexible work environment | International team cultureMid-level Full TimeNorway R19h ago
-
API Integration | Anomaly Detection | Data Modeling | Docker | Machine Learning100 percent remote work | Autonomy | Career growth | Flexible work environment | International team cultureMid-level Full TimeLuxembourg R19h ago
-
API Integration | Anomaly Detection | Data Modeling | Data Pipelines | DockerCareer growth | Flexible schedule | International team culture | Remote workMid-level Full TimeCroatia R19h ago
-
APIs | Anomaly Detection | Data Modeling | Data Pipelines | Docker100% remote work | Autonomy | Career growth | Flexible work environment | International team cultureMid-level Full TimeBulgaria R19h ago
-
Anomaly Detection | Data Modeling | Data Pipelines | Docker | Explainable AI100 percent remote work | Career growth opportunities | Flexible work environmentMid-level Full TimeDenmark R19h ago
-
Anomaly Detection | Data Modeling | Data Pipelines | Docker | JavaScript100% remote work | Career growth opportunities | Flexible work environment | International team collaboration opportunitiesMid-level Full TimeGreece R19h ago
-
Anomaly Detection | Data Modeling | Data Pipelines | Docker | JavaScript100% remote work | Career growth opportunities | Flexible work hoursMid-level Full TimeChile R19h ago
-
Anomaly Detection | Data Modeling | Data Pipelines | Docker | JavaScript100% remote work | Career growth | Flexible work environmentMid-level Full TimePoland R19h ago
-
API Integration | Anomaly Detection | Data Modeling | Data Pipelines | DockerCareer growth opportunities | Flexible work environment | Remote workMid-level Full TimeAustria R19h ago
-
API Integration | Anomaly Detection | Data Modeling | Data Pipelines | DockerCareer growth | Flexible work environment | International team collaboration | Remote workMid-level Full TimeSweden R19h ago
-
APIs | Anomaly Detection | Data Modeling | Data Pipelines | DockerCareer growth opportunities | Flexible work environment | Remote workMid-level Full TimeIsrael R19h ago
-
Anomaly Detection | Data Pipelines | Docker | JavaScript | Machine LearningCareer growth opportunities | Flexible work environment | Remote workMid-level Full TimeSaudi Arabia R19h ago
-
APIs | Anomaly Detection | Data Modeling | Docker | JavaScript100% remote work | Career growth opportunities | Flexible work environment | International team cultureMid-level Full TimeBelgium R19h ago
-
Anomaly Detection | Data Modeling | Data Pipelines | Docker | Machine LearningAutonomy | Career growth opportunities | Collaborative team culture | Flexible work environment | Global collaborationMid-level Full TimeUnited Arab Emirates R19h ago
-
Anomaly Detection | Data Modeling | Data Pipelines | Docker | JavaScript100% remote work | Autonomous work environment | Career growth | Flexible work schedule | International team cultureMid-level Full TimeTurkey R19h ago