Senior Automation Specialist, AI Operations
Tasks
- Build dataset creation automations
- Build evaluation pipelines
- Collaborate cross-functionally
- Create lightweight operational processes
- Define and track performance metrics
- Define evaluation approaches
- Design gold datasets
- Develop automated evals
- Develop manual evals
- Ensure real world scenario coverage
- Identify next steps and unblock progress
- Implement LLM as judge evaluations
- Improve evaluation data workflows
- Refine datasets evaluations and workflows
- Translate ambiguous problems into evaluation frameworks
Perks/Benefits
- N/A
Skills/Tech-stack
Benchmarking | Data Pipelines | Data labeling | Dataset QA | LLM Evaluation | Language Models | Large Language Models | Machine Learning | Prompt engineering | Python | SQL
Education
N/A
Related jobs
-
Senior Staff Engineer, PLM, AI ML Integration INR 2500K-4500KAI-enabled | AI-enabled automation | API Integration | Artificial Intelligence | BOM structuresSenior-level Full TimeRemote, India R6h ago
-
API | Airtable | Automation Anywhere | CRM | Google AIGoogle Workspace | Project management tools | Remote work | SlackEntry-level Full TimePakistan - Remote R13h ago
-
Attaccama | Data Mapping | Data Migration | Data Quality | Data ReconciliationCompany-sponsored events | Employee referral program | Flexible working conditions | Further professional development | Meal vouchersMid-level Full TimeCzechia - Remote R13h ago
-
Sr. AI Engineer INR 2500K-4800KAPI Integration | Agent memory | Anthropic API | Autogen | Automation AnywhereSenior-level Full TimeIN Mumbai Remote, India R13h ago
-
AI Agents | AI workflows | API Integration | Automation | DebuggingFast delivery | Fully remote | High ownership | Learning opportunities | MentorshipEntry-level Full TimeBucharest, Romania R19h ago
-
AI Agents | API Integration | APIs | Backend Development | DebuggingCareer growth | Fully remote | High ownership | MentorshipEntry-level Full TimeWarsaw, Poland R19h ago
-
AI Agents | API Integration | Debugging | Performance optimization | PythonFast iteration | Fully remote | High ownership | Mentorship | Work on real production systemsEntry-level Full TimeBelgrade, Serbia R19h ago
-
AI Agents | API Integration | Automation | ChatGPT | ClaudeFast-moving environment | Fully remote | High ownership | MentorshipEntry-level Full TimeBucharest, Romania R19h ago
-
AI Automation | API Integration | Agentic AI | Automation workflows | ChatGPTFlexible work from anywhere | Fully remote | High ownership | Mentorship from experienced AI developers | Opportunity to experiment with AI toolsEntry-level Full TimeBelgrade, Serbia R19h ago
-
AI Agents | API Integration | Anthropic Claude | Automation workflows | ChatGPTFully remote | MentorshipEntry-level Full TimeBogota, D.C., Colombia R19h ago
-
API Integration | Agentic AI | Automation | ChatGPT | ClaudeFast-moving environment | Fully remote | High ownership | Mentorship | Opportunity to experiment with AI toolsEntry-level Full TimeBuenos Aires, Argentina R19h ago
-
AI Engineer GBP 50K-65K.NET | A/B | A/B Testing | Async Python | AzureAnnual leave | Contributory pension | Flexible benefits | Flexible working hours | Group income protectionMid-level Full TimeUnited Kingdom R19h ago
-
Principal Architect, AI/ML USD 180K-250KAgent Builder | Agentic Systems | Agentic Workflows | Artificial Intelligence | Bias MitigationSenior-level Full TimeRemote R20h ago
-
ML Engineer Summer Intern (Remote & Paid) USD 40K-42KArtificial Intelligence | Classification | Data Visualization | Data Wrangling | Generative AI401k participation | Flexible work schedule | Remote work | Volunteer time offEntry-level InternshipUnited States, United States R20h ago
-
Data Analyst & AI Intern USD 59K-67KAudio signal processing | Convolutional Neural Networks | Data Documentation | Data Preprocessing | Data VisualizationDental insurance | Holiday pay | Medical insurance | On-site work | Sick leaveEntry-level Full Time InternshipColumbia, MARYLAND, United States R20h ago
-
AI Solution Architect USD 184K-287KAnsible | CI/CD | Containerization | Docker | Experiment trackingAdditional fully paid sick leave days | Additional vacation days | Allowance for life events and birthdays | Flexible working hours | Free snacks and drinksSenior-level Full TimePoland, Serbia, Lithuania, Cyprus, Georgia R21h ago
-
AWS EventBridge | AWS Lambda | AWS S3 | CI/CD | Data ModelingDEI initiatives | Employee development programs | Fully remote options | Healthcare stipend | Mental health recharge daysMid-level Full TimeRemote - Mexico R21h ago
-
AI Data Engineer USD 95K-124KAWS | AWS Lambda | CI/CD | Data Modeling | EmbeddingsFull Remote or Hybrid Work | Healthcare stipend | National holidays | Paid mental health days | Unlimited time offMid-level Full TimeRemote - Argentina R21h ago
-
AI Data Engineer BRL 150K-240KAWS | AWS EventBridge | AWS Lambda | AWS S3 | Automated testingDEI initiatives opportunities | Fully remote or hybrid work | Healthcare stipend | National holidays | Paid mental health recharge daysMid-level Full TimeRemote - Brazil R21h ago
-
AI Data Engineer COP 42000K-42000KAWS | CI/CD | Data Modeling | Data Quality | Data quality monitoringEAP benefits | Employee assistance program | Employee growth and leadership opportunities | Full remote work | Hybrid workMid-level Full TimeRemote - Colombia R21h ago
-
Head of AI Engineering (M/F/D) EUR 90K-120KArtificial Intelligence | Change Management | Code review | Deployment | Fine TuningFamily care policy | Flexible work environment | Health insurance | Home office equipment support | Lunch vouchersExecutive-level Full TimeParis, IDF, France R23h ago
-
AI Engineer (Hybrid) USD 50K-60KAgent systems | Autogen | CI/CD | ChromaDB | Conversational workflowsHybrid workEntry-level Full TimeGiza, El Omraniya, Egypt R1d ago
-
Technical Lead, AI Initiatives - India INR 2500K-4500KA/B | A/B Testing | APIs | AWS | Agent evaluationHybrid work options | Remote work flexibilitySenior-level Full TimeIndia R1d ago
-
Senior AI Fullstack Engineer USD 147K-203KAI orchestration | AWS Lambda | Asynchronous programming | Deployment | Fine TuningCertifications | Corporate English classes | Courses | Flexible working hours | Hybrid workSenior-level Full TimeBelgrade, Belgrade, Serbia R1d ago
-
(Senior) AI Engineer (all genders) EUR 65K-85KCloud Platforms | Containerization | DevOps | Docker | Language Models30 days vacation | Bike leasing | Company fitness | E-learning reimbursement | Family friendly policyMid-level Full TimeBremen, Munich, Mannheim, Mainz, Berlin, Remote R1d ago