AI Model Evaluation & Monitoring Specialist
Tasks
- Collaborate with IAOps and MLOps for observability and governance
- Design and implement GenAI testing frameworks
- Evaluate and compare LLM models
- Generate technical recommendations for model selection and cost optimization
- Monitor model changes and impact
- Optimize prompting strategies
Perks/Benefits
- Childcare card
- Collaborative work environment
- Employee discount portal
- Flexible compensation plan
- Gympass
- Hybrid work
- Indefinite contract
- Meal card
- Medical insurance
- Training programs
- Transport card
Skills/Tech-stack
A/B | A/B Testing | AI Governance | AWS Bedrock | AWS Lambda | Amazon CloudWatch | Amazon SageMaker | B testing | DeepEval | GenAI | Language Models | Large Language Models | Logging | Metrics | Model Observability | Prompt engineering | Ragas
Education
N/A
Related jobs
-
AI orchestration | Agent architectures | Agent systems | Anthropic API | EmbeddingsAnnual leave | Fully remote work | Hackathons | Health and wellness benefits | Open source allowanceSenior-level Full TimeSpain R1d ago
-
AI Project Manager EUR 24K-30KAPI Integration | Artificial Intelligence | CRM | Claude | Code AutomationDigital Office Tools Use | Fully remote | Global team collaboration | High autonomy | No MicromanagingMid-level Full TimeMadrid, Spain, ES R2d ago
-
A/B | A/B Testing | Amplitude | B testing | Competitive ResearchAdditional paid leave | Bicycle program | Continuous development | Employee discounts | Flexible work hoursSenior-level Full TimeSpain R4d ago
-
Agent / AI Engineer - Zaragoza, Spain EUR 60K-75KAI Evaluation | Agent Orchestration | Authentication | DevOps | Fallback BehaviorAccelerator environment | Travel opportunities | Visa sponsorshipMid-level Full TimeSpain5d ago
-
Intern Demo Experience & Applied AI EUR 21K-25KArtificial Intelligence | Language Models | Large Language Models | Process Intelligence | Process Intelligence GraphFlexible work arrangement | Hybrid work model | Mental health counseling | Paid volunteer time off | Parental leaveEntry-level InternshipMadrid, Spain5d ago
-
AI Solution Architect GBP 78K-109KAWS | Architecture Diagrams | Artificial Intelligence | Asset Management | AutomationSenior-level Full TimeRemote - Canada; Remote - Portugal; … R5d ago
-
Senior AI Engineer for Revenue & Go-To-Market EUR 56K-79KAI Agents | AI integrations | BigQuery | CRM | ClaudeClassPass subscription | Comprehensive health insurance | Flexible time off | Food or nursery benefits | Referral bonusesSenior-level Full TimeBarcelona, Spain5d ago
-
Senior AI Engineer for People & Talent EUR 60K-80KATS | BigQuery | Claude | Cloud AI | Cloud platformFlexible time off | Food or nursery benefits | Health insurance coverage | Referral bonuses | Tax-free payroll deductionsSenior-level Full TimeBarcelona, Spain5d ago
-
Senior AI Engineer for Finance EUR 56K-80KAI Agents | BigQuery | Cloud platform | Data Governance | ERPComprehensive health insurance | Flexible time off | Food or nursery allowance | Health insurance coverage for family | Referral bonusesSenior-level Full TimeBarcelona, Spain5d ago
-
AI Evaluation | AI Governance | AI systems | AWS | Agent BuilderEU work eligibility | Flexible work hours | Fully remote | Inclusive work environment | Professional growthSenior-level Full TimeSpain R6d ago
-
AI Engineer [España] EUR 35K-45KAWS | Azure | Backend Development | CrewAI | Data ScienceDevelopment and career plan | English classes | Flexible working hours | Home office equipment allowance | Paid time offMid-level Full TimeMadrid, Spain R7d ago
-
A/B | A/B Testing | AWS | Anthropic API | AzureFlexible location | Remote work | Stock optionsSenior-level Full TimeBarcelona office R7d ago
-
Senior AI Architect (pref BCN) EUR 36K-45KAccess Control | Artificial Intelligence | Azure Cognitive | Azure Cognitive Services | Azure DataCertifications | Continuous learning | Hybrid work | Mentoring | TrainingSenior-level Full TimeBarcelona, ES8d ago
-
ANTLR | AST | Agent systems | CI/CD | CobolSenior-level Full TimeAlcobendas, M, ES, 281088d ago
-
Principal AI Engineer EUR 56K-79KAgentic Frameworks | Agile | Azure | Azure DevOps | CI/CDBenefits | Remote work flexibility | Stock optionsSenior-level Full TimeSpain - Remote R11d ago
-
AI Engineer - FDE (Forward Deployed Engineer) EUR 56K-79KAWS | Agent systems | Apache Spark | Apache Spark SQL | AzureConference speaking opportunities | Professional development | Remote work | Travel as neededSenior-level Full TimeSpain12d ago
-
AI Engineer - API & Integrations Team EUR 55K-60KAPI Development | Agentic Workflows | Cloud Platforms | Evaluation Frameworks | Human-in-the-loopDiscounts | Free beverages | Gym access | Language classes | Office breakfastMid-level Full Time08005, Barcelona, CT, Spain12d ago
-
CX Automation & AI Lead EUR 57K-72KAPI Integration | Agent Orchestration | Gorgias | JavaScript | LLMAfter work events in Barcelona | Exclusive product discounts | Family life balance support | Flexible working hours | Growth and learning supportSenior-level Full TimeBarcelona, Spain12d ago
-
AI Architect (AWS) EUR 40K-50KAWS Lambda | AWS Step Functions | AgentCore) | Amazon Bedrock | Amazon Web ServicesCollaborative work environment | Employee discounts | Flexible compensation | Gympass | Indefinite contractSenior-level Full TimeMadrid, Spain13d ago
-
AI Engineer EUR 42K-48KAPI Design | AWS | Azure | FinOps | Generative AIChildcare card | Flexible compensation plan | Gym membership | Health insurance | Indefinite contractMid-level Full TimeMadrid, Spain13d ago
-
AI Data Platform Engineer (MLOps) EUR 35K-41KAWS | AWS Bedrock | Aurora PostgreSQL | Azure | CI/CDBenefit bonus | Life insurance | Nature activities | Paid training | Private medical insuranceMid-level Full TimeMADRID, Spain R14d ago
-
Senior AI Engineer - Agentic Workflows EUR 48K-72KAsyncio | Azure | Concurrent programming | Cost Optimization | Data ModelingCollective life retirement plan | Employee discounts | Flexible working model | Functional diversity benefits | Health insuranceSenior-level Full TimeBarcelona, ES14d ago
-
Senior Product Development AI Engineer EUR 56K-85KAI orchestration | AI-powered features | Agile | Java | Language ModelsDisability insurance | Life insurance | Onsite Language Classes | Paid vacation | Paid volunteeringSenior-level Full TimeR06-Barcelona, Spain15d ago
-
Agentic AI | Copilot Studio | Data Analysis | Data Visualization | DatabricksHybrid work environment | Learning opportunities | MentorshipSenior-level Full TimeMadrid, Spain R16d ago
-
AI Engineer Lead EUR 65K-90KAPIs | AWS | Agentic Workflows | Anthropic | CI/CDDiscount on events and experiences | English lessons | Flexible remuneration tax exemption | Health insurance | Home office friendly setupSenior-level Full TimeMadrid R18d ago