AI Model Evaluation & Monitoring Specialist
Tasks
- Collaborate with IAOps and MLOps for observability and governance
- Design and implement GenAI testing frameworks
- Evaluate and compare LLM models
- Generate technical recommendations for model selection and cost optimization
- Monitor model changes and impact
- Optimize prompting strategies
Perks/Benefits
- Childcare card
- Collaborative work environment
- Employee discount portal
- Flexible compensation plan
- Gympass
- Hybrid work
- Indefinite contract
- Meal card
- Medical insurance
- Training programs
- Transport card
Skills/Tech-stack
A/B | A/B Testing | AI Governance | AWS Bedrock | AWS Lambda | Amazon CloudWatch | Amazon SageMaker | B testing | DeepEval | GenAI | Language Models | Large Language Models | Logging | Metrics | Model Observability | Prompt engineering | Ragas
Education
N/A
Related jobs
-
AI Agent Engineer - Python - Generative AI EUR 48K-72KAPI Gateway | AWS Bedrock | AWS Bedrock AgentCore | AWS Cloud | AWS Cloud Development KitFlexible benefits | Free Legal Support | Free emotional support | Free family support | Gym discountsSenior-level Full TimeMadrid, Spain1d ago
-
AI Solutions Manager EUR 55K-70KAI Automation | Agent systems | Artificial Intelligence | Enterprise governance | Language ModelsEmployee referral program | Family care benefits | Healthcare plans | Learning courses | Live insuranceMid-level Full TimeSpain - Barcelona R1d ago
-
AI Agents | AI Observability | AWS | Azure | CI/CDExposure to emerging AI technologies | Fully remote work environment | Leadership opportunities | Professional development opportunities | Work-life balanceSenior-level Full TimeSpain R4d ago
-
AI Engineer EUR 54K-75KAI Foundry | AI Search | API Design | API Versioning | Agent OrchestrationFlexible Leave Paid Time Off | Global Wellbeing Days | Growth opportunities | Remote work opportunitiesSenior-level Full TimeGranada, Spain4d ago
-
Senior Manager, AI Enablement & Data Governance EUR 65K-121KArtificial Intelligence | Automation | Data Governance | Data Quality | Data-Driven Decision MakingHybrid work | Relocation Support N/ASenior-level Full TimeBarcelona Gran Vía, Spain4d ago
-
Senior Manager, AI Enablement & Data Governance EUR 65K-121KArtificial Intelligence | Automation | Data Governance | Data Quality | Data TaxonomyHybrid work | Relocation support not providedSenior-level Full TimeBarcelona Gran Vía, Spain4d ago
-
API Integration | Anthropic | Automation | Error Handling | GDPREvents and team activities | Learning programs | Mentorship | Monthly benefit allowance | Paid vacation daysMid-level Full TimeSpain5d ago
-
Google Cloud AI Engineer EUR 50K-60KAI Agent Builder | Agent Builder | Cloud platform | Dialogflow | Google CloudMid-level Full TimeMadrid, MD, Spain5d ago
-
Staff AI Engineer - Data Visualization EUR 56K-85KAgent systems | Context engineering | Conversational AI | Data Visualization | Evaluation FrameworksCommunity guilds | Continuous career development | Employee stock purchase plan | Free mental health benefit | Hybrid workSenior-level Full TimeBordeaux, France; Grenoble, France; Lyon, France; …5d ago
-
Data Scientist And Ai Engineer With Databricks EUR 28K-36KData Monitoring | Data Quality | Databricks | Delta Lake | ELTEnhanced parental leave | Hybrid working | Meal vouchers | Private health insurance | Wellness platformsMid-level Full TimeMadrid, Spain6d ago
-
AI Software Engineer (GenAI / LLM) EUR 28K-40KAPI Integration | Anthropic Claude | CI/CD | Context Management | EmbeddingsAccident insurance | Childcare vouchers | Continuous training | Discount club | Flexible compensationMid-level Full TimeBarcelona, CT, Spain R6d ago
-
AI Business Analyst EUR 54K-75KAI Governance | Acceptance criteria | Autogen | Business Analysis | Business ProcessFull remote | Part-time or full-timeSenior-level Full TimePoland7d ago
-
Senior AI Solutions Engineer EUR 56K-89KAtlassian Confluence | Atlassian Jira | Coupa | Generative AI | Google WorkspaceConference budget | Health insurance | MacBook Pro provided | Permanent contract | Remote workSenior-level Full TimeRemote Spain, Spain R7d ago
-
Junior AI Engineer EUR 35K-45KAI Agents | AWS | Anthropic | Azure | DockerFlexible hours | Free coffee | Free juice | Free language classes | Friday afternoon offEntry-level Full TimeBarcelona Carrer Badajoz, Spain R7d ago
-
AI Engineer EUR 43K-65KAWS Bedrock | CI/CD | Docker | Git | KubernetesContinuous learning resources | Employee benefit programs | Hybrid work model | Internal mobilitySenior-level Full TimeBarcelona, Spain7d ago
-
AI Engineer EUR 50K-70KAWS Bedrock | AWS EventBridge | AWS Lambda | AWS SageMaker | AWS Step FunctionsContinuous learning plans | Hybrid work model | Lunch and coffee provided | Paid time off | Shortened FridaysMid-level Full TimeES - Alicante, Spain R7d ago
-
Mid-level Full TimeMadrid Office - Spain8d ago
-
AI Engineer - API & Integrations Team EUR 55K-60KAPI Design | Agentic Workflows | Artificial Intelligence | Cloud Platforms | EvaluationCaffeine and theine | Discounts | Expense Savings Plan | Gym access | Language classesMid-level Full TimeMadrid, Spain, Spain10d ago
-
API Integration | Artificial Intelligence | Code Analysis | Code generation | Documentation automationChildcare support | Continuous learning | Experience days | Flexible working hours | Health and wellbeing supportExecutive-level Full TimeGranada, AN, Spain R14d ago
-
AI Engineer EUR 54K-75KAI Foundry | AI Search | API Design | API Versioning | Agent OrchestrationAutonomy and Career Impact | Employee resource groups | Flexible Leave Paid Time Off | Global Wellbeing Days | Growth opportunitiesSenior-level Full TimeGranada, Spain14d ago
-
FP&A Solution AI Developer EUR 39K-54KArtificial Intelligence | Budgeting | ChatGPT | Claude | Data analyticsEntry-level Full TimeMadrid15d ago
-
AI Engineering Team Lead EUR 56K-79KAPI Design | AWS | AWS Lambda | Agent workflows | Behavior-Driven DevelopmentDental insurance | Flexible vacation | Flextime | Free Friday afternoons | Full remoteSenior-level Full TimeSpain - Remote R15d ago
-
AI Architect EUR 36K-45KAIOps | AWS | Amazon SageMaker | Artificial Intelligence | AzureCertification support | Hybrid work | Indefinite contract | Jornada intensiva in July and August | Paid trainingSenior-level Full TimeRemote, Spain R16d ago
-
AI Engineer - API & Integrations Team EUR 51K-60KAPIs | Agentic Workflows | Cloud Platforms | Evaluation Frameworks | Human-in-the-loopDiscounts | Free caffeine and theine | Gym access | Language classes | Office breakfastMid-level Full TimeA Coruña, A Coruña, Spain17d ago
-
AI Engineer EUR 54K-75KAI Foundry | AI Search | API Design | Agent Orchestration | Agent systemsEmployee resource groups | Flexible leave | Global Wellbeing Days | Growth opportunities | Paid time offSenior-level Full TimeGranada, Spain18d ago