AI Model Evaluation & Monitoring Specialist
Tasks
- Collaborate with IAOps and MLOps for observability and governance
- Design and implement GenAI testing frameworks
- Evaluate and compare LLM models
- Generate technical recommendations for model selection and cost optimization
- Monitor model changes and impact
- Optimize prompting strategies
Perks/Benefits
- Childcare card
- Collaborative work environment
- Employee discount portal
- Flexible compensation plan
- Gympass
- Hybrid work
- Indefinite contract
- Meal card
- Medical insurance
- Training programs
- Transport card
Skills/Tech-stack
A/B | A/B Testing | AI Governance | AWS Bedrock | AWS Lambda | Amazon CloudWatch | Amazon SageMaker | B testing | DeepEval | GenAI | Language Models | Large Language Models | Logging | Metrics | Model Observability | Prompt engineering | Ragas
Education
N/A
Related jobs
-
Senior Python Engineer – AI Agents & LLM Systems EUR 56K-84KAPI Design | Access Control | Agentic RAG | Anomaly Detection | AuthenticationBring a friend benefit | Company events and team buildings | Day off on birthday | English classes | Extra days offSenior-level Full TimeMadrid, Spain12h ago
-
I_SDS AI Solutions Engineer EUR 18K-18KAgile | Azure Databricks | Azure DevOps | Azure Functions | CI/CDChildcare support | Flexible working hours | Gym membership | Healthcare benefits | Hybrid work modelMid-level Full TimeMaya PG1, Spain23h ago
-
AI Workflow Engineer - Freelance AI Trainer USD 100K-100KAPI Integration | Automation | Bias Mitigation | Chaining Logic | Claude APIFlexible schedule | Fully remote | Performance-based bonusesMid-level FreelanceSpain - Remote R1d ago
-
AI Architect EUR 36K-45KAI Search | ARM | Access Management | Azure AI | Azure AI SearchContract indefinite | Everywhere work model | Extra vacation days | Flexible working hours | Intensive Friday scheduleSenior-level Full TimeHíbrido (28009, Madrid, MD, España) R4d ago
-
AI Full Stack Developer ID63542 EUR 50K-66KAPI Development | FastAPI | LLM APIs | Prompt engineering | PythonEducation budget | Exciting projects | Fitness budget | Flextime | MentorshipMid-level Full TimeMadrid, Spain4d ago
-
Mid-level Full TimeSpain6d ago
-
AWS | Agile | Azure | CI/CD | ComplianceContinuous learning | Flexible working hours | Hybrid remote work | Social benefits | Workplace volunteeringEntry-level Full TimeMADRID, ES R7d ago
-
Senior Director, AI Engineering, Enabling Units EUR 62K-85KAI Act | API Development | Artificial Intelligence | Automated testing | Bias detectionFlexible working hours | In office collaboration days per weekSenior-level Full TimeSpain - Barcelona8d ago
-
AI & Automation Engineer EUR 40K-48KAI ethics | API Integration | AWS Lambda | Agentic Frameworks | AsanaComprehensive benefits package | Conferences and events | Flexible work arrangements | Professional development budgetMid-level Full TimeSpain#SP, Hybrid, Spain (Hybrid) R8d ago
-
Director Data Science & Agentic AI EUR 93K-116KAgent Orchestration | Agent systems | Agile | Artificial Intelligence | Change ManagementHybrid work arrangementExecutive-level Full TimeBarcelona Gran Vía, Spain8d ago
-
AI Solutions Architect (IT) EUR 60K-86KAWS | Artificial Intelligence | Automation | Cisco Meraki | Cloud ArchitectureSenior-level Full TimeRemote - Spain R11d ago
-
API Gateway | AWS Bedrock | AWS Lambda | Agent Framework | AlertingChildcare card | Collaborative work environment | Employee discount portal | Flexible benefits | Gym membershipSenior-level Full TimeMadrid, Spain12d ago
-
AI Developer GBP 97K-120KC++ | CrewAI | Elixir | Langchain | Language ModelsFully remote | Globally distributed teamMid-level Full TimeAustria; United Kingdom; India; Portugal; Romania; … R12d ago
-
AWS | Agentic AI | Enterprise Architecture | Generative AI | HuggingfaceMid-level Full TimeSpain - Barcelona12d ago
-
Artificial Intelligence | Data Engineering | Data Migration | Data Transformation | ETLFlexible working arrangements | Full time 40h per week experience | Study grantEntry-level Full Time InternshipGetafe, Spain12d ago
-
AI Safety | API Design | Amazon Bedrock | Automated testing | Azure OpenAIBike leasing | Discount on products | Hybrid working | Option to work abroad | Subsidised transportSenior-level Full TimeMadrid, Spain13d ago
-
API Design | API Testing | Amazon Bedrock | Azure OpenAI | Context ManagementBike leasing | Employee product discount | Hybrid working | Option to work abroad | Subsidised transportSenior-level Full TimeMadrid, Spain13d ago
-
Senior AI Engineer ( All Genders) EUR 62K-90KAPI Design | Amazon Bedrock | Automated testing | Azure OpenAI | Context ManagementAnnual vacation days | Bike leasing | Employee discount | Hybrid working | Private medical health insuranceSenior-level Full TimeMadrid, Spain15d ago
-
Senior AI Engineer ( All Genders) EUR 62K-90KAPI Design | Amazon Bedrock | Automated testing | Azure OpenAI | Context ManagementAnnual vacation days | Bike leasing | Hybrid working | Private medical health insurance | Product discountSenior-level Full TimeMadrid, Spain15d ago
-
Global AI Architect/Enabler EUR 45K-52KAlerting | Amazon Web Services | Azure | C++ | CI/CDFlexible working hours | Onboarding program | Remote work flexibility | Training program | Work-life balanceSenior-level Full TimeEDIFICIO IBERDROLA MADRID, Spain15d ago
-
Lead Specialist, AI Scientist EUR 64K-85KAcoustic Modeling | Automated scoring | Bias analysis | Deep learning | Experimental DesignSenior-level Full TimeSpain19d ago
-
Lead AI Engineer, Business Acceleration EUR 62K-79KAWS Bedrock | Agentic Systems | Amazon SageMaker | Deep learning | DockerSenior-level Full TimeMadrid, MD, Spain19d ago
-
AI Services | Azure AI | Azure AI Services | Azure Cosmos | Azure Cosmos DBFull-time | Work-at-homeSenior-level Full TimeESP Work-at-Home, Spain20d ago
-
Senior AI Engineer EUR 62K-88KA/B | A/B Testing | AWS | Access Control | AuditabilityCoworking housing support | Flexible schedule | Performance-based culture | Referral bonus program | Remote work optionSenior-level Full TimeMadrid21d ago
-
Senior AI Engineer (All Genders) EUR 62K-88KCI/CD | Context engineering | Docker | Embeddings | JavaAnnual vacation days | Bike leasing | Employee discounts | Hybrid working | Private medical health insuranceSenior-level Full TimeMadrid, Spain21d ago