Associate Director, Generative AI Evaluation & Standards
ES022 Janssen Madrid, Spain
EUR 52K-80K (estimate) Mid-level Full Time
Tasks
- Build automated evaluation pipelines
- Collaborate on data quality rubric
- Conduct platform layer evaluation
- Create evaluation training materials
- Define benchmarks
- Design evaluation frameworks
- Develop domain specific criteria
- Enforce production ready standards
- Ensure regulatory compliance
- Establish gold standard datasets
- Lead and manage evaluation team
- Lead evaluation governance
- Maintain evaluation standards
- Partner on production monitoring handoff
- Perform security assessments
- Present evaluation findings
- Publish evaluation rubrics
- Set evaluation gates
- Validate scientific accuracy
Perks/Benefits
Skills/Tech-stack
Agentic Systems | Benchmarking | CDISC | Cybersecurity | Data Governance | Data Privacy | Data Quality | Evaluation Framework | Evaluation framework design | FHIR | Framework design | Generative AI | Knowledge graphs | Language Models | Large Language Models | MLOps | Multi-Modal | Multi-modal AI | Ontologies | Prompt engineering | RAG | Regulatory Compliance | Retrieval-Augmented Generation | Scientific accuracy
Education
Related jobs
-
Senior-level Full TimeMadrid, Europe, Spain9h ago
-
AI machine learning | Azure | DAX | Data Modeling | Data QualityDiscounts on training and programs | Flexible benefits | Flexible work hours | Health insurance | Intensive work scheduleMid-level Full TimeBarcelona, Spain R10h ago
-
LLM Engineer EUR 51K-68KCI/CD | Docker | Embeddings | Fine Tuning | Generative AIAnnual leave | Commuting allowance | Employee referral scheme | Flexible schedule | Home working allowanceMid-level Full TimeBarcelona, Spain11h ago
-
Knowledge Graph Engineer, R&D Data Science & Digital Health - Data Strategy and Products EUR 52K-80KAmazon Neptune | Azure DevOps | CI/CD | Containerization | Continuous DeliveryMid-level Full TimeES022 Janssen Madrid, Spain18h ago
-
Senior Analytics Engineering Manager EUR 126KArchitecture Design | Big Data | Data Governance | Data Visualization | Data WarehouseFlexible working hours | Learning budget | MacBooks provided | On-call support | Work from home setupSenior-level Full TimeBarcelona R1d ago
-
Beca Data Engineer EUR 28K-35KData Engineering | Data Ingestion | Data Modeling | Data Pipelines | Data QualityCareer growth | Continuous training | Hybrid work | MentoringEntry-level Full TimeMadrid, Spain1d ago
-
Senior AI Engineer (All genders) EUR 60K-88KAgentic AI | Amazon Bedrock | Artificial Intelligence | Azure OpenAI | CI/CDAnnual vacation days | Bike leasing | Hybrid working | Private medical health insurance | Product discountSenior-level Full TimeMadrid, Spain1d ago
-
Senior AI Engineer EUR 51K-78KAdversarial Red Teaming | Automated testing | Bias Mitigation | Data Visualization | Dependency InjectionDental plan | Flexible remuneration | Health insurance | Home office setup allowance | Office breakfast buffetSenior-level Full TimeBarcelona - Cataluña1d ago
-
Amazon Redshift | Analytics engineering | DBT | Data Modeling | Data QualityFully remote work | Healthcare coverage | Home office equipment allowance | Learning and development budget | Paid parental leaveSenior-level Full TimeSpain R1d ago
-
Front And Ai Engineer - Ingles - Remoto EUR 30K-40KA2A | AG-UI | ASP.Net Core | Active Directory | App ServiceEmployee benefits discounts | Employee training | Flexible compensation | Indefinite contract | Remote workSenior-level Full TimeSpain - Remote R1d ago
-
Analytics & Insights Manager (Data Engineering) EUR 34K-34KAirflow | CI/CD | DBT | Data Engineering | Data ObservabilitySenior-level Full TimeMadrid Osiris, Spain1d ago
-
AWS | Amazon Redshift | Cloud Architecture | Code Versioning | Data LineageSenior-level Full TimeES022 Janssen Madrid, Spain1d ago
-
Senior-level Full TimeSpain - Barcelona : Av. Diagonal, …1d ago
-
Data Engineers EUR 30K-36KAmazon S3 | Apache Airflow | Apache Spark | CI/CD | Data QualityCompany training | Conference attendance | Flexible compensation benefits | Flexible working hours | Indefinite contractMid-level Full TimeMadrid, MD, Spain R2d ago
-
Agent architecture | Analytics engineering | Artificial Intelligence | CRM | ClassificationFlexible work arrangements | Health insurance | Mental health support | Paid public holidays | Paid vacationExecutive-level Full TimeSpain2d ago
-
Senior ML/AI Engineer EUR 60K-84KAPI Integration | AWS | Agent systems | Automated Evaluation | Cloud platformFlexible work culture | Global opportunities | High-impact environment | Professional development | Vibrant communitySenior-level Full TimeBarcelona - Spain2d ago
-
Senior Data Lead Engineer EUR 46K-73KAWS | BI Semantic Modeling | CDC | CI/CD | ConfluenceHybrid work | International projects | Multicultural team | Permanent employment | Training and continuous career developmentSenior-level Full TimeMálaga, AN, Spain2d ago
-
Senior Machine Learning Engineer EUR 56K-79KAI Pipelines | AWS | CI/CD | Containerization | Data EngineeringCharitable giving program | Fitness reimbursement | Life insurance | Pension fund | Work anywhereSenior-level Full TimeSpain - Barcelona2d ago
-
Data Engineer EUR 39K-40KAI/ML | AWS | Apache Spark | CI/CD | Cloud DataFlexible working hours | Free beers | Free coffee | International team | Professional developmentMid-level Full TimeBarcelona, Catalonia2d ago
-
Machine Learning Engineer 100% (f/m/d) EUR 60K-78KAgentic Workflows | Data Preprocessing | Debugging | Feature Engineering | Few-Shot LearningSenior-level Full TimeMadrid, Spain2d ago
-
Machine Learning Engineer 100% (f/m/d) EUR 60K-78KAgent systems | Agentic Workflows | Data Preprocessing | Debugging | DocumentationSenior-level Full TimeMadrid, Spain2d ago
-
A/B | A/B Testing | AWS | AWS Strands | B testingAbility to explain technical concepts | Adaptability | Analytical mindset | Team collaborationSenior-level Full TimeCIUDAD BBVA [00018800 BCO. BILBAO VIZCAYA …2d ago
-
Lead Data Engineer (m/f/d) EUR 53K-60KAccess Management | DBT | Data Governance | Data Lakehouse | Data ModelingFlexible hybrid working model | Individual development plans | Training opportunitiesSenior-level Full TimeBarcelona, Spain2d ago
-
Amazon Neptune | Artificial Intelligence | Biomedical Ontologies | Clinical data | Cloud ComputingDisability insurance | Kindergarten card | Language classes | Life insurance | Paid vacationMid-level Full TimeR06-Barcelona, Spain2d ago
-
AI Engineer (All Genders) EUR 56K-80KCI/CD | Context engineering | Docker | Embeddings | JavaAnnual vacation days | Bike leasing | Development opportunities | Discount on products | Hybrid workingMid-level Full TimeMadrid, Spain3d ago