AI QA Specialist (Conversational & LLM Systems)
Lisbon, Portugal
EUR 51K-69K (estimate) Senior-level Full Time
Tasks
- Create and manage versioned datasets
- Define evaluation strategies for AI and LLM systems
- Define quality metrics and thresholds
- Detect regressions and evaluation drift
- Integrate evaluation practices into CI CD
- Provide quality signals for release decisions
- Validate conversational behavior end to end
- Verify intent matching and slot extraction
- Verify state transitions and business rules
- Verify tool and function calling correctness
Perks/Benefits
Skills/Tech-stack
Accuracy | CI/CD | Data Design | Data Quality | F1-score | Function Calling | Intent Recognition | JSON | JSON Schema | Language Models | Language Processing | Large Language Models | Monitoring | NLU | Natural Language | Natural Language Processing | Observability | Precisión | Python | Recall | Slot Extraction | Test Data | Test data design | Tool-Calling | Version control
Education
N/A
Roles
AI | AI QA | AI QA Specialist | Engineer | Machine Learning QA Engineer | QA Engineer | QA Specialist | Specialist
Language: en
Views:
3
Clicks:
0
Saves: 0
Related jobs
-
Mid-level Full TimeLisbon, Lisboa, Portugal1d ago
-
Mid-level Full TimeLisbon, Lisboa, Portugal1d ago
-
Apache Airflow | Business Intelligence | Cloud platform | Data Warehousing | Data pipelineAnnual vacation | Conference support | English language discounts | Fully remote | Health reimbursementSenior-level Full TimePortugal R2d ago
-
Agentic AI | Fallback mechanisms | Guardrails | Java | Java 17Career coaching | Career progression | Health insurance | Meal card | Remote work optionSenior-level ContractPortugal2d ago
-
Junior Data Governance Engineer EUR 28K-30KApache Airflow | Apache Spark | Azure | Data Catalog | Data ClassificationEmployee share purchase plan | Flexible benefits | Free medical services | Health insurance | Hybrid working modelEntry-level Full TimeLisbon, Santos 37, Portugal2d ago
-
Senior Data Governance Engineer EUR 45K-65KApache Spark | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Employee share purchase plan | Flexability Program | Flexben Program | Health insurance | Hybrid working modelSenior-level Full TimeLisbon, Santos 37, Portugal2d ago
-
API Integration | Argo Workflows | BigQuery | CI/CD | Cloud StorageCareer progression | Discounts | Health insurance | Meal card | Welcome kitMid-level ContractLisbon2d ago
-
Senior-level Full TimeLisbon2d ago
-
AI Solutions Builder EUR 67K-90KAWS | Agentic AI | CRM data | Command Line | Data integrationDirect collaboration with executive team | Fast paced collaborative team | Fully remote work | Growth opportunities | High ownership autonomySenior-level Full TimeEurope; Portugal - Remote; Serbia - … R2d ago
-
Senior Data Engineer (m/f) EUR 48K-70KAPI Integration | Argo Workflows | Artifactory | BigQuery | CI/CDCollaborative work environment | Continuous training | Hybrid work | Professional growth | Remote work optionSenior-level Full TimePorto, Portugal2d ago
-
Apache Airflow | Argo Workflows | CI/CD | Computer Vision | Continuous RetrainingCanteen | Career progression | Continuous professional development | Discounts on partnerships and products | Flexible work conditionsSenior-level Full TimeLisboa, Lisboa, Portugal2d ago
-
Data Quality Engineer EUR 35K-45KAlerting | Automation | Data Catalog | Data Governance | Data QualityMid-level Full TimeLisboa, Lisboa2d ago
-
Embedded Systems — Autopilot Software EUR 57K-72KAgile | BSP | Bare Metal | C# | C++Continuous learning | Flexible office location | Professional development | Transportation assistanceSenior-level Full TimeCaldas da Rainha, Portugal2d ago
-
AWS Redshift | Airbyte | Amazon S3 | Apache Airflow | Apache SparkFully remote | Health technology impactMid-level Full TimePortugal, Portugal R2d ago
-
AI Solution Architect GBP 78K-109KAWS | Architecture Diagrams | Artificial Intelligence | Asset Management | AutomationSenior-level Full TimeRemote - Canada; Remote - Portugal; … R2d ago
-
Senior Data Engineer - Governance (m/f/d) EUR 32K-45KAccess Control | Amazon Athena | Amazon QuickSight | Amazon Redshift | BI DashboardCoverflex allowance | Employee discounts | Flexible work setup | Learning and knowledge exchange | Meal allowanceSenior-level Full TimeLisbon, Portugal3d ago
-
Data Analyst & Data Visualization Specialist EUR 36K-54KData Analysis | Data Modeling | Data Quality | Data Visualization | DataikuMid-level Full TimeLisbon, Lisbon, Portugal3d ago
-
NumPy | Pandas | Python | SQL | SciPyFreelance | Part-time | Project-based employmentSenior-level FreelancePortugal - Remote R3d ago
-
C# | MATLAB | NumPy | Pandas | PythonPart time freelance projectEntry-level FreelancePortugal - Remote R3d ago
-
C# | MATLAB | NumPy | Pandas | PythonFreelance opportunities | Part-time project workEntry-level FreelancePortugal - Remote R3d ago
-
API Integration | Context engineering | Data Pipelines | Data Validation | Deep learningHealthcare dental insurance | In-office four days per week | Paid time off | Remote work one day per weekExecutive-level Full TimeLisbon, Portugal R3d ago
-
Bitbake | C# | C++ | CI/CD | Cross-compilationBirthday off | Extra vacation days | Gym discounts | Health insuranceEntry-level Full TimeMatosinhos, Porto, Portugal3d ago
-
Head of AI Transformation EUR 93K-125KAI ethics | Agentic AI | Artificial Intelligence | Business Development | Client AdvisoryCareer progression | Collaborative & Inclusive Culture | Continuous learning and development | Hybrid work modelExecutive-level Full TimeLisboa, Lisboa, Portugal3d ago
-
Data Engineer - BI (m/f/d) EUR 19K-19KAI | Amazon QuickSight | Amazon Redshift | Amazon Web Services | AutomationBirthday day off | Coverflex allowance | Discounted company services | Flexible work setup | Learning and knowledge exchange eventsEntry-level Full TimeLisbon, Portugal3d ago
-
AI Agents | AI Evaluation | Apache Spark | Data Pipelines | Distributed SystemsEmployee support programs | Flexible working hours | Fully remote | Permanent employment contract | Social benefitsMid-level Full TimePortugal R3d ago