Senior AI Engineer (Evaluations - Canvas Agent)
Tasks
- Analyze failure modes and hallucinations
- Build LLM evaluation pipelines
- Collaborate on full stack AI features and UI
- Create grading rubrics and datasets
- Design evaluation framework
- Engineer judge prompts
- Integrate evaluations into CI CD
- Monitor evaluation metrics and regressions
- Translate quality signals into engineering tasks
Perks/Benefits
- Employee recognition program
- Flexible work culture
- Generous time off
- Hack weeks
- Internal conferences
- Learning and development resources
- Mental health support
- Mentorship
- Professional development tools
- Remote work flexibility
- Tuition reimbursement
- Wellness programs
Skills/Tech-stack
Agentic Architectures | Braintrust | CI/CD | LLM Observability | Langsmith | Language Models | Large Language Models | Learning evaluation | Machine Learning | Machine learning evaluation | Node.js | Prompt engineering | Promptfoo | Python | Ragas | React | TypeScript
Education
N/A
Roles
Related jobs
-
AI and Data Science Developer, SAP Development HUF 12648K-18810KAI Platform | Business AI Platform | Cloud Foundry | Deep learning | FlaskAttractive benefit package | Continuous learning | Flexible work environment | Home office | Opportunities for growthMid-level Full TimeBudapest, HU, 11176h ago
-
Data Engineer (UA/RU Language speaking) EUR 49K-67KAPI Integration | AWS | Airflow | Airflow Orchestration | CRMAccess scoped retrieval and audit focus | Distributed team | Part time 20 hours per weekMid-level Full TimePoland1d ago
-
AI Analyst (UA/RU Language speaking) EUR 49K-67KAI Governance | Agentic Systems | Business Analysis | Business Case Modeling | Business caseClient-facing role | Executive stakeholder access | Multi phase sprint programMid-level Full TimePoland1d ago
-
AI Architect (UA/RU Language speaking) EUR 54K-75KABAC | AWS | Access Control | DBT | Data GovernanceSenior-level Full TimePoland1d ago
-
Machine Learning Data Builder HUF 14400K-22882KAirflow | DVC | Dagster | Data Engineering | Data PipelinesCollaborative culture | Ongoing training and development | Social events | Team-building activitiesSenior-level Full TimeBudapest, Hungary2d ago
-
AI Assisted Development | Claude Code | Cursor | JavaScript | OpenAI CodexSenior-level Full TimeMiami2d ago
-
Apache Spark | Azure | Azure Databricks | Databricks | Delta LakeChallenging projects | Full remote work | International work environment | Long-term career opportunities | Opportunity to work with experienced professionalsMid-level Full TimeBudapest, HU2d ago
-
DAX | Data Migration | Data Modeling | Data Quality | Data pipelineHome office | Long term role | Supportive team | Training planMid-level Full TimeBudapest, HU2d ago
-
AI Governance | Agent systems | Azure | By Design | Cloud NativeCareer growth opportunities | Flexible home office | Hybrid work | International work environment | Professional developmentSenior-level Full TimeBudapest, Hungary2d ago
-
AI Services | API Development | Agile | Azure | Azure AICareer growth | Company-sponsored certifications | Flexible working hours | Home office | International work environmentSenior-level Full TimeBudapest, Hungary2d ago
-
AWS | Azure | Azure Synapse | Bash | Cloud ComputingCareer growth | Flexible time off | Health insurance | Hybrid work | International projectsMid-level Full TimeBudapest, Hungary2d ago
-
AI Agent | AI Agent Frameworks | Agent Frameworks | Agent Orchestration | AgileFlexible working models | Health and wellbeing benefits | Learning and development | Mentorship | Paid internshipEntry-level InternshipBudapest, HU, 10312d ago
-
Senior Test Automation Engineer - ML-Powered Network Optimization HUF 10627K-17818KAgile | Bash | CI/CD | Docker | HelmAccident insurance | Company share purchase plan | Employee assistance program | Free Parking in Underground Garage | Free parkingSenior-level Full TimeBudapest,Budapest,Hungary3d ago
-
Senior AI Engineer - Growth GBP 95K-129KAI orchestration | API Integration | Agent Chaining | Agentic Workflows | Audit LoggingSenior-level Full TimeUnited Kingdom3d ago
-
APIs | Agentic AI | Azure | Azure DevOps | Data GovernanceBuddy support | Company sport clubs | Employee assistance program | Gym discount | Hybrid workMid-level Full TimeHungary, Budapest4d ago
-
Axolotl | Direct Preference Optimization | Efficient Fine Tuning | Fine Tuning | Language ModelsContinuous learning | Equal opportunity | Fully remote | Inclusive workplace | International collaborationMid-level Full TimeHungary R4d ago
-
Data Engineer Expert HUF 12000K-18000KApache Flink | Apache Spark | Azure | Data Lakes | Data ModelingCafeteria package | Free coffee and tea | Health insurance | Hybrid working | Nursery and kindergarten supportSenior-level Full TimeBudapest, Budapest, Hungary4d ago
-
Expert AI Engineer (REF5708U) HUF 4800K-5800KAgent architectures | Agent systems | Azure | Azure OpenAI | CI/CDFlexible work environment | Professional growth | Remote workSenior-level Full TimeBudapest,Debrecen,Pécs,Szeged, Hungary4d ago
-
Data Analyst, Product GBP 95K-124KA/B | A/B Testing | Analytical Functions | Apache Airflow | B testingSenior-level Full TimeUnited Kingdom6d ago
-
Technical AI Engagement Lead USD 150K-180KAI architecture | AWS | Agentic coding | Bedrock | Code RepositorySenior-level Full TimePoland8d ago
-
Senior Data Engineer PLN 177K-312KAWS CloudFormation | AWS Glue | AWS Step Functions | Amazon Athena | Amazon DynamoDBIn office hybrid or remote flexibility | International projects | Medical healthcare | Ongoing learning and reimbursement | Recognition programSenior-level Full TimeBulgaria, Hungary, Poland9d ago
-
Automated testing | C plus plus | CI/CD | Data Analysis | DevSecOpsEntry-level Part TimeHungary9d ago
-
Data Engineer gyakornok HUF 4068K-4068KDatabricks | Docker | Git | Grafana | KubernetesEmployee discounts | Flexible working hours | Health insurance coverage | Home office support | Relocation supportEntry-level Full Time InternshipBudapest, Hungary10d ago
-
Apache Spark | Azure Data | Azure Data Factory | Azure Event | Azure Event HubsFlexible work arrangements | Home office possibility | On call support schedule | Secure long term job opportunityMid-level Full TimePécs, Baranya, HU, 762210d ago
-
Apache Spark | Azure Data | Azure Data Factory | Azure Event | Azure Event HubsFlexible work arrangements | Home office possibility | Long-term job opportunityMid-level Full TimeBudapest, Pest, HU, 111210d ago