Senior/Principal Engineer (m/f/x) for Evaluations Generative AI
Linz, Vienna, Klagenfurt, Graz, Upper Austria, AT, 4020
EUR 74K-112K Senior-level Full Time
Tasks
- Build multi turn tool using agent simulations
- Define AI systems technical strategy
- Define evaluation metrics and datasets
- Design evaluation and simulation systems
- Detect regressions in model versions and prompts
- Generate adversarial scenarios
- Generate evaluation reporting
- Implement developer CLIs for evaluations
- Ingest traces and run judge configuration
- Measure task completion and tool use correctness
- Mentor engineers
- Prototype solutions and ship to production
- Replay and stress test AI agents
- Research generative AI
Perks/Benefits
- N/A
Skills/Tech-stack
Adversarial Testing | Agent planning | CI/CD | Command Line | Command-line interfaces | Data Drift | Data Drift Detection | Distributed Systems | Drift Detection | Evaluation Frameworks | Generative AI | Golden datasets | LLM | LLM-as-a-Judge | Pairwise Comparisons | Prompt engineering | Retrieval | Static Analysis | Tool use
Education
N/A
Regions
Countries
States
Related jobs
-
Agentic Workflows | Anomaly Detection | Artificial Intelligence | CI/CD | ClassificationCareer growth | Inclusion and equal opportunitiesEntry-level Full TimeHaag, St. Gallen, CH1d ago
-
2026 - Internship, Quantitative Developer CHF 63K-70KAPIs | Algorithms | Data Pipelines | Data Processing | Data StructuresCollaboration | Full time graduate opportunity upon completion | MentorshipEntry-level InternshipZurich3d ago
-
Embedded Linux Software Entwickler (m/w/d) EUR 54K-54KAngular | C# | C++ | CI/CD | Code reviewAgile team | Cafeteria | Flexible working hours | Home office | International work environmentEntry-level Full TimeNenzing, AT4d ago
-
Agentic patterns | Auto Instrumentation | Context propagation | Distributed Systems | Evaluation FrameworksSenior-level Full TimeLinz, Vienna, Klagenfurt, Graz, Upper Austria, …4d ago
-
API Integration | Agent memory | Agent systems | Autogen | CrewAIHybrid work | Professional development | Remote workMid-level Full TimeSwitzerland4d ago
-
API Gateway | Agentic AI | Automated testing | CI/CD | DockerCareer development | Continuous learning | Continuous learning opportunities | Flexible work arrangements | Hybrid workSenior-level Full TimeSwitzerland4d ago
-
Access Control | Amazon Kinesis | Amazon Redshift | Apache Airflow | Automated testingCompetitive compensation package | Flexible working hours | Fully remote work | Modern engineering environment | Paid vacation daysSenior-level Full TimeSwitzerland R4d ago
-
Senior AI Solutions Engineer (m/f/d) EUR 54K-66KAI Governance | AI Security | AI orchestration | AI red teaming | AI security testingBuddy system | Flexible working hours | Individual onboarding support | Onboarding plan | Public transport ticket coverageSenior-level Full TimeWien, AT5d ago
-
AI Engineer (f/m/d) EUR 55K-55KCI/CD | Cloud Computing | Containerization | Docker | Infrastructure as CodeCompany pension scheme | Flexible working hours | Health and social benefits | Remote work option | Training and further education opportunitiesMid-level Full TimeAUT.Reutte, Austria R5d ago
-
Associate Director, Forecasting Analytics CHF 140K-280KA/B | A/B Testing | API Development | AWS | Artificial IntelligenceMid-level Full TimeCarrier Innovation Technologies GmbH, Luzern, Switzerland5d ago
-
Senior Analytics Engineer 80-100% CHF 146K-236KBigQuery | CI/CD | Code Reviews | DBT | DBT CloudEqual pay for work of equal value | Multinational environment | Supportive teamSenior-level Full TimeGlattpark AmbassadorHouse, Switzerland5d ago
-
AWS | Apache Spark | Asynchronous programming | Azure | ChromaDBAccidental death dismemberment coverage | Dental insurance | Flexible paid time off | Health insurance | Life insuranceMid-level Full TimeSwitzerland R5d ago
-
AWS IAM | AWS Lambda | Airflow | Amazon EventBridge | Amazon S3Flexible working environment | Fully remote working | Professional growth opportunitiesMid-level Full TimeSwitzerland R5d ago
-
Junior Data Engineer (m/f/d) CHF 63K-84KAWS | CI/CD | Data Modeling | Databricks | Delta LakeCollaborative team environment | Hybrid setup | Remote-first | Work on mission driven productEntry-level Full TimeNeuchâtel, Neuchâtel, Switzerland R6d ago
-
Senior Data Platform Engineer EUR 50K-78KAirflow | Alerting | Apache Airflow | Apache Spark | Argo WorkflowsBike allowance | Flexible working | Generous leave | Health coverage | In-person collaborationSenior-level Full TimeGeneva; Barcelona;6d ago
-
C++ Software Engineer (SY-EPC-CCS-2026-115-GRAP) CHF 63K-69KC++ | CI/CD | Clang | Code Analysis | Code reviewHealth insurance | Hybrid work environment | Language classes | Paid leave | Pension fundEntry-level Full TimeGeneva, GENEVA, Switzerland7d ago
-
Data Engineer CHF 106K-146KAmazon Web Services | Apache Iceberg | Azure | Bash | CI/CDWork from home up to 40%Mid-level Full TimeZurich, CH7d ago
-
Principal Data Scientist CHF 128K-192KAI Agent | AI Agent Frameworks | Agent Frameworks | Agent systems | CrewAIFlexible medical benefits | Life insurance | Retirement options | Volunteer programsSenior-level Full TimeZURICH, Switzerland7d ago
-
AI Engineer CHF 128K-192KAccess Control | CI/CD | Cost governance | Data Pipelines | DevOpsCommunity events | Complimentary gym access | Daily lunch allowance | Fitness classes | International exposureSenior-level Full TimeBaar, CH8d ago
-
AI Architect, Partner Engineering GBP 100K-120KAgent systems | AlloyDB AI | BigQuery | BigQuery Zero Copy | C++Senior-level Full TimeLondon, UK; Zürich, Switzerland9d ago
-
Sr Security Engineer, Incident Response GBP 91K-120KAI | AWS | Access Control | Agentic Platforms | Azure24x7 on call schedule | Distributed OperationsSenior-level Full TimeBelgium; Finland; Remote - Denmark; Remote … R10d ago
-
Senior Lead Engineer, Imaging Algorithm Development - RDT Biometrics and Clinical Evidence CHF 129K-204KArtificial Intelligence | Automated testing | Best practices | C++ | CDSenior-level Full TimeKaiseraugst FHLR, Switzerland11d ago
-
AI & Data Engineer Claims (m/w/d) 80-100% CHF 66K-76KAutomated Deployment | CI/CD | Cloud Computing | Data pipeline | Data platformAdditional vacation days for purchase | Flexible work environment | Paid parental leave | Training opportunitiesMid-level Full TimeWallisellen (Zürich), Zürich, CH, 830411d ago
-
AI Assisted Code Analysis | AI assisted code | ARM | Assembly | Automation frameworksFlexible working time | Home office | Meal benefitsSenior-level Full TimeGratkorn, Austria12d ago
-
Access Management | Azure | Bronze Silver Gold | CI/CD | Data GovernanceRemote workSenior-level Full TimeSwitzerland - Remote R12d ago