Senior/Principal Engineer (m/f/x) for Evaluations Generative AI
Linz, Vienna, Klagenfurt, Graz, Upper Austria, AT, 4020
EUR 74K-112K Senior-level Full Time
Tasks
- Build multi turn tool using agent simulations
- Define AI systems technical strategy
- Define evaluation metrics and datasets
- Design evaluation and simulation systems
- Detect regressions in model versions and prompts
- Generate adversarial scenarios
- Generate evaluation reporting
- Implement developer CLIs for evaluations
- Ingest traces and run judge configuration
- Measure task completion and tool use correctness
- Mentor engineers
- Prototype solutions and ship to production
- Replay and stress test AI agents
- Research generative AI
Perks/Benefits
- N/A
Skills/Tech-stack
Adversarial Testing | Agent planning | CI/CD | Command Line | Command-line interfaces | Data Drift | Data Drift Detection | Distributed Systems | Drift Detection | Evaluation Frameworks | Generative AI | Golden datasets | LLM | LLM-as-a-Judge | Pairwise Comparisons | Prompt engineering | Retrieval | Static Analysis | Tool use
Education
N/A
Regions
Countries
States
Related jobs
-
AWS | Azure | Data Management | Databases | DebuggingConference and training support | Employee equity opportunities | Equipment provided | Flexible working hours | Fully remoteSenior-level Full TimeSwitzerland R3d ago
-
Machine Learning Engineer CHF 129K-240KAPIs | Azure | Computer Vision | Databases | Distributed SystemsMid-level Full TimeZürich, ZH, Switzerland3d ago
-
AWS | Access Control | Access Management | Apache Airflow | Apache IcebergCareer growth opportunities | Flexible work arrangements | Fully remote | Knowledge sharing | Ownership and autonomyMid-level Full TimeSwitzerland R3d ago
-
API Integration | Angular | Argo Workflows | CI/CD | DBTDiscounted international travel | Free GA travelSenior-level Full TimeBern, CH3d ago
-
P2625 - Data Platform Engineer - Full Stack CHF 141K-227KAKS | AWS | Alerting | Audit Trail | AzureCross-functional collaboration | Equal-Pay certified workplace | Inclusive workplace | Professional growth | Relocation supportSenior-level Full TimeLausanne, Vaud, Switzerland3d ago
-
Apache Hadoop | Apache Kafka | Apache Spark | Batch Processing | BigQueryFully remote | International team collaboration | Professional development | Training opportunitiesSenior-level Full TimeSwitzerland R4d ago
-
Software Engineer II, GenAI Safety, Google Core CHF 140K-240KC plus plus | CSS | Data Structures | Data Structures and Algorithms | Distributed SystemsSenior-level Full TimeZürich, Switzerland4d ago
-
AI Engineer (m/w/d) EUR 36K-36KAPIs | Agentic AI | Automation | C# | CI/CDEnglish and German communication | Mentoring team culture | Professional development programs | Responsibility opportunitiesEntry-level Full TimeHart bei Graz, Steiermark, AT, 80754d ago
-
Machine Learning Engineer 100% (f/m/d) CHF 129K-192KAgent systems | Artificial Intelligence | Data Preprocessing | Debugging | DocumentationSenior-level Full TimeZurich, Switzerland4d ago
-
AI Assisted Development | Claude Code | Cursor | JavaScript | OpenAI CodexSenior-level Full TimeMiami5d ago
-
Agentic Systems | Datadog | Debugging | Distributed Systems | GPU ComputingCollaborative cross-functional environment | Flexible work across Europe | Fully remote work | High ownership role | Professional growth opportunitiesSenior-level Full TimeSwitzerland R5d ago
-
AI Workflows Engineer CHF 129K-232KCI/CD | Code review | Documentation | Git | Machine LearningFlexible hours | Hybrid work model | Multicultural collaborative environment | Professional development | Training supportMid-level Full TimeLausanne R5d ago
-
Distributed Systems | Go | Helm | Kubernetes | LinuxHealth insurance | Hybrid work schedule | Language classes | Paid leave | Pension fundEntry-level Full TimeGeneva, GENEVA, Switzerland5d ago
-
API Design | Authentication | Automation | Cloud Computing | Data ManagementFormal training | Health insurance | Hybrid work | Language classes | On-the-job trainingEntry-level Full TimeGeneva, GENEVA, Switzerland5d ago
-
Cloud infrastructure | Consistency models | Data Pipelines | Distributed Systems | ExperimentationCompany-provided equipment | Continuous learning allowance | Flexible work arrangement | Flexible working hours | Fully remote workSenior-level Full TimeSwitzerland R6d ago
-
AWS DMS | AWS Glue | AWS Lambda | AWS Step Functions | Amazon AthenaCollaborate with international team | Flexible working environment | Fully remote | Home office arrangement | Professional growth opportunitiesSenior-level Full TimeSwitzerland R6d ago
-
Forward Deployed AI Engineer EUR 49K-67KAWS | Agentic Workflows | Apache Iceberg | Apache NiFi | Apache SparkContinued Career Development | Employee resource groups | Flexible WFH policy | Generous PTO | Internet reimbursementMid-level Full TimeAustria-Remote R6d ago
-
Mid-level Full TimeBasel, Basel City, Switzerland - Remote R6d ago
-
Senior AI Engineer - Growth GBP 95K-129KAI orchestration | API Integration | Agent Chaining | Agentic Workflows | Audit LoggingSenior-level Full TimeUnited Kingdom7d ago
-
Platform Engineer (AI / Data Platform) EUR 47K-48KCI/CD | Data Engineering | Docker | Kubernetes | LinuxOn site visits by agreement | Remote workEntry-level Contract Full TimeWien7d ago
-
MLOps Engineer EUR 35K-35KCI/CD | Cloud Computing | Data Versioning | MLOps | Machine LearningFlexible onsite visits | Remote workEntry-level Contract Full TimeWien7d ago
-
Access Transparency | Artificial Intelligence | C++ | Cloud APIs | Customer-managed encryption keysSenior-level Full TimeZürich, Switzerland10d ago
-
Senior AI Engineer CHF 129K-192KBehavior analytics | Detection and Response | Fine Tuning | Langchain | Language ModelsAnnual training budget | Company events and workshops | Enhanced maternity leave | Enhanced paternity leave | Hybrid optionsSenior-level Full Timeremote or office, Zürich, Switzerland R10d ago
-
Senior AI Engineer CHF 129K-192KBehavior analytics | Cybersecurity | Detection and Response | Fine Tuning | LangchainAnnual training budget | Collaborative workshops | Enhanced maternity leave | Enhanced paternity leave | Health work-life balanceSenior-level Full Timeremote or office, Zürich, Switzerland R10d ago
-
Senior AI Engineer / Technical Consultant CHF 128K-192KAPI Development | Amazon Web Services | Azure | CI/CD | Cloud platformSenior-level Full TimeLausanne, Vaud, Switzerland11d ago