Principal AI Engineer
Tasks
- Architect evaluation platform
- Automate LLM evaluation lifecycle
- Build scalable infrastructure for AI evaluations
- Define AI quality metrics
- Design tools and workflows to reduce iteration time
- Establish observability feedback loop for evaluation datasets
- Guide agent design and failure analysis
- Implement production monitoring for AI systems
- Mentor engineers on LLM and agent system design
- Serve as AI technical lead for product squads
Perks/Benefits
- Health insurance
- Learning budget
- Paid parental leave
- Paid sick leave
- Paid transportation reimbursement
- Remote work
- Workstation equipment
Skills/Tech-stack
AI Agents | API Integration | Distributed Systems | Evaluation Frameworks | Event Driven | Event-driven architecture | Kafka | LLM | Observability | Orchestration | Prompt engineering | Pub/Sub | Python | RAG | Semantic Caching | State management | System Performance | System monitoring | Vector Databases
Education
N/A
Related jobs
-
AWS Lambda | AWS RDS | AWS S3 | Angular | BLASTEntry-level InternshipSerbia2d ago
-
AI Engineer PLN 100K-120KAWS | Azure | GCP | Git | LangchainAccess to 100 plus courses via iGaming Academy | Birthday leave | Flexible hours | Hybrid setup | Remote workEntry-level Full TimeBelgrade, Serbia R2d ago
-
Gen AI Engineer USD 125K-234K.NET | AI frameworks | API Integration | Agentic AI | AzureEquipment provided | Fitness Program Access | Flexible schedule | Hybrid work | Paid sick leaveMid-level Full TimeSerbia3d ago
-
Senior AI-First Data Engineer EUR 75K-125KApache Airflow | Business Intelligence | CRM | Customer Data | Customer data platformsRemote-friendlySenior-level Full TimeArmenia - Remote; Europe; Georgia - … R3d ago
-
Senior Data Engineer EUR 55K-76KAlerting | Amazon Kinesis | Amazon S3 | Anomaly Detection | Apache AirflowSenior-level Full TimeOffsite Serbia R4d ago
-
Senior Data Platform Engineer (Python) CAD 110K-160KAWS | Amazon Redshift | Amazon S3 | Apache Airflow | Apache IcebergHybrid work | Mentoring and code review culture | On-call supportSenior-level Full TimeOffsite Serbia R4d ago
-
Internship - Search Machine Learning Engineer USD 80K-159KA/B | A/B Testing | B testing | Classification | Information RetrievalEntry-level InternshipBelgrade6d ago
-
Auto Scaling | CI/CD | CUDA | Container Orchestration | Distributed SystemsEnglish language support | On-call support | Remote workSenior-level Full TimeUkraine R8d ago
-
Data Engineer USD 100K-140KAWS | Airflow | Amazon Redshift | Apache Iceberg | BigQueryAutonomy | Growth opportunities | Occasional travel | Supportive work environment | Work-life balance initiativesMid-level Full TimeEMEA - Remote; Estonia; Malta; Serbia R9d ago
-
Bash | Cloud platform | Data Processing | Docker | GCPAsynchronous culture | Friendly laid-back atmosphere | Remote/distributed workMid-level Full TimeNovi Sad, Serbia9d ago
-
Machine Learning Engineer USD 115K-120KCloud technologies | Docker | Language Models | Language Processing | Large Language ModelsContinuous training | Fitpass membership | Flexible hours | Hybrid work model | Mental health supportMid-level Full TimeSerbia - Novi Sad9d ago
-
Machine Learning Engineer USD 115K-120KCloud technologies | Docker | Language Models | Language Processing | Large Language ModelsAccredited certifications access | Continuous training | Fitpass membership | Flexible hours | Hybrid work optionsMid-level Full TimeSerbia - Belgrade9d ago
-
Senior AI Engineer USD 100K-147KAPIs | AWS | Artificial Intelligence | CI/CD | Cloud platform25 vacation days per year | Discounts at local stores | Free LinkedIn Learning | Gym membership | Hybrid work environmentSenior-level Full TimeBelgrade, Serbia10d ago
-
Software Engineer - Foundational Services USD 143K-220KAWS | Angular | Distributed Systems | Go | Google CloudFlexible vacation policy | Flexible working hours | Work from anywhereMid-level Full TimeBelgrade, Serbia R11d ago
-
Lead Software Engineer, Foundational Service Team USD 150K-220KAWS | Angular | Cloud infrastructure | Distributed Systems | GoFlexible vacation policy | Flexible working hours | Work from anywhereSenior-level Full TimeBelgrade, Serbia R11d ago
-
Senior AI Engineer USD 170K-225KAPIs | AWS | Agents | Azure | Cloud infrastructureHybrid work option | Relocation support | Remote work flexibility | Stock options | Travel supportSenior-level Full TimeBelgrade R16d ago
-
DevOps Engineer (AI Inference) USD 184K-287KAnsible | Argo CD | Bash | CI/CD | Cluster APIExtra Paid Sick Leave | Extra paid vacation | Flexible working hours | Hybrid work | Language coursesMid-level Full TimePoland, Serbia,, Cyprus, Georgia R17d ago
-
DevOps Engineer (AI Inference) USD 152K-287KAnsible | Argo CD | Bash | CI/CD | CNIExtra Paid Sick Leave | Extra paid vacation | Flexible working hours | Hybrid or remote options | Language coursesMid-level Full TimePoland, Serbia,, Cyprus, Georgia R18d ago
-
Senior Data Engineer (Contractor) USD 60K-130KAWS | Access Control | Apache Airflow | Audit Logs | AuditabilityInternational networking | MyPOS Academy upskilling training | Refer a friend bonus | Social activities | TeambuildingSenior-level ContractRemote, Serbia R18d ago
-
AI Technology Specialist AUD 87K-127KAI Agents | Artificial Intelligence | Compliance | Copilot | Enterprise ArchitectureMid-level Full TimeBelgrade, Serbia23d ago
-
Expert Embedded Software Engineer EUR 48K-68KAgile | Bare Metal | C# | C++ | DevOpsCertified course participation | Conference budget | Flexible working hours | Technical camp | Work from homeSenior-level Full TimeBelgrade, Serbia1mo ago
-
Senior AI & Machine Learning Security Engineer EUR 55K-75KART | AWS SageMaker | Adversarial Robustness | Azure AI | Azure AI/MLComprehensive benefits | Flexible work model | Growth opportunities | Hybrid work model | In-office optionSenior-level Full TimeSpain; Portugal; Poland; Romania; Serbia; Czechia1mo ago
-
Senior-level Full TimeWarsaw, PL / Serbia / Romania1mo ago
-
Senior AI Engineer (Core Engine) EUR 60K-84KAI Agents | AST | Information Retrieval | Java | Java Virtual MachineSenior-level Full TimeAmsterdam, Netherlands; Belgrade, Serbia; Berlin, Germany; …1mo ago
-
Staff/Senior AI Engineer, AI for Code GBP 81K-115KAgentic Workflows | Benchmarking | Context engineering | Cost Optimization | Fine TuningExtra time off | Flexible work location | Internal events | Language classes | Learning and development opportunitiesSenior-level Full TimeAmsterdam, Netherlands; Belgrade, Serbia; Berlin, Germany; … R1mo ago