Senior Software Engineer - AI Model Evaluation
Tasks
- Analyze agent solution failures
- Build simulated developer environments
- Define solved criteria for tasks
- Design AI coding agent evaluation tasks
- Implement evaluation tests for agent solutions
- Iterate tasks based on QA feedback
- Refine tests and task criteria for robustness
- Write functional tests
- Write integration tests
Perks/Benefits
- N/A
Skills/Tech-stack
Case Development | Docker | FastAPI | Feedback iteration | Functional testing | Integration Testing | JavaScript | Kafka | Postgres | Python | QA Feedback Iteration | React | Redis | Test Case | Test case development | Testing | TypeScript
Education
N/A
Related jobs
-
1104 | Senior Data Engineer (Databricks) PLN 270K-384KACID | AWS | Access Control | Apache Spark | AzureSenior-level Full TimePoland - Remote R3d ago
-
Senior Machine Learning Engineer (E3) PLN 283K-358KAB Testing | AWS | BigQuery | Cloud Dataflow | Data DriftSenior-level Full TimeKrakow, Poland R4d ago
-
Generative AI Pipeline Engineer (Tech Lead) PLN 180K-360KAPI Integration | Agentic Workflows | Blender | CAD | ComfyUIFitness allowance | Home office support | Medical insurance coverage | Ongoing learning | Paid vacationSenior-level Full TimeWarsaw, Remote, PL R4d ago
-
AWS | Azure | Data Analysis | GCP | GitAutonomy | Continuous learning | Flexible working environment | Fully remote work | Opportunity to contribute to impactful projectsMid-level Full TimePoland R4d ago
-
Analyst II, Full Stack (Revenue Analytics) PLN 226K-346KAutomation | Cloud Computing | DBT | Data Governance | Data ModelingEmployee Resource and Community Groups | Employee stock purchase plan | Flexible spending wallets | Learning and development programs | Parental benefitsMid-level Full TimeRemote Poland R5d ago
-
Airflow | ArgoCD | BigQuery ML | CI/CD | Cloud RunCafeteria fringe benefits | English classes | Flexible working hours | Hackathons | MacBook or Dell equipmentEntry-level Full TimeWarszawa, PL, 00-841 R5d ago
-
LLM Engineer - Freelancer PLN 190K-276KAnthropic | Azure | Cost Optimization | Embeddings | LLM APIsFreelance work | Full-time work | Part-time workMid-level FreelanceWrocław, Lower Silesian Voivodeship, Poland - … R5d ago
-
Senior Data Engineer (Poland, Contract) PLN 152K-171KAWS ECS | Agile | Amazon Aurora | Apache Spark | AutomationAnnual company retreat | Confidential advice line | Flexible working hours | Remote work | Wellness supportSenior-level ContractPoland R9d ago
-
Senior AI Engineer PLN 242K-242KAPI Gateway | API Management | AWS Bedrock | AWS ECS | AWS ECS FargateEmployer supported pension plan | Extra paid holidays | Flexible benefits | Home-office allowance | Hybrid work modelSenior-level Full TimeWarszawa, Mazowieckie, PL R9d ago
-
Senior Software Engineer (Ruby/RoR, Chatbots & AI) PLN 240K-354KAngularJS | Debugging | Development Lifecycle | Go | JavaScriptBenefits package | Distributed team collaboration | Remote workSenior-level Full TimeWarsaw, Poland R10d ago
-
Middle Data Engineer PLN 206K-328KApache Airflow | Apache Spark | Data Modeling | Data Observability | Data Pipeline MonitoringBonuses | Corporate gifts | Educational courses | English classes | Hybrid workSenior-level Full TimePoland - Remote R11d ago
-
Senior Data Engineer GBP 91K-124KAccess Control | Airflow | BigQuery | CI/CD | Census401k employer match | Hackathons | Health and wellness benefits | Learning and development budget | Open source support fundSenior-level Full TimeGermany R13d ago
-
Senior Java Engineer PLN 231K-328KBig Data | Concurrency | Data Governance | Data Lakes | Data ManagementSenior-level ContractRemote (Poland) R13d ago
-
Mid/Senior SASE Automation Engineer PLN 237K-353KAnsible | CI/CD | Continuous integration | Docker | NGFWFlexible hours | Hands-on onboarding | Hybrid work | Internal training sessions | Remote workSenior-level Full TimeWarszawa, Poland R13d ago
-
Cyber Security AI Engineer (Cybersecurity Platforms) PLN 237K-353KAgent Orchestration | Azure | CI/CD | Django | DockerCompany events | Discounts and perks | Employee capital plans | Fitness and wellness programs | Free parkingSenior-level Full Time(HE)Remote KRK Pawia, Poland R13d ago
-
Staff/Lead Python Engineer (FastAPI, Orchestration) EUR 82K-112KAsynchronous programming | Cloud platform | Concurrent programming | FastAPI | GenAIAnnual learning and development budget | Co-working budget | Company laptop | Enhanced parental leave | Flexible scheduleSenior-level Full TimeRemote Anywhere R14d ago
-
Lead AI Engineer (Python) PLN 254K-380KAWS | CI/CD | Data Warehousing | Langchain | Language ModelsEmployee workshops | Flexible working hours | Life insurance | Paid annual service break | Paid public days off PolandSenior-level Full TimeKrakow/Remote within Poland R16d ago
-
Engineering - Internal AI Transformation USD 74K-110KAPIs | Agentic Workflows | Auditability | Authentication | DockerAnnual company offsite | Co-working stipend | Learning and development stipend | Social travel stipendEntry-level Full TimeUnited States R16d ago
-
Principal Data Engineer PLN 270K-414KAnalytics | Apache Spark | Cloud platform | Data Architecture | Data GovernanceE-learning access | Hybrid work | Life insurance | Lunch subsidy | Medical insuranceSenior-level Full TimeŁódź, PL, 90-118 R16d ago
-
Senior Data Engineer PLN 258K-360KAWS | AWS Glue | Amazon Kinesis | Apache Airflow | Apache FlinkAutonomy | High-impact projects | Influence | Mentorship | Room to growSenior-level Full TimePoland - Remote R16d ago
-
Principal Data Engineer with GCP PLN 22K-34KApache Spark | Cloud platform | Databricks | Google Cloud | Google Cloud PlatformE-learning platform | Hybrid work | Life insurance | Lunch subsidy | Medical insuranceSenior-level Full TimeŁódź, PL, 90-118 R16d ago
-
Staff AI Engineer - AI Infrastructure & Agentic Platform PLN 355K-384KAWS | Agentic Frameworks | Amazon Bedrock | Asynchronous programming | ConcurrencyAccident insurance | Additional vacation days | Life insurance | Private medical insurance | Remote work optionsSenior-level Full TimePoland R17d ago
-
AI-First Data Engineer (m/f/d) PLN 288K-336KAWS | Amazon Aurora | Amazon EMR | Amazon Kinesis | Amazon RedshiftCompany events | Flexible working hours | International team | Private medical care | Remote workMid-level Full TimeRemote (Poland) R17d ago
-
Senior AI Engineer PLN 242K-242KAPI Gateway | API Management | AWS Lambda | AWS Secrets | AWS Secrets ManagerEmployer supported pension plan | Extra paid holidays | Flexible benefits | Home-office allowance | Hybrid work modelSenior-level Full TimeWarszawa, Mazowieckie, PL R17d ago
-
Agentic Workflows | CI/CD | Control integration | Dependency graphs | Deployment AutomationGrowth mindset | Inclusive culture | Mentorship | Professional development opportunitiesSenior-level Full TimePoland R19d ago