Senior Software Engineer - AI Model Evaluation
Tasks
- Analyze agent solution failures
- Build simulated developer environments
- Define solved criteria for tasks
- Design AI coding agent evaluation tasks
- Implement evaluation tests for agent solutions
- Iterate tasks based on QA feedback
- Refine tests and task criteria for robustness
- Write functional tests
- Write integration tests
Perks/Benefits
- N/A
Skills/Tech-stack
Case Development | Docker | FastAPI | Feedback iteration | Functional testing | Integration Testing | JavaScript | Kafka | Postgres | Python | QA Feedback Iteration | React | Redis | Test Case | Test case development | Testing | TypeScript
Education
N/A
Related jobs
-
Senior Software Engineer - Testing AI Coding Agents USD 100K-100KDocker | FastAPI | Functional testing | Integration Testing | JavaScriptFlexible schedule | Project based workSenior-level Full TimePoland - Remote R13h ago
-
Senior Software Engineer - AI Coding Benchmarks USD 80K-80KDocker | FastAPI | Functional testing | Integration Testing | JavaScriptSenior-level Full TimePoland - Remote R13h ago
-
Product Intelligence Engineer PLN 258K-329KAutogen | CrewAI | Data Governance | Data Modeling | Function CallingCareer development | Distributed team collaboration | Learning opportunities | Work from anywhereSenior-level Full TimeBielsko-Biała, Poland R22h ago
-
Embedded Software Engineer (Poland) PLN 180K-338KAnalog circuits | Automated testing | BLE | Bash | Bluetooth Low EnergyEducation Training Programs | English teachers | Medical insurance | Paid time off | Remote workSenior-level Full TimeKraków, Poland R1d ago
-
Asset Management | Data Analysis | Failure Mechanisms | Predictive Modeling | Probabilistic ModelingAnnual performance bonus | Disability programs | Flexible work hours | Hybrid work | Life insuranceSenior-level Full TimePL-PDK-WARSAW-ALEJA KRAKOWSKA 110/114 BLD 3014613, Poland R1d ago
-
Senior Data Engineer - Databricks PLN 270K-384KACID transactions | AWS | Access Control | Apache Spark | Automated testingAfter hours incident response | On-call rotationSenior-level Full TimePoland - Remote R1d ago
-
Test - nazwa stanowiska PLN 237K-328KAI Agents | APIs | Amazon Web Services | Automated testing | AzureBuddy onboarding program | Certificate training programs | Flexibility in working hours | Remote work option | Udemy learning platformSenior-level Freelance Full TimePoland R2d ago
-
Apache Airflow | BigQuery | Cloud platform | DBT | Data WarehousingAnnual off-site events | Co-working space allowance | Employee stock ownership program | Flexible asynchronous schedule | Fully remoteSenior-level Full TimePoland R2d ago
-
Senior AI Engineer (Remote) PLN 211K-310KAPIs | AWS | Agentic Systems | Azure | Backend DevelopmentHybrid work | Professional development | Remote work | Stock options program | Wellbeing supportSenior-level Full TimeWarsaw R2d ago
-
Data Engineer PLN 140K-224KApache Airflow | Apache Spark | CI/CD | DBT | Data LakehouseAccess to e-learning platform | Hybrid work | Life insurance | Lunch subsidy | Medical insuranceMid-level Full TimeŁódź, PL, 90-118 R3d ago
-
Senior Data Engineer PLN 175K-267KApache Airflow | Apache Spark | CI/CD | Cloud platform | DBTE-learning platform | Hybrid work | Life insurance | Lunch subsidy | Medical insuranceSenior-level Full TimeŁódź, PL, 90-118 R3d ago
-
Expert Data Engineer PLN 208K-320KApache Airflow | Apache Spark | CI/CD | Cloud platform | DBTE-learning platform | Hybrid work | Life insurance | Lunch subsidy | Medical insuranceSenior-level Full TimeŁódź, PL, 90-118 R3d ago
-
Alerting | BigQuery | CI/CD | Cloud Run | Cloud SQLContinuous learning | Flexible working arrangements | Fully remote | High autonomy | International engineering teamSenior-level Full TimePoland R3d ago
-
Associate AI Engineer PLN 139K-139KAI Evaluation | API Development | AWS | Agent systems | AzureEmployer supported pension plan | Extra paid holidays | Flexible benefits | Home-office allowance | Hybrid work modelMid-level Full TimeWarszawa, Mazowieckie, PL R5d ago
-
Data & Advanced Process Analytics Engineer PLN 276K-276KArtificial Intelligence | Data Visualization | ETL | Machine Learning | Process miningHybrid workMid-level Full TimePoland - Warsaw R6d ago
-
Senior-level Contract Full Timewarsaw, Poland - Remote R6d ago
-
Data Engineer | NDA PLN 276K-276KApache Airflow | Apache Spark | Azure Data | Azure Data Lake | CI/CDMid-level Full TimePoland - Remote R7d ago
-
Software Developer / Data Engineer PLN 276K-276KAPI Design | AWS | Airflow | Angular | AzureHybrid work modelMid-level Full TimePoland - Warsaw R7d ago
-
DevOps Engineer (AI Inference) USD 184K-287KAnsible | Argo CD | Bash | CI/CD | Cluster APIExtra Paid Sick Leave | Extra paid vacation | Flexible working hours | Hybrid work | Language coursesMid-level Full TimePoland, Serbia,, Cyprus, Georgia R8d ago
-
DevOps Engineer (AI Inference) USD 152K-287KAnsible | Argo CD | Bash | CI/CD | CNIExtra Paid Sick Leave | Extra paid vacation | Flexible working hours | Hybrid or remote options | Language coursesMid-level Full TimePoland, Serbia,, Cyprus, Georgia R8d ago
-
AI Architecture Engineer - Supply Chain Team (f/m/d) PLN 206K-254KAmazon Web Services | Artificial Intelligence | Azure | Data Architecture | Data PipelinesContinuous improvement culture | On-site work | Remote work flexibilitySenior-level Full TimePOL - Remote, Poland R9d ago
-
Senior/Lead Software Data Engineer (Roads Team) USD 75K-112KAWS | AWS Glue | AWS IAM | Amazon Athena | Amazon EC2Flexible work schedule | Health care | Parental leave | Remote work optionSenior-level Full TimeMapbox Poland R10d ago
-
Lead GIS Data Engineer PLN 206K-282KArcGIS | ArcGIS Enterprise | ArcGIS Network Analyst | ArcGIS Pro | ArcpyBonus | Flexible working hours | Life insurance | Medical coverage | Paid time offSenior-level Full TimePoland Home Office, Poland R10d ago
-
Staff Cyber Security Engineer – AI Data Protection PLN 284K-391KAI Security | AWS | Agile | Automation accounts | AzureFlexible working | Health and wellness coverage | Retirement and savings plans | Work-life balance supportSenior-level Full TimeKrakow, Poland R10d ago
-
Senior Machine Learning Engineer PLN 302K-454KA/B | A/B Testing | AWS | B testing | BentoMLCode review pairing sessions | Hybrid work | Mentorship and knowledge-sharing | Remote work flexibilitySenior-level Full TimePoland - Krakow - Hybrid R10d ago