AI Evaluation & Automation Engineer
Tasks
- Build automated evaluation pipelines
- Build internal prompt testing tools
- Create golden datasets
- Design evaluation architectures
- Detect regressions and enforce quality gates
- Implement scoring systems
- Instrument traces outputs and debugging
- Integrate evaluation insights into product improvements
- Monitor model performance in production
Perks/Benefits
- N/A
Skills/Tech-stack
A/B | A/B Testing | Agent workflows | B testing | Benchmarking | CI/CD | Experiment tracking | Golden datasets | LLM-as-judge | Language Models | Large Language Models | Logging | Machine Learning | Observability | Python | RAG | Regression testing | Test automation
Education
N/A
Related jobs
-
Mid-level Full TimeBnei Brak, Tel Aviv District, IL6h ago
-
Artificial Intelligence | Competency Mapping | Content Review | Curriculum Design | Data ScienceCross-cultural experience | Free training | Networking opportunities | Opportunity to develop as a public expert | Remote workSenior-level Part TimeTel Aviv, Gush Dan, IL7h ago
-
AI orchestration | AWS | Apache Spark | Data Modeling | Data TransformationSenior-level Full TimeKfar Saba, Israel12h ago
-
Senior-level Full TimeJerusalem, Israel13h ago
-
Computer Vision | Deep learning | Diffusion Models | Foundation Models | Generative ModelsHybrid work setup | Mentorship | Onsite Days Per WeekEntry-level Internship Part TimeGM Israel - Technical Center Israel … R20h ago
-
Data Analysis | Data Classification | MATLAB | Performance Metrics | PythonMid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL1d ago
-
Mid-level Full TimeTel-Aviv, Israel, IL1d ago
-
Senior-level Full TimeBnei Brak, Tel Aviv District, IL1d ago
-
AI API | API Integration | Anthropic API | Automation Pipelines | Google AIOnsite N/A | Professional development opportunities | Remote-friendlySenior-level Full TimeRamat Gan, Israel, IL1d ago
-
Code Versioning | Computer Vision | Continuous integration | Data Analysis | Data PipelinesSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL1d ago
-
Access Control | Automation | CI/CD | Cloud platform | CloudflareCollaborative work culture | Flexible work environment | Mental health support | Wellness supportMid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL1d ago
-
AWS | CI/CD | CUDA | Containerization | Data StreamingOpen source contributionsMid-level Full TimeRamat Gan, Israel1d ago
-
Artificial Intelligence | Computer Vision | Flask | Generative AI | JupyterAcademic hours | Evening courseEntry-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL1d ago
-
Mid-level Full TimeRa'anana, Center District, IL1d ago
-
Apache Spark | Data Modeling | Data Warehousing | Event Streaming | LLMSenior-level Full TimeHerzliya1d ago
-
Big Data | Data Analysis | Machine Learning | Python | Scientific ComputingMid-level Full TimeJerusalem, Israel1d ago
-
Anomaly Detection | Dashboards | Data Modeling | Data Quality | Data ReconciliationContinuous learning opportunities | Education support | Employee discounts | Family-friendly benefits | Flexible time offMid-level Full TimeIsrael1d ago
-
3D computer graphics | AI-assisted coding | Artificial Intelligence | Assisted coding | Computer GraphicsMid-level Full TimeTel-Aviv, Tel Aviv District, IL1d ago
-
Mid-level Full TimeRamat-Gan, Tel Aviv District, IL1d ago
-
C++ | CUDA | CUDA profiling | Collective communication | Communication Compute OverlapSenior-level Full TimeIsrael, Tel Aviv R1d ago
-
C++ | Computer Vision | Deep learning | Machine Learning | Model EvaluationMid-level Full TimeRamat Gan, Israel2d ago
-
Senior-level Full TimeTel Aviv, Israel2d ago
-
Mid-level Full TimeTel Aviv, Israel2d ago
-
Senior-level Full TimeRa'anana, Center District, IL2d ago
-
Senior Machine Learning Engineer USD 160K-250KCaching | Cloud Platforms | GPU Computing | Kubernetes | LLM InferenceSenior-level Full TimeIsrael, center, IL2d ago