Senior AI/ML Engineer - AI Systems Evaluation
Tasks
- Build automated evaluation pipelines
- Build prompt testing and dataset generation tools
- Close the loop between evaluation insights and product improvements
- Create golden datasets and edge case suites
- Define AI quality evaluation systems
- Design evaluation architectures
- Detect regressions and enforce quality gates in CI CD
- Implement LLM as judge scoring
- Instrument traces outputs and debugging
- Monitor model performance in production
- Track experiments and evaluate model performance
- Translate model behavior into measurable signals
Perks/Benefits
- N/A
Skills/Tech-stack
A/B | A/B Testing | B testing | Benchmarking | CI/CD | Data Pipelines | Debugging | Evaluation | Experiment tracking | LLM | LLM-as-judge | Logging | MLflow | Machine Learning | Observability | OpenTelemetry | Prompt engineering | Python | RAG | Regression testing | Retrieval-Augmented Generation | Tracing
Education
N/A
Related jobs
-
AWS | Active Directory | Authentication | Azure | C#Fertility assistance | Hybrid work model | Parental leaveMid-level Full TimeTel Aviv18h ago
-
Senior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL18h ago
-
AI Agent | AI Agent Frameworks | AI orchestration | API Integration | Agent FrameworksMid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL18h ago
-
Artificial Intelligence | Automation | LLM Agents | Language Models | Large Language ModelsEntry-level Part TimeRamat Gan, Tel Aviv District, IL19h ago
-
Senior Agentic AI Developer and Malware Analysis Expert ILS 380K-473KAgent Orchestration | Air gapped deployment | Air-gapped | Autonomous Agents | Binary AnalysisSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL20h ago
-
AWS | Agentic AI | Algorithms | Apache Spark | Cloud platformSenior-level Full TimePetah Tikva, Central District1d ago
-
AI | Acceptance criteria | Agile | Business Process | Business process modelingFlexible work arrangements | Hybrid work | Up to 20 percent travelSenior-level Full TimeISR - Central District of Israel …1d ago
-
Senior AI Engineer ILS 341K-443KAWS | Azure | Cloud platform | DevOps | DockerCommuter benefits | Equity | Extra Time Off for Parents and Caregivers | Lunch stipend | Parking benefitsSenior-level Full TimeTel Aviv1d ago
-
Mid-level Full TimeTel Aviv-Yafo, Israel, IL1d ago
-
Enterprise IT | Enterprise IT infrastructure | File systems | Hardware management | High PerformanceTravel up to 20 percentMid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL1d ago
-
Mid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL1d ago
-
Amazon EMR | Amazon Web Services | Apache Airflow | Apache Kafka | Apache SparkCareer coaching | Happy hours | Learning opportunities | Team outings | Work partially from homeMid-level Full TimeTel Aviv, Israel1d ago
-
Senior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL1d ago
-
Cloud Native | Data Integrity | Data Lake | Data Processing | Data analyticsSenior-level Full TimeIsrael, Raanana2d ago
-
Sr Staff AI Software Engineer (CORA AI) ILS 341K-443KA2A | AWS | Agentic architecture | Language Models | Large Language ModelsFlexible work arrangement | In-office collaborationSenior-level Full TimePetah Tikva, Central District2d ago
-
Senior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL2d ago
-
Staff Data Science Researcher ILS 285K-366KA/B | A/B Testing | AI Agents | AWS Bedrock | Agent systemsFlexible schedule | Hybrid work model | Mentorship culture | Remote work daysSenior-level Full TimeIsrael - Raanana R2d ago
-
Mid-level Full TimeJerusalem, Israel2d ago
-
3D Geometry | Active Learning | Airflow | Argo | C++On site work several days per weekMid-level Full TimeRamat Gan, Israel2d ago
-
Senior-level Full TimeTel Aviv, Israel2d ago
-
Mid-level Full TimeTel Aviv, Israel2d ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerMid-level Full TimeTel Aviv, Israel3d ago
-
AI Engineering Tech Lead (Core Engineering) ILS 420K-504KAWS | Agentic Frameworks | CI/CD | Elasticsearch | LLMsFlexible work environment | OwnershipSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL4d ago
-
AI Engineering Team Lead ILS 341K-443KAgentic AI | Cloud Computing | Compliance Automation | Cost Optimization | Data OperationsDirect access to leadership | Flexible culture | Meaningful equity | Output first cultureSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL5d ago
-
Computer Vision | Data Engineering | Data Pipelines | Data Storage | Data VersioningCareer growth opportunitiesMid-level Full TimeTel Aviv, Tel Aviv District, IL5d ago