Senior AI/ML Engineer - AI Systems Evaluation
Tasks
- Build automated evaluation pipelines
- Build prompt testing and dataset generation tools
- Close the loop between evaluation insights and product improvements
- Create golden datasets and edge case suites
- Define AI quality evaluation systems
- Design evaluation architectures
- Detect regressions and enforce quality gates in CI CD
- Implement LLM as judge scoring
- Instrument traces outputs and debugging
- Monitor model performance in production
- Track experiments and evaluate model performance
- Translate model behavior into measurable signals
Perks/Benefits
- N/A
Skills/Tech-stack
A/B | A/B Testing | B testing | Benchmarking | CI/CD | Data Pipelines | Debugging | Evaluation | Experiment tracking | LLM | LLM-as-judge | Logging | MLflow | Machine Learning | Observability | OpenTelemetry | Prompt engineering | Python | RAG | Regression testing | Retrieval-Augmented Generation | Tracing
Education
N/A
Related jobs
-
C# | C++ | Dashboarding | Data Architecture | Data GovernanceCareer growth | Community collaboration | Skill developmentSenior-level Full TimeTel Aviv, Israel12h ago
-
Mid-level Full TimeNetanya, Center District, IL22h ago
-
Airflow | Dagster | Data Engineering | Data Structures | Data Structures and AlgorithmsMid-level Full TimeNetanya, Center District, IL22h ago
-
Mid-level Full TimeTel Aviv Office1d ago
-
Entry-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL1d ago
-
AI Backend Engineer ILS 170K-230KAWS | Authentication | Azure | Cloud Computing | Data PipelinesContinuous learning | MentorshipEntry-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL1d ago
-
Data labeling | Deep learning | Financial Risk Management | Financial risk | Fraud DetectionSenior-level Full TimeTel Aviv, Israel1d ago
-
Apache Kafka | C++ | Cause analysis | Concurrency | DebuggingSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL1d ago
-
Mid-level Full TimeGiv'atayim, Tel Aviv District, IL1d ago
-
Artificial Intelligence | Assembly | CPU architecture | Communication Protocols | Communication SystemsMid-level Full TimeHod Hasharon, Haifa District, Israel1d ago
-
Acceptance criteria | Agile | ETL | Lean | Machine LearningFlexible work arrangements | Hybrid workSenior-level Full TimeISR - Central District of Israel …1d ago
-
Senior Delivery Consultant – AI/ML - Application Development, Professional Services Israel ILS 336K-443KAWS | Amazon Bedrock | Amazon SageMaker | Bias detection | Computer VisionFlexibility | Mentorship and career growth | Work-life balanceSenior-level Full TimeTel Aviv-Yafo, Tel Aviv, ISR1d ago
-
Senior Machine Learning Engineer- HiredScore ILS 338K-473KAPI Design | CI/CD | Data Modeling | Deep learning | Generative AIFlexible schedule | Hybrid work scheduleSenior-level Full TimeIsrael, Tel Aviv1d ago
-
Mid-level Full TimeTel Aviv-Yafo, Tel Aviv District, Israel2d ago
-
AWS | AWS Bedrock | Agents | Azure | Azure OpenAIGlobal projects | Hybrid work | Work with client facing teamsMid-level Full TimeTel Aviv, Tel Aviv District, IL2d ago
-
Mid-level Full TimeAshkelon, South District, IL2d ago
-
Senior ML Engineer (Token Factory) GBP 75K-130KAttention | CI/CD | CUDA | Cutlass | FP8Career growth and learning opportunities | Collaborative and innovative culture | Flexibility | International environment | Opportunity to work on impactful AI projectsSenior-level Full TimeAmsterdam, Netherlands; Berlin, Germany; Israel; London, … R2d ago
-
Senior ML Engineer (Token Factory) GBP 80K-130KCI/CD | Distributed Training | Inference Optimization | JAX | JAX Speculative DecodingCareer growth and learning opportunities | Collaborative culture | Flexibility | International environment | OwnershipSenior-level Full TimeGermany; Israel; Netherlands; Prague, Czech Republic; … R2d ago
-
Senior-level Full TimeTel Aviv, Israel2d ago
-
Mid-level Full TimeTel Aviv, Israel2d ago
-
Agile | Bare Metal | C# | C++ | CellularCollaborative team environment | Mentorship opportunities | Opportunities for innovation | Purpose-driven workSenior-level Full TimeYokne'am Illit, North District, IL3d ago
-
API Integration | Agent systems | Embeddings | Language Models | Large Language ModelsEvening meeting flexibility | Global collaboration | Hybrid work | Long term role opportunity minimum two years | On site work three times per weekMid-level Full TimeMevaseret Zion, Jerusalem, IL3d ago
-
Amazon CloudWatch | Amazon EKS | Amazon Web Services | Argo CD | BitbucketSenior-level Full TimeYizra'el, North, IL3d ago
-
Senior-level Full TimeRamat Gan, Israel3d ago
-
Asynchronous programming | CI/CD | Distributed Systems | Docker | Embedding ModelsSenior-level Full TimeIsrael, Yokneam3d ago