AI QA Trainer - LLM Evaluation
Turkey
R
A TRY 840K-1080K (estimate) Senior-level Contract Full Time
Tasks
- Analyze model behavior and recommend improvements
- Assess bias and safety risks
- Automate testing with Python or SQL
- Build evaluation frameworks
- Collaborate on evaluation metrics dashboards and quality monitoring
- Create evaluation rubrics and benchmarks
- Design test plans and regression suites
- Evaluate large language models
- Identify and document model failures
- Maintain documentation of findings and root cause analysis
- Perform adversarial testing and red teaming
- Validate tool usage accuracy and workflow reliability
- Verify grounding retrieval augmented generation outputs
Perks/Benefits
- Flexible contract opportunity
- Fully remote work
- Health insurance not included
- Paid time off not included
Skills/Tech-stack
Adversarial Testing | Bias Assessment | Experiment tracking | Factual consistency | Hallucination detection | LLM Evaluation | Language Models | Large Language Models | Machine Learning | Prompt engineering | Python | Quality Benchmarks | RAG | Red Teaming | Regression testing | Retrieval-Augmented Generation | SQL | Safety testing | Test automation
Education
Related jobs
-
Senior Machine Learning Engineer (Agentic AI) TRY 1080K-1185KAWS | Agile | Code review | Deployment | KubernetesESOP | Internal training | Learning resources | Meal allowance | Private health insuranceSenior-level Full TimeIstanbul, Turkiye R2d ago
-
Mid-level Full TimeTürkiye, Remote R3d ago
-
Alerting | Amazon Redshift | Batch Processing | CI/CD | Cause analysisCross-functional collaboration | Fully remote work | Opportunity for continuous improvement | Ownership of data reliability strategySenior-level Full TimeTurkey R4d ago
-
Senior-level Full Timeİstanbul, Turkey - Remote R5d ago
-
Senior AI/LLM Data Scientist in Agentic AI TRY 840K-1080KAPI Integration | Embeddings | Evaluation | Fine Tuning | Generative AIEnglish Language Development Support | Hybrid work | Meal card | Multisport membership | Private health insuranceSenior-level Full TimeIstanbul (VADI ISTANBUL PARK), Türkiye R12d ago
-
Auto Scaling | CI/CD | CUDA | Container Orchestration | Distributed SystemsEnglish language support | On-call support | Remote workSenior-level Full TimeUkraine R14d ago
-
AI Engineer TRY 400K-570KAsynchronous programming | BigQuery | CI/CD | Claude | Cloud NativeCareer opportunities | English Language Development Support | Health insurance | Hybrid work | Meal cardMid-level Full TimeIstanbul (VADI ISTANBUL PARK), Türkiye R16d ago
-
AI Engineer TRY 400K-570KAgent systems | Arize | Asynchronous programming | BigQuery | CI/CDCareer opportunities | English Language Development Support | Health insurance | Hybrid work | Meal cardMid-level Full TimeIstanbul (VADI ISTANBUL PARK), Türkiye R16d ago
-
Senior Machine Learning Engineer (Search) TRY 960K-1132KA/B | A/B Testing | AWS | Airflow | Artificial IntelligenceESOP (equity) | Health insurance | Learning budget | Meal allowance | Referral bonusesSenior-level Full TimeIstanbul, Turkiye R29d ago
-
ClickHouse | Data Mining | Machine Learning | MongoDB | Power BIFlexible schedule | Free parking | International environment | Meal card | Paid time offMid-level Full TimeIstanbul, İstanbul, Türkiye R1mo ago