AI QA Trainer - LLM Evaluation
Mexico
R
A MXN 789K-1062K (estimate) Senior-level Contract Full Time
Tasks
- Analyze model failures
- Assess bias
- Assess reasoning quality
- Automate testing using Python
- Build evaluation frameworks
- Conduct red teaming
- Create evaluation rubrics
- Define pass fail criteria
- Design test plans
- Detect hallucinations
- Develop quality benchmarks
- Document findings and root cause analysis
- Evaluate large language models
- Evaluate retrieval augmented generation
- Evaluate robustness
- Execute regression suites
- Maintain reproducible issue reports
- Measure factual accuracy
- Perform adversarial testing
- Provide prompt recommendations
- Recommend guardrails
- Support evaluation metrics and dashboards
- Test AI safety
- Use SQL for evaluation workflows
- Validate tool usage accuracy
- Verify grounding
Perks/Benefits
- Contract position
- Flexible hours
- Health insurance not included
- Paid time off not included
- Remote work
Skills/Tech-stack
Adversarial Testing | Bias Assessment | Evaluation Frameworks | Experiment tracking | Factual consistency | Grounding | Guardrails | Hallucination detection | LLM Evaluation | Language Models | Large Language Models | Machine Learning | Prompt engineering | Python | Quality Benchmarks | RAG | Red Teaming | Regression testing | Retrieval-Augmented Generation | Robustness Evaluation | SQL | Safety testing | Test automation
Education
Related jobs
-
Ingeniero de Datos SSIS MXN 240K-300KAWS | Automation | Azure | Data Governance | Data LakeEnglish basic support | Hybrid workMid-level Full TimeMexico R13h ago
-
Senior Data Engineer – GCP - LTI MXN 950K-1300KApache Airflow | BigQuery | CI/CD | Cloud Composer | Cloud platformAdvanced English | Advanced Spanish | Documentation | Fully remote | International work experienceSenior-level Full TimeCiudad de México, CDMX, MX R1d ago
-
Lead AI Engineer (LATAM Remote) MXN 960K-1300KDatabricks | Embeddings | Evaluation | Function Calling | GoRemote workSenior-level Full TimeRemote, Guadalajara, Mexico R1d ago
-
AWS | Agent systems | Azure | Compliance | Context ManagementFlexible consulting environment | Fully remote | Potential contract extension | US time zone overlapSenior-level Contract Full TimeMexico R1d ago
-
Senior-level Full TimeRemote, Mexico R1d ago
-
Access Control | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCompany-sponsored certifications | Company-sponsored coursework | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeZapopan, Jalisco, Mexico R2d ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOpsCompany-sponsored certifications | Conference attendance | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeTijuana, Mexico R2d ago
-
Agile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCompany-sponsored certifications | Conference attendance | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeSan Luis Potosi, Mexico R2d ago
-
Access Control | Automated scaling | Azure Data | Azure Data Factory | Azure Data LakeCompany-sponsored certifications | Conferences | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeSaltillo, Coahuila, Mexico R2d ago
-
Access Control | Agile | Azure Data | Azure Data Factory | Azure Data LakeCompany Sponsored Coursework Exams and Certifications | Continuing education | Flexible working hours | Paid time off | Remote workSenior-level Full TimePuebla City, Mexico R2d ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOpsCompany-sponsored certifications | Conference attendance | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeNaucalpan, Mexico R2d ago
-
Access Control | Agile | Azure Data | Azure Data Factory | Azure Data LakeCompany-sponsored certifications | Company-sponsored coursework | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeMérida, Yucatán, Mexico R2d ago
-
Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCompany-sponsored certifications | Conferences | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeMonterrey, Nuevo León, Mexico R2d ago
-
Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCompany-sponsored certifications | Continuing education | Flexible working hours | Paid time off | Remote workSenior-level Full TimeMexico City, Mexico R2d ago
-
Agile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCompany Sponsored Coursework Exams and Certifications | Continuing education training conferences | Flexible working hours | Paid time off | Remote workSenior-level Full TimeMexicali, Baja California, Mexico R2d ago
-
Agile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCompany-paid certifications | Conferences | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeLeón, Guanajuato, Mexico R2d ago
-
Access Control | Automation | Azure Data | Azure Data Factory | Azure Data LakeCompany-sponsored certifications | Conference opportunities | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeHermosillo, Sonora, Mexico R2d ago
-
Agile | Azure | Azure Data | Azure Data Factory | Azure Data LakeCompany-sponsored certifications | Conference attendance | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeGuadalajara, Jalisco, Mexico R2d ago
-
Access Control | Agile | Azure | Azure Data | Azure Data FactoryCompany-sponsored certifications | Company-sponsored coursework | Conference attendance | Continuing education | Flexible working hoursSenior-level Full TimeGuadalupe, Nuevo León, Mexico R2d ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOpsCompany sponsored certifications and exams | Conference support | Continuing education and training | Flexible working hours | Paid time offSenior-level Full TimeEcatepec de Morelos, Mexico R2d ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOpsCompany sponsored exams | Company-sponsored certifications | Conferences | Continuing education | Flexible working hoursSenior-level Full TimeCuliacán, Sinaloa, Mexico R2d ago
-
Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCompany-sponsored certifications | Conferences | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeCiudad Juárez, Mexico R2d ago
-
Agile | Azure | Azure Data | Azure Data Factory | Azure Data LakeCompany sponsored certifications exams and coursework | Conference opportunities | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeChihuahua City, Mexico R2d ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOpsCompany sponsored certification support | Conferences | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeAguascalientes, Mexico R2d ago
-
Senior AI Engineer - Generative AI & LLMs - México Only MXN 444K-456KAI Agents | Amazon Web Services | Azure | CI/CD | Cloud platformAccess to courses and certifications | English classes | Home office | Life insurance | Medical insuranceSenior-level Contract Full TimeMexico - Remote R2d ago