[ Job 28396] Mid QA Automation — Generative AI Testing, Brazil
Tasks
- Automate and maintain automated LLM response validation
- Collaborate with developers to debug failures and improve robustness
- Create prompt datasets and test cases with expected outputs
- Define test strategies for generative AI agent pipelines
- Develop unit tests functional tests regression tests adversarial tests
- Document test procedures acceptance criteria and results
- Monitor report and analyze agent performance accuracy and security metrics
- Participate in agile rituals reviews and release strategy
- Use observability tracing and evaluation tools for LLM results
Perks/Benefits
- Childcare assistance
- Dental insurance
- Extended parental leave
- Health insurance
- Life insurance
- Meal allowance
- Paid parental leave
Skills/Tech-stack
Adversarial Testing | CI/CD | Case management | Continuous integration | Functional testing | Kafka | LLM Evaluation | Langchain | Langsmith | Observability | Prompt engineering | Promptfoo | Pytest | Python | Regression testing | Test Case | Test Case Management | Test automation | TruLens | Unit Testing | Unittest
Education
N/A
Regions
Countries
States
Related jobs
-
AWS | CI/CD | Data integration | GCP | LLMFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsMid-level Full TimeCampinas, Brazil7h ago
-
.NET | Apex | C# | C++ | DartRemote workSenior-level Part TimeBrasília, Brazil R8h ago
-
Data Engineer BRL 240K-300KAWS | Amazon S3 | Azure | Azure Blob | Azure Blob StorageWork from homeMid-level Full TimeBrazil19h ago
-
Computational Algorithms | Generative AI | Machine Learning | NumPy | Numerical MethodsPart-time freelance | Remote workMid-level FreelanceBrazil - Remote R21h ago
-
Computational Algorithms | Machine Learning | NumPy | Numerical Methods | PandasFlexible schedule | Freelance work | Part-time | Remote workMid-level FreelanceBrazil - Remote R21h ago
-
Especialista de Engenharia de Dados I BRL 54K-60KAWS | CI/CD | Clean Code | Data Lake | Data MartHybrid work | Inclusive culture | Professional certifications support | Wellbeing focusSenior-level Full TimeSão Paulo, Brazil1d ago
-
DevOps & Site Reliability Engineer BRL 108K-136KAmazon Web Services | Azure | Azure Pipelines | Bash | CI/CDCareer development | English program | Fitness program | Home office | International projectsMid-level Full TimeBrasília, Federal District, Brazil R1d ago
-
DevOps & Site Reliability Engineer BRL 108K-136KAmazon Web Services | Azure | Azure Pipelines | Bash | CI/CDCareer development | English program | Fitness program | Games and competitions | International projectsMid-level Full TimeSão Paulo R1d ago
-
AWS | AWS Athena | AWS Glue | Apache Spark | ETLChildcare support | Continuous learning platform | Dental insurance | Extended parental leave | Gym membershipSenior-level Full TimeBrazil1d ago
-
Research Engineer BRL 200K-220KAblation Studies | Code review | Data Curation | Data Validation | Data denoising5 days per week working | Collaborative team | Flexible working hours | Remote work | Supportive work environmentMid-level Full TimeBrazil1d ago
-
APIs | AWS | CI/CD | Connectors | GCPEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsMid-level Full TimeRecife, Brazil1d ago
-
Senior Analytics Engineer (Contractor - ASAP) BRL 305K-396KCI/CD | DBT | DBT Cloud | Data Governance | Data LineageFlexible scheduling | Fully remote | Long-term collaboration | Professional supportSenior-level ContractBrazil R1d ago
-
BRA Data Scientist Spec III BRL 20K-31KAPIs | Airflow | Apache Spark | Azure | CI/CDEnglish communication support | Hybrid work | Mentorship | Technical leadershipSenior-level Full TimeSANTANDER SEDE - 13 ANDAR, Brazil1d ago
-
Analista de Dados Pleno - 131550 BRL 96K-118KAWS | Agile | Apache Airflow | Azure | Azure DataFlexible work arrangement | Hybrid workMid-level Full TimeAlphaville - Barueri, BR, 06.454-0001d ago
-
Mid-level Full TimeSão Paulo, SP2d ago
-
CI/CD | Database backups | Database performance | Database performance tuning | Database replicationEducation budget | Fitness budget | Flextime | Growth roadmap | MentorshipEntry-level Full TimeSão Paulo, Brazil3d ago
-
CI/CD | Data Integrity | Database backups | Database performance | Database performance tuningEducation budget | Fitness budget | Flextime | Mentorship | Personalized growth roadmapEntry-level Full TimeRio de Janeiro, Brazil3d ago
-
AWS Glue | AWS RDS | CI/CD | Cloud SQL | Data IntegrityEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsEntry-level Full TimePorto Alegre, Brazil3d ago
-
AWS RDS | CI/CD | Cloud SQL | Database backups | Database performanceFlextime | Mentorship | Personalized growth roadmap | Team activities | TechtalksEntry-level Full TimeCuritiba, Brazil3d ago
-
CI/CD | Data Migration | Database backups | Database performance | Database performance tuningEducation budget | Fitness budget | Flextime | Mentorship | Personalized growth roadmapsEntry-level Full TimeCampinas, Brazil3d ago
-
AWS Glue | AWS RDS | CI/CD | Cloud SQL | Data MigrationEducation budget | Fitness and team activities | Flextime | Mentorship | Modern projectsEntry-level Full TimeFlorianópolis, Brazil3d ago
-
CI/CD | Data Migration | Database backups | Database performance | Database performance tuningFlexible schedule | Mentorship | Personalized growth roadmaps | Tech education budget | TechtalksEntry-level Full TimeRecife, Brazil3d ago
-
CI/CD | Database backups | Database performance | Database performance tuning | Database replicationEducation budget | Fitness budget | Flextime | Mentorship | Personalized growth roadmapsEntry-level Full TimeSalvador, Brazil3d ago
-
CI/CD | Data Migration | Database Backup | Database performance | Database performance tuningEducation budget | Fitness budget | Flextime | Mentorship | Personalized growth roadmapsEntry-level Full TimeBelo Horizonte, Brazil3d ago
-
Backup and Recovery | CI/CD | Data Integrity | Data Migration | Database performanceEducation budget | Fitness budget | Flextime | Mentorship | Personalized growth roadmapsEntry-level Full TimeBrasília, Brazil3d ago