Product Manager - AI Evaluation & Developer Platforms
São Paulo, State of São Paulo, Brazil
A BRL 271K-370K (estimate) Senior-level Full Time
Tasks
- Build evaluation and developer tooling systems
- Collaborate with engineering on delivery cycles
- Create benchmarks and scoring methodologies
- Define AI quality standards
- Define and monitor KPIs and OKRs
- Deliver evaluation dashboards and developer experiences
- Develop human feedback annotation workflows
- Enable dataset creation experiment execution and results analysis
- Ensure traceability from requirements to delivered capabilities
- Own AI evaluation roadmap
- Present findings to stakeholders
- Track tool adoption effectiveness and business impact
- Write product requirements and acceptance criteria
Perks/Benefits
- N/A
Skills/Tech-stack
AI Agents | AI Quality Assurance | AI quality | API Integration | Agile | Annotation | Artificial Intelligence | Benchmarking | Data Governance | Data Ingestion | Data Quality | Data quality monitoring | ETL | Evaluation methodology | Human Feedback | Knowledge of Metrics | LLM | Language Models | Large Language Models | Machine Learning | Product Management | Quality Assurance | Quality monitoring | RAG | Retrieval-Augmented Generation | Test strategy
Education
N/A
Roles
Related jobs
-
Senior AI Prompt Engineer & GenAI Adoption Consultant BRL 300K-390KAI Agent | AI Agents | AI/Agent Platforms | API Integration | Agent PlatformsHybrid work flexibility | International projects | Medical healthcare | Ongoing learning reimbursement | Recognition programSenior-level Full TimeSão Paulo, Brazil13h ago
-
Lead Product Manager - AI Solutions BRL 126K-160KAPI platforms | Agile | Backlog prioritization | Confluence | Customer JourneysSenior-level Full TimeSão Paulo, Brazil23h ago
-
Agente de Integração IA - 134021 BRL 117K-119KAgile | Artificial Intelligence | CI/CD | Change Management | CloudOpsSenior-level Full TimeAlphaville - Barueri, BR, 06.454-0001d ago
-
AI Lead (Brazil) BRL 144K-146KArtificial Intelligence | Data Analysis | Language Processing | Machine Learning | Natural LanguageRemote-firstSenior-level Full TimeRemote/Brazil R2d ago
-
10773 - Product Owner / Data Product Manager – Produtos de Dados e Transformação Ágil - Sênior A BRL 80K-84KAgile | Data Product Management | Data analytics | Data product | KanbanHybrid workMid-level Full TimeSão Paulo, Brasil3d ago
-
Software Engineering Manager II, AI Productivity BRL 319K-443KAI Evaluation | Agentic Platforms | Automated Red Teaming | Code review | Generative AISenior-level Full TimeSão Paulo, State of São Paulo, …3d ago
-
AI Engineer BRL 74K-129KAWS | Anthropic API | Azure | CI/CD | Cloud platformChildcare assistance | Dental insurance | English classes | Food vouchers | Gym discountsSenior-level Full TimeBrazil4d ago
-
A/B | A/B Testing | B testing | Docker | EmbeddingsCollaborative environment | Continuous learning | Cutting-edge AI exposure | High-impact role | Remote first work within BrazilSenior-level Full TimeBrazil4d ago
-
Computer Vision | Data Pipelines | MLOps | Machine Learning | Model versioningHybrid work | Innovation teamMid-level Full TimeSão Paulo, State of São Paulo, … R4d ago
-
Engenheiro(a) de IA [Agents & LLM] BRL 60K-96KAI RMF | AWS | Agent systems | Autogen | AzureChildcare support | Dental insurance | Extended maternity leave | Extended paternity leave | GympassMid-level Full TimeSao Paulo, Torre Paineira, Brazil4d ago
-
Product Manager – AI / Analytics - LATAM PHP 150K-200KAgentic AI | Agile | Artificial Intelligence | Business Intelligence | Data analyticsHealth insurance | Professional developmentSenior-level Full TimeLOC5028: MSSC, Boulevard Díaz Ordaz 130, …4d ago
-
Machine Learning Engineer - AI Applications BRL 305K-396KAgile methodology | Azure Cloud | CI/CD | Deep learning | ElasticsearchMid-level Full TimePOA05 - RS POA 95 (POA05), …4d ago
-
Contract AI Developer USD 50K-60KAgentic Workflows | Anthropic API | Async Programming | Automated testing | Claude CodeMid-level ContractBrazil5d ago
-
[Job-29568] AI Developer com foco em .NET PL/ SR BRL 23K-24K.NET | AI code generation | API Integration | AWS | Agentic WorkflowsChildcare assistance | Continuous learning platform | Dental insurance | Extended parental leave | Gym membership discount programMid-level Full TimeBrazil5d ago
-
AWS Lambda | AWS S3 | Amazon EventBridge | Automated data quality | Automated data quality checksDEI initiatives | Employee development programs | Fully remote | Healthcare stipend | Hybrid workMid-level Full TimeRemote - Argentina; Remote - Brazil; … R5d ago
-
AI Coding Subject Matter Expert BRL 192K-210KAI Agents | AWS | Application Security | CI/CD | JavaScriptReward & recognition | Volunteering | Wellness | Work-life balanceSenior-level Full TimeSão Carlos, SP, Brazil5d ago
-
GPM de Transformação Digital e AI Solutions BRL 126K-172KAI Agents | Application development | Artificial Intelligence | Compliance | Data GovernanceSenior-level Full TimeSão Paulo, Brazil5d ago
-
AI Agents | AWS | CI/CD | Cloud infrastructure | DockerChildcare assistance | Continuous learning | Dental insurance | External online courses | Fitness discountsSenior-level Full TimeBrazil5d ago
-
Analyst Applications – ServiceNow Conversational & GenAI BRL 253K-390KAI Agent | AI Agent Studio | AI Search | Agent Studio | ArchitectureDental plan | Employee resource group support | Extended maternity and paternity leave | Life and personal insurance | Meal assistanceSenior-level Full TimePiracicaba, São Paulo, Brazil5d ago
-
.NET | API Integration | AWS CloudFormation | Agent Orchestration | Agent systemsChildcare assistance | Dental insurance | Extended parental leave | Gympass/Wellhub | Health insuranceSenior-level Full TimeBrazil6d ago
-
A/B | A/B Testing | B testing | Caching | Citation GroundingSenior-level Full TimeCuritiba, State of Paraná, Brazil6d ago
-
AI Architect (LatAm) BRL 212K-270KAI Agents | AI copilots | AI-assisted coding | Agentic Workflows | Artificial IntelligenceDirect Manager Mentorship | Knowledge sharing | Paid sick leave | Paid vacation | Performance reviewsSenior-level Full TimeBrazilia, Federal District, Brazil6d ago
-
Dashboards | Data Modeling | Data Science | Data Visualization | Data analyticsDental insurance | Flexible work arrangement | Health insurance | Learning and development | Life insuranceMid-level Full TimeBrazil6d ago
-
API Integration | AWS Neptune | Generative AI | Graph Database | GraphQLInclusive culture | Remote workMid-level Full TimeBrazil - Sao Paulo R6d ago
-
Operations Tech & AI Manager - Logistics BRL 181K-210KAutogen | Automated Alerts | CrewAI | Dashboarding | Data AutomationHybrid workSenior-level Full TimeSao Paulo, SP, Brazil6d ago