Product Manager - AI Evaluation & Developer Platforms
São Paulo, State of São Paulo, Brazil
A BRL 271K-370K (estimate) Senior-level Full Time
Tasks
- Build evaluation and developer tooling systems
- Collaborate with engineering on delivery cycles
- Create benchmarks and scoring methodologies
- Define AI quality standards
- Define and monitor KPIs and OKRs
- Deliver evaluation dashboards and developer experiences
- Develop human feedback annotation workflows
- Enable dataset creation experiment execution and results analysis
- Ensure traceability from requirements to delivered capabilities
- Own AI evaluation roadmap
- Present findings to stakeholders
- Track tool adoption effectiveness and business impact
- Write product requirements and acceptance criteria
Perks/Benefits
- N/A
Skills/Tech-stack
AI Agents | AI Quality Assurance | AI quality | API Integration | Agile | Annotation | Artificial Intelligence | Benchmarking | Data Governance | Data Ingestion | Data Quality | Data quality monitoring | ETL | Evaluation methodology | Human Feedback | Knowledge of Metrics | LLM | Language Models | Large Language Models | Machine Learning | Product Management | Quality Assurance | Quality monitoring | RAG | Retrieval-Augmented Generation | Test strategy
Education
N/A
Roles
Related jobs
-
Senior AI Engineer BRL 300K-370KAI Services | API Management | Agentic Applications | Apache Spark | Azure AICareer advancement | Dental insurance | Health insurance | Health savings account contributions | Hybrid workSenior-level Full TimeBrazil - Remote R16h ago
-
Consultor(a) Artificial Intelligence BRL 22K-22KAI Search | App Service | Artificial Intelligence | Azure AI | Azure AI SearchBirthday day off | Career mentoring | Childcare support | Company results bonus | Dental insuranceMid-level Full TimeSao Paulo, Torre Paineira, Brazil1d ago
-
Dashboards | Data Governance | Data Lakes | Data Literacy | Data ModelingMid-level Full TimeSão Paulo, State of São Paulo, …1d ago
-
AI Lead (Brazil) BRL 144K-146KAnalytics | Artificial Intelligence | Data Analysis | Language Processing | Machine LearningProfessional growth | Remote workSenior-level Full TimeRemote/Brazil R2d ago
-
Agentic AI | Cloud Computing | DB2 | Language Models | Language ProcessingFlexible work environment | Innovation-driven culture | Learning opportunities | Professional growthSenior-level Full TimeBrazil2d ago
-
API Integration | APIs | Azure Data | Azure Data Factory | CI/CDChildcare assistance | Continuous learning access | Dental insurance | Discounts and partnerships | Extended parental leaveSenior-level Full TimeBrazil2d ago
-
API Development | Agent systems | Auditing | Autonomous Agents | Data maskingBYOD support | Birthday day off | Continuous learning | Discount club access | Fully remoteSenior-level Full TimeBrazil R3d ago
-
[Job - 30169] AI Data Engineer Mid/Senior BRL 58K-60KAgent systems | Agentic Workflows | Azure Data | Azure Data Factory | Azure DatabricksChildcare support | Dental insurance | Discount club | Extended parental leave | GympassSenior-level Full TimeBrazil3d ago
-
Agent systems | Backend Development | CI/CD | Data Analysis | Deep learningAccess to educational materials for children | Childcare assistance | Dental insurance | Discounts for postgraduate study | Education incentivesSenior-level Full TimeBrazil4d ago
-
S&OP, efficiency and AI Leader (São Paulo/SP) BRL 144K-146K5S | Advanced Analytics | Artificial Intelligence | Automation | Chain managementRemote workSenior-level Full TimeRemote, Brazil R4d ago
-
Líder de Ciência de Dados BRL 21K-23KAnalytics | Artificial Intelligence | Data Science | Data analytics | MLOpsSenior-level Full TimeSão Paulo, SP, BR, 04730-0004d ago
-
Principal Data and AI Architect BRL 300K-360KA2A | Agent Builder | Agent interoperability | Agentforce Studio | Amazon RedshiftSenior-level Full TimeBrazil - Sao Paulo4d ago
-
Business Intelligence | Dashboard Development | Data Automation | Data Visualization | Data consolidationGrowth opportunities | Hybrid work | Learning opportunities | Supportive work environmentMid-level Full TimeSudeste,Brazil4d ago
-
API Integration | AWS | AWS Bedrock | Anthropic API | AutogenCareer growth | Continuous learning | Innovation-driven environment | Remote workSenior-level Full TimeBrazil R4d ago
-
API Integration | Application Insights | Async IO | Azure Blob | Azure Blob StorageCollaborative team | Professional growth opportunities | Remote-first work environment | Work from anywhereSenior-level Full TimeSão Paulo, São Paulo, Brazil - … R5d ago
-
AWS | Analytics | Billing | Cloud Native | ConfluenceAnnual birthday day off | Childcare support | Dental insurance | Health insurance | Life care supportMid-level Full TimeSao Paulo - Paulista, Brazil5d ago
-
Senior Business Analyst, AI & Data USD 140K-181KAcceptance Testing | Acceptance criteria | Adoption Management | Agile | AuditabilityOwnership and autonomy | Remote-first | Travel 50 percent per monthSenior-level Full TimeBrazil - Uruguay - Remote R5d ago
-
Agentic patterns | FastAPI | LLM Evaluation | Language Models | Large Language ModelsSenior-level Full TimeBrazil (Remote) R6d ago
-
ServiceNow Analyst – ServiceNow Conversational & GenAI BRL 300K-368KAI Agent | AI Agent Studio | AI Search | Agent Studio | Azure DevOpsDental plan | Employee resources group | Extended maternity leave | Extended paternity leave | Finance support programSenior-level Full TimePiracicaba, São Paulo, Brazil6d ago
-
AI Engineer - Forward Deployed Engineer BRL 319K-390KAPIs | Agentic AI | Apex | BigQuery | Data ModelingCareer growth | Employee benefits | Mentorship | Travel opportunitiesMid-level Full TimeBrazil - Sao Paulo6d ago
-
Mid-level Full TimeSao Paulo, SP, BR, 05423-0106d ago
-
GPM de Transformação Digital e AI Solutions BRL 117K-160KA/B | A/B Testing | Artificial Intelligence | B testing | ComplianceSenior-level Full TimeSão Paulo, Brazil6d ago
-
Azure | CI/CD | Docker | FastAPI | Generative AIChildcare assistance | Continuous learning platform | Dental insurance | Extended parental leave | Food allowanceSenior-level Full TimeBrazil6d ago
-
Gerente de People Analytics BRL 20K-31KAnalytics | Artificial Intelligence | Business Intelligence | Data Architecture | Data GovernanceChildcare support | Dental plan | Employee Assistance Program (EAP) | Extended maternity leave | Extended paternity leaveSenior-level Full TimeSão Paulo7d ago
-
Distributed Systems | Embeddings | Java | Kubernetes | LLM InferenceCollaborative flat structure culture | Direct access to technical leadership | Exposure to cutting edge generative AI technologies | High autonomy and flexibility | High ownership of projectsEntry-level Full TimeBrazil R7d ago