Data Engineer Manager | Azure Databricks
Sao Paulo, Torre Paineira, Brazil
BRL 96K-119K (estimate) Mid-level Full Time
Tasks
- Build batch data ingestion pipelines
- Collaborate with multidisciplinary teams in Agile
- Convert legacy scripts to modern data architectures
- Create scalable resilient data pipelines with Apache Spark and Databricks
- Develop data migration workloads to Azure Databricks
- Develop data transformations with Python SQL and Spark
- Ensure data quality, traceability, and documentation
- Follow architecture standards and technical leadership patterns
- Implement ETL/ELT pipelines
- Implement data observability monitoring and governance
- Integrate with corporate systems APIs and relational databases
- Model data for Lakehouse using Delta Lake
- Modernize Python scripts
- Run unit and integration tests for migrated assets
- Support UAT stabilization and defect fixing
Perks/Benefits
- Birthday off
- Career mentoring
- Childcare assistance
- Life insurance
- Meal allowance
- Medical and dental insurance
- Multibenefit card
- Private pension
- Vehicle for managerial level
- Wellbeing Sessions
Skills/Tech-stack
Agile | Apache Spark | Azure | Azure Databricks | Cloudera | Delta Lake | ELT | ETL | Git | Hadoop | Kanban | Python | REST APIs | Relational databases | SQL | Scrum
Education
N/A
Related jobs
-
AWS Athena | AWS EC2 | AWS EMR | AWS Kinesis | AWS LambdaCareer Development Programs | Childcare support | Dental insurance | Extended family leave | Food VoucherMid-level Full TimeSão Paulo, São Paulo, Brazil23h ago
-
Senior Manager, Visa Consulting and Analytics (VCA), Core Consulting Practice, São Paulo or Brasilia, Brazil BRL 140K-146KAgile | Business Process | Business process improvement | Communication | ConsultingBi lingual support | Collaborative team environment | In-office work scheduleSenior-level Full TimeBrasilia, Brazil1d ago
-
Arquitetura de Dados | Híbrido/SP - 135094 BRL 118K-118KAzure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageHybrid work modelSenior-level Full TimeAlphaville - Barueri, BR, 06.454-000 R2d ago
-
AWS | Amazon DynamoDB | Amazon Elasticsearch | Amazon Kinesis | Amazon RedshiftSenior-level Full TimeSao Paulo, Sao Paulo, BRA3d ago
-
Data Product Manager | Machine Learning Platform BRL 98K-107KAzure | CI/CD | Confluence | Data Drift | Data Drift DetectionChildcare subsidies | Dental insurance | Discounts on company products | Health insurance | Language and training platformsMid-level Full TimeRemote, Brazil R6d ago
-
ABAC | AWS | Airflow | Amazon Web Services | Apache SparkChildcare assistance | Continuous learning platform | Dental insurance | Discount club | Extended parental leaveSenior-level Full TimeBrazil7d ago
-
AWS | Alteryx | Amazon Redshift | Azure Synapse | Azure Synapse AnalyticsContinuous learning | Fully remote work | Professional growthSenior-level Full TimeBrazil R7d ago
-
Artificial Intelligence | Cloud Computing | Data Architecture | Data Engineering | Data ModelingEmployee wellbeing programs | Inclusion and diversity | Psychological safety culture | Remote workSenior-level Full TimeRemoto, Brasil R7d ago
-
AI | Backend Development | BigQuery | Data Science | JavaMid-level Full TimeSudeste,Brazil8d ago
-
Especialista em Data & AI | Asset Servicing - Innovation BRL 108K-119KAgent systems | Amazon Athena | Apache Airflow | Apache Iceberg | Conversational AnalyticsFull onsite workSenior-level Full TimeSão Paulo, Brazil9d ago
-
Data Engineer Manager BRL 13K-18KAzure | Cloud Architecture | Data Architecture | Data Engineering | Data GovernanceChildcare support | Corporate university | Dental insurance | Educational partnerships | Health insuranceMid-level Full TimeOSASCO, Brazil; CURITIBA, PR, Brazil; OSASCO, …14d ago
-
Analytical modeling | B2B | Cloud | Credit Risk | Data LakeCareer development | Health and wellness benefits | Hybrid workMid-level Full TimeSão Paulo, Brazil14d ago
-
Specialist - Data Engineering USD 95KAPIs | Async workflows | Azure | Azure Data | Azure Data FactoryCollaborative work environment | English fluency support | Hybrid work model | Professional developmentMid-level Full TimeSao Paulo - EZTowers, Brazil R15d ago
-
Agile | Apache Airflow | BigQuery | CI/CD | Cloud Storage1 on 1 feedbacks | Career development | Flexible working environment | Hybrid workSenior-level Full TimeSão Paulo16d ago
-
Trader Pleno Oleaginosas BRL 21K-22KBasis Analysis | Commodities trading | Data Analysis | Demand analysis | ExcelMid-level Full TimeSao Paulo, SP, Brazil17d ago
-
Complexity analysis | Data Analysis | Machine Learning | Optimization | Parallel ComputingSenior-level Full TimeSao Paulo, SP, BR, 05423-01018d ago
-
Data Governance & Platform Manager USD 75K-100KAccess Control | Amazon Redshift | Apache Airflow | DBT | Data ContractsAI tools access | Self serve analytics autonomySenior-level Full TimeBrazil - Remote R22d ago
-
Gerente de Dados e Analytics BRL 18K-18KAWS | Apache Spark | Cloud Computing | Data Engineering | Data ModelingChildcare allowance | Dental insurance | Employee assistance program | Extended maternity leave | Extended paternity leaveMid-level Full TimeSão Paulo, São Paulo, Brazil30d ago
-
Manager Machine Learning Platform - Bees Data BRL 200K-228KAlerting | Azure | Azure DevOps | BentoML | CI/CDCasual office and dress code | Childcare subsidies | Clube Ben partnership | Days off | Discounts on Ambev productsMid-level Full TimeCampinas, Brazil1mo ago
-
Data Analytics & AI Solutions Group Leader BRL 210K-210KAgentic AI | Agile | Automation | Data Engineering | Data GovernanceFlexible location | Remote workSenior-level Full TimeSao Paulo, SP, Brazil R1mo ago
-
Senior-level Full TimeBRA-Barueri, Brazil1mo ago
-
Java + Data Technical Leader BRL 304K-370KAWS | Agile | Apache Kafka | Apache Spark | CI/CDCareer development opportunities | International projects | Multicultural environment | Training opportunitiesSenior-level Full TimeRio de Janeiro, RJ, Brazil1mo ago
-
Gerente de People Analytics BRL 20K-31KAnalytics | Artificial Intelligence | Business Intelligence | Data Architecture | Data GovernanceChildcare support | Dental plan | Employee Assistance Program (EAP) | Extended maternity leave | Extended paternity leaveSenior-level Full TimeSão Paulo1mo ago
-
Software Engineering Manager BRL 113K-132KAWS | Acceptance Testing | Angular | CI/CD | CheckmarxHybrid work modelMid-level Full TimeSão Paulo, SP, Brazil R1mo ago
-
Data & Analytics Manager BRL 102K-107KArtificial Intelligence | Data Analysis | Data Architecture | Data Automation | Data GovernanceDay off on birthday | Health insurance | Home-office allowance | Life insurance | Meal voucherMid-level Full TimeSão Paulo, SP, Brazil1mo ago