Lead Data Engineer
Tasks
- Architect data pipelines and infrastructure in Azure
- Develop data transformation pipelines with Databricks and DBT
- Document data architecture and processes
- Drive observability and performance optimization
- Enforce data governance and engineering standards
- Establish data integration between Snowflake and Azure
- Implement CI/CD workflows
- Implement infrastructure as code with Terraform
- Lead and mentor data engineering team
- Migrate data from Snowflake to Azure Databricks and DBT
- Own data migration strategy
- Perform code reviews and enforce best practices
- Set up deployments and environment automation
- Troubleshoot data pipeline issues
Perks/Benefits
Skills/Tech-stack
Azure | Azure Databricks | CI/CD | Compliance | DBT | Data Catalog | Data Engineering | Data Governance | Data Lineage | Data Modeling | Data Quality | Delta Live Tables | Delta Live) | ELT | ETL | Event Hubs | Git | GitHub | Insurance Domain | Kafka | Lakehouse | Observability | SQL | Security | Snowflake | Terraform
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
AWS | Airflow | Amazon Redshift | Azure | BigQueryFlexible working hours | Remote workSenior-level Full TimeBrazil - Remote R22h ago
-
Data Engineer BRL 18K-18KAWS | Apache Spark | Azure | CI/CD | DatabricksDental insurance | Employee assistance | Flexible schedule | Gympass | Health insuranceSenior-level Full TimeBrasil1d ago
-
Staff Machine Learning Operations Engineer - Brazil BRL 230K-265KAWS | Alerting | Apache Airflow | Azure | CI/CDFlexible work hours | Home office setup support | Remote environment | Self-managed PTO | Work in your own time zoneSenior-level Full TimeBrazil R1d ago
-
Apache Airflow | Apache Spark | Data Lakes | Data Warehouses | ELTAccess to internal learning platform | Discounts on educational programs | Meal or food allowance | Medical and dental assistance | Mentoring and professional development programsSenior-level Full TimeBrazil1d ago
-
Engenheiro de Dados SR BRL 27K-40KBigQuery | CI/CD | Cloud Data | Cloud Data Pipelines | Cloud platformCareer development | Collaborative work environment | Employee well-being support | Training and workshops | Work-life balance supportSenior-level Full TimeSão Paulo, State of São Paulo, …1d ago
-
Airflow | Apache Spark | BigQuery | CI/CD | Cloud ComposerContinuous learning | Flexible schedule | Knowledge sharing | Professional development | Remote workMid-level Full TimeBrazil R1d ago
-
Apache Flink | Apache Kafka | Apache Spark | CI/CD | Data ModelingBirthday day off | Childcare support | Dental insurance | Educational partnerships | Extended parental leaveSenior-level Full TimeBrazil1d ago
-
Agile | Apache Hadoop | Apache Hive | Apache Spark | Azure BlobHybrid workSenior-level Full TimeAlphaville - Barueri, BR, 06.454-000 R1d ago
-
AWS | AWS Glue | AWS Lambda | AWS Step Functions | Amazon AthenaChildcare assistance | Dental insurance | Gym access | Health insurance | Language learning supportMid-level Full TimeBrazil1d ago
-
Apache Airflow | CI/CD | Cloud platform | DBT | Data LakehouseDental insurance | Employee discounts | Extended parental leave | Gym and wellness partnerships | Health insuranceSenior-level Full TimeBrazil1d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAmazon Redshift | Apache Airflow | Apache Hive | Apache Kafka | Apache SparkFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeSão Paulo, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAmazon Redshift | Apache Airflow | Apache Hive | Apache Kafka | Apache SparkFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeRio de Janeiro, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAirflow | Amazon Redshift | Apache Hive | Apache Spark | DBTFlexible schedule | Mentorship | Office options | Personalized growth roadmaps | Remote optionsSenior-level Full TimeRecife, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAmazon Redshift | Apache Airflow | Apache Hive | Apache Kafka | Apache SparkEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsSenior-level Full TimeCampinas, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAgile | Amazon Redshift | Apache Airflow | Apache Hive | Apache KafkaFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeBrasília, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAirflow | Amazon Redshift | Apache Spark | DBT | Data ArchitectureFlexible schedule | Mentorship | Personalized growth roadmaps | Professional growth | Remote and office optionsSenior-level Full TimeCuritiba, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAmazon Redshift | Apache Airflow | Apache Hive | Apache Kafka | Apache SparkFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeFlorianópolis, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAmazon Redshift | Apache Airflow | Apache Hive | Apache Kafka | Apache SparkFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeSalvador, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAmazon Redshift | Apache Airflow | Apache Hive | Apache Kafka | Apache SparkEducation budget | Fitness budget | Flextime | Mentorship | Office work optionSenior-level Full TimeBelo Horizonte, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAirflow | Amazon Redshift | Apache Spark | DBT | Data ArchitectureFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeSão Bernardo do Campo, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAmazon Redshift | Apache Airflow | Apache Hive | Apache Kafka | Apache SparkEducation budget | Fitness budget | Flextime | Mentorship | Office optionsSenior-level Full TimePorto Alegre, Brazil3d ago
-
Senior Data Engineer ID65436 BRL 304K-370KAgile | Amazon Redshift | Apache Airflow | Apache Hive | Apache KafkaEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsSenior-level Full TimeSorocaba, Brazil3d ago
-
Artificial Intelligence | Bias Mitigation | Bitbucket | Credit Risk | Data version controlSenior-level Full TimeSão Paulo, Brazil4d ago
-
CI/CD | Data Engineering | Data Governance | Data Modeling | Data ObservabilityBirthday day off | Childcare assistance | Culture leisure partnerships | Dental plan | Employee discountsSenior-level Full TimeBrazil4d ago
-
DevOps Architect BRL 240K-275KAmazon Linux | Ansible | Azure | Bash | CI/CDExchange program | Health insurance | Life insurance | OPD | Paid time offSenior-level Full TimeBrazil - Remote R4d ago