Databricks Data Engineer| Senior
Tasks
- Develop automated testing for data pipelines
- Document migrated business rules and workflows
- Implement Medallion architecture data layers
- Maintain coexistence until cutover
- Migrate ETL pipelines by business domain
- Optimize pipeline performance and cloud costs
- Perform data reconciliation and parity validation
- Rebuild legacy data warehouse pipelines
- Transform business logic into production pipelines
Perks/Benefits
Skills/Tech-stack
AWS Athena | AWS DMS | AWS EMR | AWS Glue | AWS Lambda | AWS S3 | AWS Step Functions | Asset bundles | Automated testing | Azure Pipelines | Azure SQL | Azure SQL Pools | Azure Synapse | Azure Synapse Analytics | Batch Ingestion | CDC | CI/CD | Data Quality | Data Reconciliation | Databricks | Delta Lake | Delta Live Tables | Delta Live) | ETL | Git | Lakeflow | Medallion Architecture | Partitioning | PySpark | Python | SQL Pools | Spark SQL | Step Functions | Synapse Analytics | Unity Catalog | Z-Order
Education
N/A
Roles
Related jobs
-
AI Assisted Development | AI-Assisted Development Tools | API Design | Automation | Development toolsBirthday day off | Comprehensive healthcare coverage | Continuous training opportunities | Extended paternity leave | Financial support programsMid-level Full TimeBrazil R11h ago
-
API Integration | Asynchronous processing | Authentication | Celery | DjangoFlexible, async culture | Founding team opportunity | Fully remote | High ownership | Influence on product directionSenior-level Full TimeBrazil R11h ago
-
Airflow | Azure | Azure Data | Azure Data Factory | Azure SynapseContinuous learning | Flexible schedule | Hybrid work | Professional growth opportunities | Remote workMid-level Full TimeBrazil11h ago
-
AWS Glue | AWS Lambda | AWS Step Functions | Amazon AWS | Amazon AWS GlueMid-level Full TimeAlphaville - Barueri, BR, 06.454-00012h ago
-
Engenheiro de Software Fullstack | BTG Empresas BRL 54K-72K.NET | AWS Lambda | Agile | Amazon ECS | Amazon EKSChildcare allowance | Dental insurance | Employee Assistance Program (EAP) | Extended maternity leave | Extended paternity leaveSenior-level Full TimeSão Paulo18h ago
-
[Job-30817] Senior AI Engineer, Brazil BRL 230K-265KAI Security | AI Steering | Agent Coordination Protocol | Agent Orchestration | Agent coordinationChildcare assistance | Continuous learning platform | Dental insurance | Discount club | Extended paternity leaveSenior-level Full TimeBrazil21h ago
-
Applied AI solutions Architect BRL 319K-443KCI/CD | Drift Detection | Feature Store | Generative AI | GovernanceOwnership opportunities | Professional learning and development | Remote work flexibilitySenior-level Full TimeBrazil1d ago
-
API Integration | Access Control | Audit Logging | Behavioral Testing | CI/CDAnnual company off-site | Annual professional development budget | Coworking allowance | Dependent coverage support | Employee equity ownershipMid-level Full TimeBrazil R1d ago
-
AWS | AWS Lambda | Agent workflows | Amazon SQS | Distributed SystemsBirthday leave | Career development opportunities | Dental insurance | Flexible benefits program | Flexible work scheduleSenior-level Full TimeBrazil1d ago
-
Analista Data Engineer BRL 60K-60KABAC | Apache Airflow | Azure | Azure Databricks | Azure Key VaultCareer mentoring | Childcare assistance | Dental insurance | Health insurance | Life insuranceSenior-level Full TimeSao Paulo, Torre Paineira, Brazil1d ago
-
Senior Analytics Engineer USD 66K-66KAirflow | Business Intelligence | Code review | DBT | DagsterEquity | Flexible time off | MacBook equipment | Paid Holidays | Remote-friendly workSenior-level Full TimeBrazil R1d ago
-
Senior Data Scientist / Deployment Engineer BRL 183K-183KC# | Data Pipelines | ETL | Fraud Detection | JavaExtended parental leave | Flexible remote and hybrid working options | Generous PTO | Learning and development opportunities | Mental health benefitsSenior-level Full TimeBrazil - Sao Paulo1d ago
-
Senior-level Full TimeBrazil1d ago
-
DevOps Engineer BRL 180K-240KAKS | AWS | AWS certification | Active Directory | AnsibleRemote workSenior-level Full TimeBrazil1d ago
-
Jr Data Engineer BRL 25K-25KAWS | Apache Airflow | Azure | CI/CD | Cloud ComputingChildcare support | Dental insurance | Extended parental leave | Gympass | Health insuranceEntry-level Full TimeSão Paulo,São Paulo,BRA1d ago
-
AWS ECS | AWS ECS Fargate | Agentic tools | Anthropic Claude | Async pipelinesAsync communication | Continuous learning | High ownership | Remote workMid-level Full TimeBrazil (Remote) R1d ago
-
[Job-30874] Senior AI Engineer, Brazil BRL 183K-183KAPI Integration | Agentic Systems | BPMN | Business Process | Business process modelingChildcare assistance | Continuous learning | Dental insurance | Gym membership discounts | Health insuranceSenior-level Full TimeBrazil1d ago
-
Software Engineer, Data & AI Applications BRL 305K-396KAPI | Access Control | Agent workflow | Airbyte | Automated testingFlexible work arrangements | Professional developmentSenior-level Full TimeBrazil R2d ago
-
Software Engineer, Data & AI Applications BRL 319K-412KAPI Development | Access Control | Agent workflow | Airbyte | Automated testingCollaborative team | Continuous learning | Flexible work arrangements | Professional developmentMid-level Full TimeBrazil R2d ago
-
APIs | Agent systems | Cloud Consulting | Cloud platform | Cost PerMid-level Full TimeSão Paulo, State of São Paulo, …2d ago
-
AWS | Airflow | CI/CD | Cloud platform | Distributed SystemsEnglish proficiency requirement | Mandarin plus | Remote workSenior-level Full TimeBrasília, Brazil R2d ago
-
AI SDR | Agile | CRM automation | Calendly API | JavaFlexible time zone alignment | Remote workSenior-level Full TimeBrazil / Argentina / Peru / …2d ago
-
Bash | Cloud infrastructure | Data Ingestion | Data Processing | DockerAsynchronous culture | Flexible work environment | Remote workMid-level Full TimeFlorianópolis, Brazil2d ago
-
Specialist Solutions Architect - AI/ML BRL 319K-455KAI guardrails | Agent Orchestration | Agentic Systems | Artificial Intelligence | Delta LakeMentorship | Remote work | Travel up to 30 percentSenior-level Full TimeSao Paulo, Brazil2d ago
-
Mid-level Full TimeBrazilia, Federal District, Brazil3d ago