Databricks Engineer
Tasks
- Build and maintain ETL/ELT processes
- Collaborate with data consumers on SLAs and data contracts
- Design and implement data pipelines
- Develop Delta Lake data models
- Ensure data quality and reliability
- Implement CI CD pipelines for Databricks
- Implement data governance with Unity Catalog
- Integrate Databricks with cloud storage services
- Manage and optimize Databricks clusters
- Optimize Spark workloads
- Troubleshoot Spark performance issues
- Work with structured and semi-structured data
Perks/Benefits
- N/A
Skills/Tech-stack
AWS S3 | Airflow | Apache Spark | Auto Loader | Avro | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure DevOps | CI/CD | CloudFormation | DBT | Data Governance | Data Lake Storage | Databricks | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | ELT | ETL | GCP BigQuery | GitHub Actions | IAM | Infrastructure as Code | JSON | MLflow | Parquet | PySpark | SQL | Scala | Structured Streaming | Terraform | Unity Catalog | “as-code”
Education
N/A
Roles
Related jobs
-
Data Engineer (Python | Airflow | Machine Learning) BRL 214K-252KAmazon Redshift | Amazon Web Services | Apache Airflow | Apache Kafka | CI/CDEnglish program | Fitness program | Home office | International projects | Remote workMid-level Full TimeSão Paulo R21h ago
-
Data Scientist Specialist (Lending) BRL 184K-184KA/B | A/B Testing | AUC | AWS | Apache SparkDental insurance | Gym membership | Health insurance | Home-office allowance | Life insuranceSenior-level Full TimeBrazil - Remote R1d ago
-
Data Engineer Azure Senior BRL 78K-129KARM | Apache Kafka | Azure Data | Azure Data Factory | Azure Data LakeSenior-level Full TimeTWGSP - Teleworker/Offsite - BRA - … R1d ago
-
Engenheiro de Dados Azure BRL 48K-80KAzure Blob | Azure Blob Storage | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimeTWGSP - Teleworker/Offsite - BRA - … R4d ago
-
AI Data Engineer BRL 150K-240KAWS | AWS EventBridge | AWS Lambda | AWS S3 | Automated testingDEI initiatives opportunities | Fully remote or hybrid work | Healthcare stipend | National holidays | Paid mental health recharge daysMid-level Full TimeRemote - Brazil R5d ago
-
A/B | A/B Testing | AWS | Azure | B testingCompany-provided laptop | Fully remote work setup | Home office stipend | Learning and development budget | Paid Maternity LeaveSenior-level Full TimeBrazil R5d ago
-
Senior AI Engineer BRL 234K-295KAWS | Anthropic API | Async Programming | Azure | Chain orchestrationSenior-level Full TimeBrazil R5d ago
-
Senior AI Software Engineer (.NET) - OP01998 BRL 234K-275K.NET | AWS | Agile | Azure | C#30 paid days off per year | DevPro University programs | English lessons | Overtime coverage | Paid sick daysSenior-level Full TimeSão Paulo, São Paulo, Brazil - … R5d ago
-
Senior Analytics Engineer BRL 78K-108KA/B | A/B Testing | Apache Airflow | B testing | BigQueryRemote workSenior-level Full TimeBrasil R5d ago
-
Principal Data Engineer Team Lead - OP02110 BRL 271K-370KAWS | Apache Airflow | Code Reviews | DBT | DagsterEnglish lessons | Health insurance | Paid sick days | Paid time off | Remote workSenior-level Full TimeSão Paulo, São Paulo, Brazil - … R6d ago
-
Senior Data Engineer | Unverified Partner Content Team BRL 156K-216KAmazon EMR | Apache Airflow | Big Data | Data Enrichment | Data ModelingCareer growth learning opportunities | Dental insurance | Flexible work options | Healthcare insurance | Home office reimbursementSenior-level Full TimeBrazil (Remote) R7d ago
-
Senior MLOps Engineer – Digital Transformation (Brazil) BRL 234K-295KAWS | Apache Airflow | Azure | CI/CD | DockerFlexible schedule | Paid time off | Remote work | Work with top companiesSenior-level Full TimeSão Paulo R7d ago
-
Azure | Azure DevOps | Bash | CI/CD | DatabricksCareer growth | Collaborative and international engineering culture | Continuous learning and innovation | Full-time permanent position | Fully remote work modelSenior-level Full TimeBrazil R8d ago
-
Analytics Engineer BRL 23K-23KAWS | Big Data | Data Governance | Data Modeling | Data QualityCareer development support | Casual environment | Dental insurance | Health insurance | Meal benefit cardEntry-level Full TimeHybrid, SP R11d ago
-
AWS CloudFormation | Airflow | Amazon Kinesis | Amazon Redshift | BigQueryFlexible hours | Remote workMid-level Full TimeBrazil - Remote R11d ago
-
Senior Data Engineer BRL 304K-370KAWS Lambda | Amazon RDS | Amazon S3 | Amazon Web Services | Apache SparkEquipment stipend | Flexible PTO | Learning and development stipend | Paid certifications exams | Professional development planSenior-level Full TimeBRAZIL R12d ago
-
Airflow | Automated testing | BigQuery | CI/CD | Cloud ComposerRemote workMid-level Full TimeRemoto, Brasil R12d ago
-
Staff Data Engineer BRL 325K-443KAWS Glue | AWS Glue Catalog | Amazon MWAA | Amazon S3 | Apache AirflowRemote workSenior-level Full TimeSão Paulo, SP, Brazil R12d ago
-
Sr Analytics Engineer BRL 230K-295KAmazon Redshift | Apache Spark | DBT | Data Governance | Data MartsDental insurance | Health insurance | Home-office allowance | Language classes | Life insuranceSenior-level Full TimeBrazil - Remote R13d ago
-
Senior Data Engineer (SQL DBA) BRL 234K-368KAccess Control | Auditing | Backup and Recovery | Data Governance | Database AdministrationSenior-level Full TimeRio de Janeiro, RJ, Brazil R18d ago
-
Senior Data Governance - Remoto | Inglês BRL 80K-114KAmazon Web Services | Amazon Web Services (AWS) | Data Catalog | Data Classification | Data GovernanceCareer growth | Inclusive culture | Remote workSenior-level Full TimeBrazil - Sao Paulo R19d ago
-
Staff AI Engineer (Remoto) MXN 420K-480KAPIs | AWS | Amazon SageMaker | Azure | CI/CDChild care assistance | Dental insurance | Education benefit | Flexible schedule | GympassSenior-level Full TimeFlorianopolis, Santa Catarina, Brasil R19d ago
-
Senior-level Full TimeBrazil - Remote R20d ago
-
Agile | Anomaly Detection | Azure Data | Azure Data Factory | Azure DatabricksSenior-level Full TimeBrazil - Remote R20d ago
-
AWS CloudFormation | AWS Glue | Amazon EMR | Amazon Kinesis | Amazon RDSStock optionsSenior-level Full TimeRemote (Brazil) R21d ago