10938 - Engenheiro(a) de Dados Pleno – Databricks - Pleno
Tasks
- Apply Spark and Databricks performance best practices
- Develop and optimize distributed data loads
- Develop data ingestion pipelines
- Implement data read and write routines
- Integrate data pipelines with Azure Data Factory
- Transform and process structured and unstructured data
- Troubleshoot performance partitioning and distributed processing issues
- Use Delta Lake for data storage and versioning
- Write SQL queries for data transformation and analysis
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Spark | Avro | Cluster computing | Column Pruning | Databricks | Delta Lake | Distributed Computing | Narrow Transformations | Parquet | Predicate pushdown | SQL | Shuffle | Time Travel | Wide Transformations | Window Functions
Education
N/A
Roles
Related jobs
-
Senior Data Engineer ID71671 BRL 305K-390KAWS Kinesis | Apache Airflow | Apache Kafka | CI/CD | DagsterFlexible hours | Learning budget | Mentorship | Remote work | Well-being programsSenior-level Full TimeSão Paulo, Brazil7h ago
-
AWS | Apache Spark | Artificial Intelligence | Cloud Computing | Cloud platformChildcare support | Corporate university | Dental insurance | Educational partnerships | Health insuranceMid-level Full TimeCURITIBA, Brazil; CURITIBA, PR, Brazil; OSASCO, …21h ago
-
Automated Workflows | CI/CD | Cloud Native | Data Lineage | Deep learningCollaborative environment | Inclusive workplace | Professional growth opportunitiesSenior-level Full TimeBrazil1d ago
-
Access Control | Azure | Azure Bicep | Azure Data | Azure Data FactoryCareer growth | Collaborative cross-functional environment | Exposure to modern data technologies | Hands on cloud engineering | Professional developmentMid-level Full TimeBrazil1d ago
-
AIP | Agent workflows | Dashboards | Data Pipelines | DatabricksContract extension opportunities | Fully remote | Training and knowledge transferSenior-level Full TimeBrazil R1d ago
-
API Integration | Ariba APIs | Azure | CI/CD | Data GovernanceChildcare assistance | Continuous learning platform | Dental insurance | Discount club benefits | Extended parental leaveSenior-level Full TimeBrazil1d ago
-
Business Intelligence II - Dados e Analytics BRL 13K-18KAWS | Artificial Intelligence | Azure | Databricks | GCPChildcare allowance | Corporate university | Dental insurance | Educational partnerships | Extended parental leaveMid-level Full TimeOSASCO, Brazil; OSASCO, SP, Brazil; OSASCO1d ago
-
Data Engineer | Remota - 135416 INT BRL 118K-118KApache Airflow | DBT Core | Data Compliance | Data Governance | Data QualitySenior-level Full TimeAlphaville - Barueri, BR, 06.454-0001d ago
-
Data Engineer BRL 122K-146KAgile | Azure Data | Azure Data Factory | CI/CD | DBTChild care | Employee assistance program | Funeral assistance | Health insurance | Life insuranceMid-level Full TimeCuritiba, PR, BR, 80420-0001d ago
-
Engenheiro(a) de Dados Júnior BRL 21K-24KAirflow | Azure Data | Azure Data Factory | Data Factory | Data GovernanceChildcare support | Dental plan | Employee benefits club | Health insurance | Life insuranceEntry-level Full TimeSão Paulo, SP1d ago
-
AWS Glue | AWS IAM | Amazon EMR | Amazon Redshift | Amazon S3Birthday day off | Childcare support | Dental insurance | Gym membership | Health insuranceSenior-level Full TimeBelo Horizonte, MG; Curitiba, PR; Recife, …1d ago
-
Senior Data Engineer BRL 271K-370KAPI Development | AWS | Agile | Alerting | AzureCasual office and dress code | Childcare subsidies | Days off | Discounts on company products | Health, dental, and life insuranceSenior-level Full TimeRemote, Brazil R2d ago
-
Engenheiro de Dados Jr BRL 21K-25KAzure Data | Azure Data Factory | Azure Databricks | Common table expressions | DAXCareer growth | Inclusive work environmentEntry-level Full TimeBrazil - Sao Paulo2d ago
-
Senior Data Engineer ID81652 BRL 319K-396KAirflow | CRM | DBT | Data cloud | GCPAnnual learning budget | Collaborative culture | Flexible hours | Internal technical talks | MentorshipSenior-level Full TimeBelo Horizonte, Brazil2d ago
-
Senior Data Engineer ID81652 BRL 319K-396KApache Airflow | DBT | Data cloud | GCP | JiraCollaborative culture | Flexible hours | Learning budget | Mentorship | Performance reviewsSenior-level Full TimeSorocaba, Brazil2d ago
-
Senior Data Engineer ID81652 BRL 319K-396KAirflow | CRM data | DBT | Data Modeling | Data cloudFlexible hours | Learning budget | Mentorship | People focused support | Remote workSenior-level Full TimeRecife, Brazil2d ago
-
Lead Data Engineer ID71008 BRL 319K-396KAI Assisted Development | AI-Assisted Development Tools | Airflow | Amazon Athena | Amazon EKSAnnual learning budget | Collaborative culture | Flexible hours | Local well being programs | MentorshipSenior-level Full TimeSão Bernardo do Campo, Brazil2d ago
-
Lead Data Engineer ID71008 BRL 319K-396KAWS Bedrock | Airflow | Amazon S3 | Athena | BigQueryAnnual learning budget | Collaborative culture | Flexible hours | Internal TechTalks | MentorshipSenior-level Full TimeSão Paulo, Brazil2d ago
-
Lead Data Engineer ID71008 BRL 319K-396KAirflow | Amazon S3 | Apache Spark | Athena | BigQueryAnnual learning budget | Flexible hours | Learning budget | Local well being support | MentorshipSenior-level Full TimeSalvador, Brazil2d ago
-
Lead Data Engineer ID71008 BRL 319K-396KAI Assisted Development | AI-Assisted Development Tools | AWS Athena | Airflow | Amazon S3Annual learning budget | Collaborative culture | Flexible hours | Internal TechTalks | MentorshipSenior-level Full TimeBrasília, Brazil2d ago
-
Lead Data Engineer ID71008 BRL 319K-396KAWS S3 | AWS Step Functions | Amazon Athena | Apache Airflow | BigQueryAnnual learning budget | Flexible hours | Mentorship | People focused support | Performance reviewsSenior-level Full TimeCampinas, Brazil2d ago
-
Lead Data Engineer ID71008 BRL 319K-396KAI code assistant | AWS EKS | Amazon Athena | Amazon S3 | Apache AirflowAnnual learning budget | Flexible hours | Internal tech talks | Mentorship | People focused supportSenior-level Full TimeRecife, Brazil2d ago
-
Lead Data Engineer ID71008 BRL 319K-396KAI Assisted Development | AI-Assisted Development Tools | AWS Step Functions | Amazon Bedrock | Amazon S3Annual learning budget | Annual performance and compensation reviews | Collaborative culture | Flexible hours | Local supportSenior-level Full TimeBelo Horizonte, Brazil2d ago
-
Lead Data Engineer ID71008 BRL 319K-396KAirflow | Amazon S3 | Athena | BigQuery | ClickHouseAnnual learning budget | Flexible hours | Mentorship | Performance reviews | Remote workSenior-level Full TimeCuritiba, Brazil2d ago
-
Lead Data Engineer ID71008 BRL 319K-396KAWS Bedrock | Apache Airflow | Athena | BigQuery | CachingAnnual performance reviews | Flexible hours | Learning budget | Mentorship | People focused supportSenior-level Full TimePorto Alegre, Brazil2d ago