Especialista de Engenharia de Dados II – Tech Lead
Tasks
- Build real time ingestion with Spark Structured Streaming
- Conduct technical code and architecture reviews
- Create Data Mesh standards and best practices
- Define lakehouse architecture
- Design batch and streaming pipelines architectures
- Develop scalable pipelines in Python Scala SQL and Java
- Ensure data quality
- Ensure scalability security governance and performance
- Establish data observability monitoring alerts metrics
- Facilitate discovery and design for data products
- Implement AWS Lake Formation governance
- Implement Delta Live Tables
- Lead and mentor data engineers
- Lead internal workshops and technical communities
- Manage data access with AWS IAM
- Set up Kafka and Kinesis ingestion
- Support CI CD and versioning for pipelines
- Translate business needs into scalable data products
- Use Unity Catalog for data governance
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS Glue | AWS IAM | AWS Lake Formation | AWS Lambda | Alerting | Amazon Kinesis | Amazon S3 | Apache Spark | CI/CD | Data Governance | Data Observability | Data Quality | Data mesh | Databricks | Delta Lake | Delta Live Tables | Delta Live) | Java | Kafka | Lake Formation | Lakehouse | Metrics | Monitoring | Python | SQL | Scala | Structured Streaming | Unity Catalog | Version control
Education
N/A
Roles
Architect | Data Engineer | Data Platform | Data Platform Architect | Engineer | Lead | Platform Architect | Tech Lead
Related jobs
-
[Job 28454] IA Data Engineer, Pleno BRL 20K-23KAutogen | Azure | Azure Data | Azure Data Factory | Azure DevOpsChildcare support | Dental insurance | Extended parental leave | Gympass | Health insuranceNone Full TimeBrazil13h ago
-
AI Engineer I (Remote) USD 30K-39KAWS | CI/CD | Docker | Entity recognition | Feature EngineeringFull-time employment | Remote workEntry-level Full TimeBrazil, Remote R17h ago
-
Machine Learning Engineering Specialist BRL 80K-80KAWS | Azure | Batch Processing | CI/CD | DatadogAdditional day off on birthday | Dental insurance | Education assistance | Flexible work culture | Language classesMid-level Full TimeCuritiba or São Paulo | On-site18h ago
-
Lead Data Engineer BRL 305K-396KBig Data | Continuous Delivery | Data Architecture | Data Governance | Data LakeSenior-level Full TimeBrazil, Brazil18h ago
-
[Job - 28500] Senior Data Analytics, Brazil BRL 20K-40KAWS | Amazon QuickSight | Data Exploration | Data Modeling | Data QualityChildcare support | Dental insurance | Discount club | Extended parental leave | GympassSenior-level Full TimeBrazil20h ago
-
AWS | AWS Glue | AWS Lambda | Amazon Athena | Amazon EC2Childcare support | Continuous learning platform | Dental insurance | Extended parental leave | Gym membership discountMid-level Full TimeBrazil21h ago
-
[Job-28403] Sr DataBricks DATAOps Engineer, Brazil BRL 212K-270KAzure DevOps | CI/CD | Cloud Engineering | Code review | Data GovernanceChildcare assistance | Continuous learning platform | Dental insurance | Discount club | Extended paternity leaveSenior-level Full TimeBrazil22h ago
-
Amazon Athena | Amazon Redshift | Amazon S3 | Apache Airflow | BashAnnual profit-sharing | Childcare support | Dental insurance | Education budget | Extended maternity leaveMid-level Full TimeBRASIL, SÃO PAULO; BRASIL,BELO HORIZONTE23h ago
-
Especialista de Engenharia de Dados II BRL 23K-23KAPI Development | AWS Glue | AWS Lake Formation | AWS Lambda | Amazon AthenaCollaboration culture | Development opportunities | Wellness | Work-life balanceEntry-level Full TimeSão Paulo, Brazil23h ago
-
APIs | AWS | Backend Development | CI/CD | Cost OptimizationChildcare support | Dental insurance | Education budget | Extended maternity leave | Extended paternity leaveMid-level Full TimeBRASIL,BELO HORIZONTE23h ago
-
Entry-level Full TimeSão Paulo, Brazil1d ago
-
Data Scientist Specialist (Lending) BRL 184K-184KA/B | A/B Testing | AUC | AWS | Apache SparkDental insurance | Health insurance | Home-office allowance | Life insurance | Meal allowanceSenior-level Full TimeBrazil - Remote R1d ago
-
Airflow | CI/CD | Clean Code | Cloud platform | DataflowBike parking | Breastfeeding room | Changing Room | Childcare assistance | Dental insuranceSenior-level Full TimeSão Paulo, Brazil1d ago
-
[GenAI] Sênior Software Engineer - híbrido Florianópolis MXN 120K-120KAPI Development | AWS | Amazon SQS | Azure | CI/CDChildcare support | Dental insurance | Education benefit | Flexible hours | GympassMid-level Full TimeFlorianopolis, Santa Catarina, Brasil1d ago
-
CRM | Data Modeling | Data Visualization | ERP | Performance AnalysisChildcare support | Dental insurance | Discount on Flights | Education incentive | Extended parental leaveEntry-level Full TimeSão Paulo1d ago
-
Middle Software Engineer (Python) BRL 304K-370KAgile | Apache PySpark | CI/CD | CSV | GitHybrid work | International projects | Medical healthcare | Ongoing learning reimbursement | Recognition programSenior-level Full TimeBrazil1d ago
-
Lead Solution Architect (AI & Data Applications) BRL 325K-430KAI Agents | Angular | CI/CD | Databricks | DockerSenior-level Full TimeBlumenau, SC, Brazil1d ago
-
Data Engineer (Senior) ID53687 BRL 305K-396KAWS EMR | Airflow | Apache Spark | Data Partitioning | Data groupingEducation budget | Fitness budget | Flexible schedule | Mentorship | Office workSenior-level Full TimeCampinas, Brazil2d ago
-
AI First | Artificial Intelligence | Big Data | Consulting | Credit RiskEnglish fluency | Hybrid work | Spanish preferredMid-level Full TimeSao Paulo, BRAZIL, Brazil2d ago
-
AWS Glue | Azure Data | Azure Data Factory | Data Factory | Data GovernanceChildcare assistance | Dental insurance | Flexible schedule | Health insurance | Life insuranceEntry-level Full TimePompeia, State of São Paulo, Brazil2d ago
-
A/B | A/B Testing | B testing | Context Management | Data AnalysisFully remote | Healthcare coverage | Home office setup allowance | International company retreats | Learning and development stipendSenior-level Full TimeBrazil R2d ago
-
Agile | Azure | Azure Data | Azure Data Factory | Azure SynapseSenior-level Full TimeSão Paulo, State of São Paulo, …2d ago
-
.Net 8.0 | AI Pipelines | API Integration | ASP.Net Core | AlertingLeadership opportunity | Remote work | U.S. business hoursSenior-level Full TimeBrazil - Remote R2d ago
-
AWS Glue | Airflow | Azure Data | Azure Data Factory | BigQueryFlexible work hours | Remote workMid-level Full TimeBrazil - Remote R2d ago
-
Analista De Dados Pl BRL 108K-108KAPI Authentication | Azure | Data Factory | Data Governance | Data ModelingEnglish intermediate advanced support | Hybrid work | Technical mentorshipSenior-level Full TimeBR Sao Paulo (Alameda dos Arapanés), …2d ago