Lead Spark Data Engineer
Brazil - Remote
R
BRL 304K-380K (estimate) Senior-level Contract Found 12d ago
Tasks
- Architect data solutions on Spark and Flink
- Build Spark Adapter
- Develop Query Validator
- Document solutions and workflows
- Enforce data governance, security, and quality standards
- Ensure parity between Spark and Flink implementations
- Implement IoT Query Language grammar
- Implement relationships logic to optimize graph traversal
- Monitor and tune workloads for performance
- Optimize Azure and Databricks resources
- Participate in agile activities
- Transform and clean data using SQL, Python, Java
Perks/Benefits
- N/A
Skills/Tech-stack
ANTLR v4 | ASTs | Apache Spark | Azure DevOps | Azure ecosystem | CI/CD | DSLS | Data Governance | Data Modeling | Data Quality | Data Security | Data Warehousing | Databricks | Delta Lake | GitHub | Graph/Ontology | Java | Python | SQL
Education
Roles
Language: en |
Views: 1 |
Clicks: 0
Related jobs
-
AI Training | Big Data | Data Manipulation | Data Processing | Generative AIAdvanced AI projects | Flexible schedule | Freelance | Part-time | Portfolio buildingMid-level FreelanceBrazil - Remote R5d ago
-
AI | Computational Algorithms | Machine Learning | NumPy | Numerical MethodsImpact in AI development | Part-time flexibility | Project experience | Remote workMid-level FreelanceBrazil - Remote R5d ago
-
Senior Machine Learning Engineer - Refer BRL 271K-370KAWS | Azure | CI/CD | Cloud services | DatabricksCareer development support | Remote workSenior-level Full TimeBrazil, Brazil R5d ago
-
Senior Infrastructure Engineer (AWS) - Technology BRL 234K-316KAWS | Bash | CDK | CI/CD | CloudFormationCompetitive USD pay | High-impact projects | Paid time off | Remote work | Work autonomySenior-level Full TimeSão Paulo R8d ago
-
Azure | Business Intelligence | Cloud Storage | DBT Cloud | Data AnalysisCompetitive USD pay | High-impact projects | Paid time off | Remote work | Work autonomySenior-level Full TimeBrazil R9d ago
-
Senior-level ContractBrazil - Remote R9d ago
-
Senior-level Full TimeSao Paulo, Brazil R9d ago
-
AI ethics | AI frameworks | Benchmarking | Cloud deployment | Content workflowsCross-functional collaboration | Leadership role | Long-term contract | Remote workSenior-level Full TimeBrazil - Remote R10d ago
-
APIs | AWS | Agile | Airflow | CI/CDAutonomy | Competitive pay | High-impact projects | Paid time off | Remote workSenior-level Full TimeBrazil R10d ago
-
AI machine learning | Agile | Airflow | Automation | CDKAutonomy | Competitive USD pay | Diverse global network | Paid time off | Remote workSenior-level Full TimeSão Paulo R11d ago
-
AWS | Cloud Data | Cloud Data Services | Data Governance | Data ModelingRemote work | Work-life balanceSenior-level Full TimeBRAZIL, Brazil R12d ago
-
Lead Flink Data Engineer IoT BRL 304K-370KAPI Development | Agile methodology | Apache Flink | Azure Data | Azure Data ServicesSenior-level ContractBrazil - Remote R12d ago