Lead Spark Data Engineer
Tasks
- Build Query Validator
- Design scalable data solutions on Spark and Flink
- Develop Spark Adapter for metrics translation
- Enforce data governance, security, and quality practices
- Ensure parity between Spark and Flink implementations
- Implement IoT Query Language grammar
- Implement relationships logic using Graph/Ontology
- Maintain documentation
- Manage Azure and Databricks resources
- Monitor and optimize workloads
- Participate in agile activities
- Transform and prepare data with SQL, Python, Java
Perks/Benefits
- N/A
Skills/Tech-stack
ANTLR | Agile methodologies | Apache Flink | Apache Spark | Artifact management | Azure DevOps | Azure ecosystem | CI/CD | Data Governance | Data Modeling | Data Quality | Data Security | Data Warehousing | Databricks | Delta Lake | GitHub | Graph/Ontology | Java | Python | SDLC | SQL
Education
Roles
Regions
Countries
States
Related jobs
-
AWS CDK | AWS CloudFormation | AWS Glue | AWS Lambda | AWS Step FunctionsPaid time off | Remote work | Work autonomySenior-level Full TimeBogota R19h ago
-
Airflow | Apache Hadoop | Apache Hive | Apache Spark | Data ModelingPaid time off | Remote work | Work autonomySenior-level Full TimeBogota R1d ago
-
APIs | Azure | Azure Functions | Azure Redis | Azure Redis CacheRemote workSenior-level Full TimeRemote but local to Bogotá, Colombia R1d ago
-
AWS Glue | AWS QuickSight | Agile | Amazon Athena | Amazon S3Paid time off | Remote work | Work autonomy | Work-life balanceSenior-level Full TimeBogota R2d ago
-
Mid-level Full TimeCOL - Remote - TaskVerse Delivery, … R3d ago
-
APIs | Automation tools | Business Intelligence | Data Architecture | Data PipelinesExtra leave days | Health benefits | Innovation culture | Life insurance | Permanent contractSenior-level Full TimeColombia R3d ago
-
AI Engineer I (Remote) USD 30K-39KAutomation platforms | CI/CD | Cloud AWS | Data Science | Distributed SystemsProfessional growth opportunities | Remote workEntry-level Full TimeBogota, Colombia R4d ago
-
AWS | Azure | CI/CD | Distributed Systems | GCPHealth reimbursement | Holidays | Home office reimbursement | Learning Bonus | Lifestyle reimbursementSenior-level Full TimeColombia - Remote R5d ago
-
AI workflows | API Integration | Azure | Cloud Platforms | Cloud Platforms (AWSCollaboration with experienced engineers | Exposure to modern AI practices | Work on production AI systemsSenior-level Full TimeBogota, Colombia (Remote Friendly) R8d ago
-
AI Solutions Engineer USD 59KAI | API Design | Cloud infrastructure | Data Engineering | DevOpsCommunity engagement | Egoless collaboration | Impactful work | Mutual trust | Professional growthSenior-level Full TimeRemote (Colombia) R8d ago
-
AWS | Airflow | Bash | CloudWatch | DBTFlexible schedules | Professional development opportunities | Remote or hybrid workSenior-level Full TimeMedellin, CO, 050021 R9d ago
-
AWS Lambda | AWS SageMaker | CI/CD | CNN | Deep learningAWS certification sponsorship | Learning Support | Medical insurance | Paid sick leave | Paid vacationSenior-level Full TimeMedellín, Antioquia R9d ago
-
AWS | CI/CD | Code Quality | Data Engineering | Deep learningFully remote setup | Medical insurance | Paid sick leave | Paid vacation | Public holidaysSenior-level Full TimeMedellín, Antioquia R9d ago
-
AWS | DBT | DMS | Data Lake | Data ModelingFlexible work hours | Remote work supportSenior-level Full TimeColombia, Remote R10d ago
-
Automation | Data Modeling | Data Pipelines | Data Warehousing | Database securityCompetitive salary USD | Flexible schedule | Paid time off | Remote work | Work with leading companiesSenior-level Full TimeBogota R11d ago
-
AI Voice Technologies | AI voice | Cloud services | Deep learning | DockerContinuing education | Flexible hours | Health coverage | Paid time off | Remote workSenior-level Full TimeMedellin, Antioquia, Colombia R13d ago
-
AI Voice Technologies | AI voice | AWS | Azure | Cloud ComputingContinuing education | Flexible hours | Health coverage | Paid time off | Remote workSenior-level Full TimeBarranquilla, Atlantico, Colombia R13d ago
-
AWS | Azure | Deep learning | Docker | Google CloudFlexible hours | Health coverage | Paid time off | Remote work | Technology stipendSenior-level Full TimePalmira, Valle del Cauca, Colombia R13d ago
-
AI Voice Technologies | AI voice | Cloud services | Deep learning | DockerHealth coverage | Paid time off | Performance bonuses | Remote work | StipendSenior-level Full TimeCartagena, Bolivar, Colombia R13d ago
-
AWS | Azure | Deep learning | Docker | Google CloudBonus opportunities | Continuing education | Flexible hours | Health coverage | Paid time offSenior-level Full TimeColombia R13d ago
-
AWS | Azure | Deep learning | Docker | Google CloudCompany-sponsored coursework | Conferences | Continuing education | Flexible hours | Health coverageSenior-level Full TimeCali, Valle del Cauca, Colombia R13d ago
-
AWS | AWS CodePipeline | CI/CD | Docker | GitLab CICertifications support | Continuing education | Flexible hours | Remote work | Training opportunitiesMid-level Full TimeCartagena, Bolivar, Colombia R13d ago
-
AWS | AWS CodePipeline | CI/CD | Docker | GitLab CICertifications | Continuing education | Flexible hours | Remote work | TrainingMid-level Full TimeBogota, D.C., Capital District, Colombia R13d ago
-
AWS | AWS CodePipeline | CI/CD | Docker | GitLab CIContinuing education | Flexible working hours | Paid time off | Performance bonuses | Work remotelyMid-level Full TimeColombia R13d ago
-
AWS | AWS CodePipeline | CI/CD | Docker | GitLab CIContinuing education | Flexible hours | Paid time off | Remote work | Training and certificationsMid-level Full TimeMedellin, Antioquia, Colombia R13d ago