Senior Data Engineer
Tasks
- Analyze source systems
- Assess Databricks environment
- Create architecture delivery documentation
- Define Medallion Architecture layers
- Define naming conventions
- Define target data architecture
- Design and validate SQL transformation logic
- Design data flow and ingestion patterns
- Document data structures and dependencies
- Document metadata lineage and usage expectations
- Document reuse refactoring recommendations
- Establish data modeling standards
- Liaise between Architecture Data Engineering Analytics and Platform teams
- Maintain runbooks and architectural artifacts
- Prototype with SQL or PySpark
- Support data ownership and domain concepts
- Support governance standards with Unity Catalog
- Support security and metadata documentation
- Validate data modeling approaches
Perks/Benefits
Skills/Tech-stack
Auto Loader | Azure | Data Governance | Data Lineage | Data Modeling | Databricks | Delta Lake | Delta Live Tables | Delta Live) | ELT | ETL | Lakehouse | Medallion Architecture | Metadata Management | PySpark | Python | SQL | Unity Catalog
Education
Bachelor of Engineering | Bachelor of Engineering in Computer Science | Bachelor of Engineering in Information Systems | Bachelor of Science | Bachelor of Science in Computer Science | Bachelor of Science in Information Systems
Roles
Regions
Countries
States
Related jobs
-
Data Consultant COP 27600K-240000KAgile | Alation | Apache Spark | Azure Blob | Azure Blob StorageCareer coaching | Employee assistance programme | Flexible working hours | Hybrid work | Learning workshops conferencesSenior-level Full TimeMedellín, Antioquia, Colombia15h ago
-
AWS | AWS CloudFormation | AWS DMS | AWS Glue | AWS LambdaEducation budget | Fitness budget | Flexible schedule | Growth roadmaps | MentorshipSenior-level Full TimeArmenia, Colombia2d ago
-
AWS | AWS CloudFormation | AWS DMS | AWS Glue | AWS LambdaEducation budget | Fitness budget | Flexible schedule | Growth roadmaps | MentorshipSenior-level Full TimePalmira, Colombia2d ago
-
AWS | AWS CloudFormation | AWS DMS | AWS Glue | AWS LambdaEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsSenior-level Full TimeValledupar, Colombia2d ago
-
AWS | AWS CloudFormation | AWS DMS | AWS Glue | AWS LambdaEducation budget | Fitness budget | Flextime | Growth roadmaps | MentorshipSenior-level Full TimeTunja, Colombia2d ago
-
AWS | AWS CloudFormation | AWS DMS | AWS Glue | Access ControlEducation budget | Fitness budget | Flextime | Mentorship | Office work optionsSenior-level Full TimeEnvigado, Colombia2d ago
-
AWS | AWS DMS | AWS Glue | AWS Lambda | AWS Step FunctionsEducation budget | Fitness budget | Flexible schedule | Mentorship | Office work optionsSenior-level Full TimeBucaramanga, Colombia2d ago
-
AWS | AWS CloudFormation | AWS DMS | AWS Glue | AWS LambdaEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsSenior-level Full TimeBogota, Colombia2d ago
-
AWS | AWS CloudFormation | AWS DMS | AWS Glue | AWS LambdaEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsSenior-level Full TimeBello, Colombia2d ago
-
AWS | AWS CloudFormation | AWS DMS | AWS Glue | AWS LambdaEducation budget | Fitness budget | Flextime | Growth roadmaps | MentorshipSenior-level Full TimeIbague, Colombia2d ago
-
AWS | AWS CloudFormation | AWS DMS | AWS Glue | AWS LambdaEducation budget | Fitness budget | Flexible schedule | Growth roadmaps | Professional mentorshipSenior-level Full TimePereira, Colombia2d ago
-
AWS | AWS DMS | Airflow | Athena | CloudFormationEducation budget | Fitness budget | Flextime | Growth roadmaps | MentorshipSenior-level Full TimeVillavicencio, Colombia2d ago
-
AWS | AWS DMS | Airflow | Athena | CloudFormationEducation budget | Fitness budget | Flextime | Mentorship | Office optionsSenior-level Full TimeCartagena, Colombia2d ago
-
AWS | AWS CloudFormation | AWS DMS | AWS Glue | AWS LambdaEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsSenior-level Full TimeManizales, Colombia2d ago
-
AWS | AWS CloudFormation | AWS DMS | AWS Glue | AWS LambdaEducation budget | Fitness budget | Flexible schedule | Mentorship | Office work optionSenior-level Full TimeMedellín, Colombia2d ago
-
AWS CloudFormation | AWS DMS | AWS Glue | AWS Lambda | AWS S3Education budget | Fitness budget | Flexible work schedule | Mentorship | Personalized growth roadmapsSenior-level Full TimePasto, Colombia2d ago
-
AWS | AWS DMS | Airflow | Athena | CloudFormationEducation budget | Fitness budget | Flextime | Growth roadmaps | MentorshipSenior-level Full TimeSanta Marta, Colombia2d ago
-
AWS | AWS CloudFormation | AWS Database | AWS Database Migration Service | AWS GlueEducation budget | Fitness budget | Flextime | Mentorship | Personalized growth roadmapsSenior-level Full TimeItagüí, Colombia2d ago
-
AWS CloudFormation | AWS DMS | AWS Glue | AWS Lambda | AWS Step FunctionsEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsSenior-level Full TimeCucuta, Colombia2d ago
-
AWS | AWS CloudFormation | AWS Database | AWS Database Migration Service | AWS GlueEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsSenior-level Full TimeSabaneta, Colombia2d ago
-
AWS CloudFormation | AWS DMS | AWS Glue | AWS Lambda | AWS S3Education budget | Fitness budget | Flextime | Mentorship | Office optionsSenior-level Full TimePopayan, Colombia2d ago
-
AWS | AWS CloudFormation | AWS DMS | AWS Glue | AWS LambdaEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsSenior-level Full TimeBarranquilla, Colombia2d ago
-
AWS DMS | Amazon Web Services | Apache Airflow | Apache Kafka | Apache SparkEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsSenior-level Full TimeArmenia, Colombia2d ago
-
AWS | AWS CloudFormation | AWS DMS | AWS Glue | AWS LambdaEducation budget | Fitness budget | Flexible schedule | Mentorship | Office work optionsSenior-level Full TimeSoledad, Colombia2d ago
-
AWS | AWS CloudFormation | AWS DMS | AWS Glue | AWS LambdaEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsSenior-level Full TimeChia, Colombia2d ago