Senior Data Engineer Lead- ScotiaTech
Tasks
- Build, test, and optimize data pipelines
- Collaborate with cross-functional partners
- Conduct code and design reviews
- Deliver production-grade code
- Design data pipelines and lakehouse solutions
- Enforce data governance, security, and compliance
- Estimate effort identify risks and dependencies
- Evaluate pilot and recommend new services and tools
- Implement permanent fixes
- Lead mentor and develop data engineers
- Migrate legacy workloads to Azure Databricks
- Monitor and maintain pipeline availability and data quality
- Own end-to-end solution design
- Perform root cause analysis and incident resolution
- Produce technical documentation
Perks/Benefits
- N/A
Skills/Tech-stack
Access Control | Asset bundles | Automated testing | Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure DevOps | Azure Entra | Azure Entra ID | Azure Security | Azure Synapse | Bicep | CI/CD | Data Factory | Data Governance | Data Lake Storage Gen2 | Data Lineage | Data Vault | Databricks (asset) | Databricks Asset Bundles | Databricks Workflows | Delta Lake | Dimensional Modeling | ELT | ETL | Encryption | Entra ID | Event Hubs | FinOps | Git | GitHub Actions | Infrastructure as Code | Key Management | Lakehouse | Medallion Architecture | PySpark | Python | RBAC | Role-Based Access | Role-Based Access Control | SQL | Spark | Streaming Data | Terraform | Unity Catalog | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Bachelor of Technology
Regions
Countries
States
Related jobs
-
Data Engineer Specialist- ScotiaTech COP 54000K-74400KAccess Control | Apache Spark | Azure Databricks | Bicep | CI/CDInclusive work culture | Professional development opportunitiesMid-level Full TimeBogota, DC, CO13h ago
-
DataOps Engineer Specialist- ScotiaTech COP 54000K-74400KApache Spark | Automated testing | Azure Databricks | CI/CD | Cloud modernizationMid-level Full TimeBogota, DC, CO13h ago
-
Senior Data Engineer COP 88000K-120000KAI | Agentic Engineering | BigQuery | Cloud Storage | Cloud platformSenior-level Full TimeColombia R14h ago
-
Analytics Consultant COP 31853K-31853KApache Spark | Artificial Intelligence | Automation | Classification | ClusteringMid-level Full TimeBogotá, Colombia17h ago
-
Automated testing | Azure | Azure Databricks | CI/CD | DBTSenior-level Full TimeColombia; Costa Rica; Peru20h ago
-
AWS ECS | AWS ECS Fargate | AWS EMR | AWS S3 | AWS SNSDiversity and inclusion working groups | Flexible working practices | Parental leaveEntry-level Full TimeColombia - Remote R23h ago
-
Ingeniero de datos - Senior COP 78000K-83999KAWS Glue | AWS IAM | AWS Lambda | Airflow | Amazon AthenaSenior-level Full TimeBogotá, Bogota, Colombia1d ago
-
Data Engineer Specialist COP 54000K-74400KAccess Control | Apache Spark | Automated testing | Azure Databricks | CI/CDMid-level Full TimeBogota, DC, CO1d ago
-
Senior Data Engineer Lead COP 60000K-83999KAccess Control | Apache Spark | Asset bundles | Automated testing | Azure DataSenior-level Full TimeBogota, DC, CO1d ago
-
DataOps Engineer Specialist COP 54000K-74400KAgile | Apache Spark | Automated testing | Azure Databricks | CI/CDMid-level Full TimeBogota, DC, CO1d ago
-
Business Analytics Specialist COP 49838K-57505KAPIs | BI Integration | Batch Processing | Cloud Object Storage | DBTCatered lunches | Co-working travel perk | Health insurance | Home-office equipment | Indefinite term contractMid-level Full TimeBogotá, Bogotá D.C.1d ago
-
Senior-level Full TimeBogota, Colombia / Argentina (Remote Friendly) R3d ago
-
Conversational AI Engineer - Google/GECX & Code Agents COP 240000K-310000KBigQuery | CCaaS | CI/CD | CRM | Cloud loggingMid-level Full TimeBogota3d ago
-
Lead Data Engineer COP 89364K-102180KApache Spark | CI/CD | CTE | DBT | DBT CoreFixed-term contract | Fully remoteSenior-level Full TimeColombia R3d ago
-
Senior Applied Scientist COP 88000K-94620KAWS Bedrock | AWS EKS | AWS Lambda | AWS OpenSearch | AWS TextractFuneral assistance | Home office stipend | Internet allowance | Life insurance | Paid time offSenior-level Full TimeBogotá, Colombia R4d ago
-
Senior Applied Scientist COP 88000K-94620KAWS | AWS AgentCore | AWS Bedrock | AWS Lambda | AWS TextractFuneral assistance | Home office stipend | Internet allowance | Life insurance | MentorshipSenior-level Full TimeBogotá, Colombia R4d ago
-
Data Quality Assurance Engineer (Temporary) COP 36000K-36000KALM | Apache Airflow | Autosys | Case design | Cloud platformEntry-level Full Time TemporaryBogotá, Bogota, Colombia4d ago
-
Fairlead - Lead Data Engineer COP 113244K-129420KBigQuery | Cloud Functions | Cloud Run | Cloud Storage | Cloud platformSenior-level Full TimeBogotá, CO, 1102316d ago
-
Senior-level Full TimeBogotá, CO, 1102316d ago
-
Data Automation | Data Governance | Data Integrity | Data Quality | Data VisualizationMid-level Full TimeBogota, Colombia - Connecta 266d ago
-
Ingeniero de Datos Azure / Microsoft Fabric Semi Senior COP 70000K-83999KAzure Data | Azure Data Factory | Data Factory | Databricks | Delta LakeCareer development | Full-time employment | Remote workSenior-level Full TimeBogotá, Colombia R6d ago
-
Apache Spark | Azure Synapse | DAX | Data Lake | Data ModelingBirthday day off | Life insurance | Prepaid medicine | Work from homeMid-level Full TimeColombia, United States R7d ago
-
Analista BI & GenAI COP 66000K-78000KAccess Control | Azure | Azure Data | Azure Data Factory | Azure SynapseSenior-level Full TimeBogotá, Colombia7d ago
-
Analytics Manager COP 65000K-72000KAccess Control | Amazon Redshift | Apache Airflow | BigQuery | DBTMid-level Full TimeMedellín, Colombia7d ago
-
Senior Data Engineer COP 90840K-102180KApache Spark | CI/CD | CTE | DBT | Databricks SQLRemote workSenior-level Full TimeColombia7d ago