Lead Data Engineer ID71008
Manizales, Colombia
COP 89364K-94620K (estimate) Senior-level Full Time
Tasks
- Configure pipeline structure dependencies and triggering strategy
- Define partitioning strategy file formats schema strategy
- Design DAG based batch workflows in Airflow
- Design and own ETL pipelines
- Drive AWS data lake and analytics architecture
- Extract transform validate data from databases and APIs
- Implement near real time processing for OLAP workloads
- Lead engineering team and ensure alignment with engineering practices
- Partner with DevOps to clarify requirements
- Review pull requests enforce code quality standards
Perks/Benefits
- Annual learning budget
- Annual performance reviews
- Flexible hours
- Internal tech talks
- Mentorship
- People focused support
- Remote work
- Well-being programs
Skills/Tech-stack
Airflow | Apache Spark | Apache Spark SQL | Athena | BigQuery | ClickHouse | DAG Workflow | DAG Workflow Orchestration | Data Lake | Docker | EKS | ETL | FastAPI | Flask | GraphQL | Kubernetes | OLAP | ORC | Object storage | Parquet | Partitioning | PostgreSQL | Presto | Python | REST | S3 | Schema Design | Serverless Query | Snowflake | Spark SQL | Trino | Workflow Orchestration
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Regions
Countries
Related jobs
-
Lead Data Engineer ID71008 COP 89364K-94620KAirflow | Amazon S3 | Athena | BigQuery | ClickHouseFlexible hours | Learning budget | Mentorship | People focused support | Remote workSenior-level Full TimeCali, Colombia6h ago
-
Lead Data Engineer ID71008 COP 89364K-94620KAirflow | Amazon S3 | Athena | BigQuery | ClickHouseFlexible hours | Learning budget | Mentorship | Remote work | TechtalksSenior-level Full TimeCartagena, Colombia6h ago
-
Lead Data Engineer ID71008 COP 89364K-94620KAirflow | Athena | BigQuery | ClickHouse | DockerAnnual learning budget | Flexible hours | Internal TechTalks | Mentorship | People focused supportSenior-level Full TimePereira, Colombia6h ago
-
Lead Data Engineer ID71008 COP 89364K-94620KAWS EKS | Amazon S3 | Apache Airflow | Athena | BigQueryAnnual performance reviews | Flexible hours | Learning budget | Mentorship | People focused supportSenior-level Full TimeBucaramanga, Colombia6h ago
-
Lead Data Engineer ID71008 COP 89364K-94620KAWS | Amazon S3 | Apache Airflow | Athena | BigQueryAnnual learning budget | Flexible hours | Mentorship | No micromanagement | People focused supportSenior-level Full TimeVillavicencio, Colombia6h ago
-
Lead Data Engineer ID71008 COP 89364K-94620KAirflow | Amazon EKS | Amazon S3 | Apache Parquet | AthenaFlexible hours | Learning budget | Mentorship | People focused support | Performance reviewsSenior-level Full TimeMedellín, Colombia6h ago
-
Arquitecto de Datos COP 70000K-83999KAzure Databricks | Azure DevOps | Azure SQL | Delta Lake | ETLSenior-level Full TimeMedellin, Colombia7h ago
-
AI Analytics Engineer COP 89364K-94620KAI orchestration | Airflow | Anomaly Detection | BigQuery | DBTEquity | Health benefitsSenior-level Full TimeColombia R22h ago
-
Senior-level Full TimeBarranquilla, //ATLANTICO, Colombia1d ago
-
Data Engineer (CRM) USD 85K-97KApache Airflow | Cloud platform | DBT | Data Governance | Data ModelingAgile team collaboration | English communication environment | Remote workMid-level Full TimeBrazil / Colombia / Argentina / … R1d ago
-
Data Engineering Senior Expert I - Data Hub Riesgos - Ai Transformation & Engineering Colombia COP 27600K-240000KAWS | Agile | Azure | Big Data | CI/CDSenior-level Full TimeSING - DIR GENERAL PISO 6, …1d ago
-
Data QA Engineer – Enterprise Data | Colombia COP 60000K-72000KApache JMeter | Database Indexing | ELT | ETL | Execution PlansPaid time off | Remote work | Work with autonomy | Work with top companiesMid-level Full TimeBogota R1d ago
-
Data Engineer Manager COP 58800K-58800KAI | AWS | AWS Glue | AWS Lambda | Agent-basedCompany-provided equipment | Conference attendance | English lessons | Flexible working options | Learning opportunitiesSenior-level Full TimeBogotá, Bogota, Colombia2d ago
-
Senior Data Engineer COP 88000K-94620KAWS Glue | AWS Lambda | AWS S3 | Data Classification | Data GovernanceCareer development plans | Company-provided equipment | English lessons | Flexible working options | Learning opportunitiesSenior-level Full TimeBogotá, Bogota, Colombia2d ago
-
DevOps Engineer COP 89364K-98400KAWS | Ansible | Automation | Bash | CI/CDAgile work environment | Remote workSenior-level Full TimeColombia R2d ago
-
Mid-level Full TimeBogota, CO2d ago
-
Data Visualization Engineer SAP AC/BO Sr COP 70000K-90000KAnalytics | Analytics Cloud | Analytics designer | Augmented analytics | Business IntelligenceHybrid work | Remote workMid-level Full TimeColombia R3d ago
-
Senior Data Engineer + AI USD 119K-167KAI Engineering | API Development | AWS | Apache Spark | CI/CDRemoteSenior-level Full TimeBolivia; Colombia; Costa Rica; Peru3d ago
-
Data Engineering Associate COP 54000K-74400KAvro | CSV | Cloud Migration | Data Ingestion | Data ModelingCompetitive benefits | Inclusive work environmentMid-level Full TimeBogota, DC, CO3d ago
-
Data Engineer (Ingeniero de Datos) COP 27600K-184800KAPI Integration | Apache Spark | Azure Data | Azure Data Factory | Azure Data LakeBirthday day off | Connectivity allowance | Employee fund | Flexible schedule | Hybrid workSenior-level Full TimeBogotá, Bogota, Colombia3d ago
-
Senior Analytics Engineer COP 113244K-125000KCause analysis | DBT | Data Modeling | Data Monitoring | Data QualityMentorship | Opportunity for growth and impact | Remote work during US business hours | Travel for team/offsitesSenior-level Full TimeBogotá, Colombia R3d ago
-
Cloud Infrastructure & Data Platform Engineer COP 78000K-90000KAI Agents | AWS | Apache Kafka | Apache Spark | Autonomous DatabaseFlexible medical insurance | Life insurance | Retirement options | Volunteer programsMid-level Full TimeBOGOTA, DISTRITO CAPITAL, Colombia3d ago
-
Machine Learning Engineer COP 18000K-18000KBigQuery | Data Pipelines | HuggingFace Transformers | LLM-Ops | Machine LearningHybrid workMid-level Full TimeBogotá,Colombia R4d ago
-
Senior AI Engineer COP 95420K-101090KAWS | Amazon Textract | Confidence scoring | Data Validation | Document AISenior-level Full TimeColombia R4d ago
-
AI SDR | Agile | CRM automation | Calendly API | JavaFlexible time zone alignment | Remote workSenior-level Full TimeBrazil / Argentina / Peru / …4d ago