Senior Data Engineer

Company Overview:Lean Tech is a rapidly expanding organization situated in Medellín, Colombia. We pride ourselves on possessing one of the most influential networks within software development and IT services for the entertainment, financial, and logistics sectors. Our corporate projections offer many opportunities for professionals to elevate their careers and experience substantial growth. Joining our team means engaging with expansive engineering teams across Latin America and the United States, contributing to cutting-edge developments in multiple industries.We are seeking a proactive Senior Data Engineer to design and implement scalable, fault-tolerant data pipelines and platforms, collaborate with developers, data scientists, and product managers to deliver data-driven insights, leveraging Python, Java, DBT, and tools like Hadoop, Spark, and Google Cloud.

 

Position Title: Senior Data Engineer

 

Location: Remote - LATAM

 

What you will be doing:As a Data Engineer, you will be instrumental in designing and implementing robust data pipelines and platforms that support our customer-facing products. As a key contributor, you will collaborate closely with developers, data scientists, and product managers to address vital questions and deliver data-driven insights. You will be responsible for developing highly available, scalable, and fault-tolerant systems capable of batch and real-time data analysis. You'll leverage advanced programming skills in Python and Java, alongside expertise in tools such as DBT, to create essential reporting metrics. Within our team, you will play an essential role in exploring new data technologies and advising on industry best practices, ensuring that our data infrastructure remains at the cutting edge. The unique challenges of this position include the integration of diverse data processing technologies like Hadoop, Spark, Dataflow, and maintaining an effective workflow on platforms such as Google Cloud, Snowflake, and BigQuery. Key responsibilities include:

• Build and maintain robust data pipelines and platforms to support customer-facing products, ensuring seamless data flow and accessibility.
• Collaborate effectively with developers, data scientists, and product managers to interpret data-related queries and provide strategic solutions.
• Develop highly available, scalable, and fault-tolerant systems for both batch and real-time data analysis utilizing advanced programming skills in Python or Java.
• Employ DBT to create comprehensive reporting metrics, enhancing data visibility and decision-making processes.
• Ensure efficient real-time streaming data ingestion and processing using tools such as Kafka or Pub/Sub, tailored to project needs.
• Apply intermediate skills in ETL/ELT processes with tools like Airflow to optimize data integration and processing workflows.
• Engage in exploring innovative data technologies, providing informed advice on best practices to enhance departmental capabilities.

Requirements & QualificationsTo excel in this role, you should possess:

• Advanced programming proficiency in Python or Java.
• 4+ years of experience in building data lakes within cloud environments.
• Extensive experience with DBT for developing data warehouse metrics.
• Advanced skills in building real-time streaming data ingestion and processing pipelines using Kafka or Pub/Sub.
• Proven ability to construct scalable ETL/ELT pipelines, with familiarity in tools such as Airflow.
• Demonstrated proficiency in writing complex SQL statements.
• Solid experience with data processing tools like Hadoop, Spark, and Dataflow.
• Practical experience utilizing Google Cloud and Google BigQuery.
• Experience working with Snowflake data warehousing solutions.
  

Nice to have skills

• Experience with data modeling and data architecture design.
• Knowledge of containerization tools such as Docker or Kubernetes.
• Familiarity with DevOps practices and CI/CD pipelines.
• Experience with other cloud platforms like AWS or Azure.
• Ability to use visualization tools such as Tableau or Power BI.
• Certification in Google Cloud Platform (GCP) or other relevant cloud certifications.
• Strong communication and teamwork skills.

 

Why you will love Lean Tech:

• Join a powerful tech workforce and help us change the world through technology
• Professional development opportunities with international customers
• Collaborative work environment
• Career paths and mentorship programs that will lead to new levels. 

Join Lean Tech and contribute to shaping the data landscape within a dynamic and growing organization. Your skills will be honed, and your contributions will be vital to our continued success. Lean Tech is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.