Senior Data Engineer

Company OverviewLean Tech is committed to driving innovation and excellence through our comprehensive data solutions and advanced technology platforms. Our mission is to empower informed decision-making by delivering robust and efficient data engineering services. With a firm foundation in integrity and collaboration, we uphold a culture that values continuous learning and adaptability. Our unique selling point lies in harnessing cutting-edge cloud services, such as AWS and PostgreSQL, enabling us to consistently achieve high-quality data infrastructure. As we operate at the forefront of the tech industry, we maintain a strong market position through our global footprint and resilient network. Headquartered in a strategic location, our expanding team of skilled professionals collaborates to ensure streamlined data integration and management.Position OverviewThis role offers an exciting opportunity for a dedicated individual to take on a senior position within our dynamic data team. The primary purpose of this role is to lead the development and management of our data infrastructure and processes. As a Senior Data Engineer, you will be instrumental in designing and maintaining sophisticated ETL workflows using Apache Airflow and ensuring data quality through robust data assurance frameworks. Your work will involve collaborating with data scientists and software engineers to meet comprehensive data requirements. With a focus on AWS cloud services, including EC2, Lambda, and AWS Glue, you will integrate and manage data across platforms, supporting data-driven decisions. Your mastery in PostgreSQL will be pivotal in executing ETL processes and managing database structures. By contributing to documentation and version-controlled changes using Liquibase, you will help standardize our data infrastructure. This role presents unique challenges, including leading Apache Airflow orchestration and leveraging scripting skills in Python or Bash for complex ETL functions. As part of Lean Tech, your impact will resonate across the organization, empowering all teams with accurate and accessible data.Key Responsibilities

• Lead the design, creation, and maintenance of complex relational database structures using PostgreSQL, ensuring efficient ETL processes.
• Manage and optimize ETL/ELT workflows with Apache Airflow, providing leadership in orchestration tasks and ensuring high reliability.
• Oversee the integration and management of data across systems utilizing AWS services such as EC2, Lambda, S3, and AWS Glue, ensuring robust cloud infrastructure.
• Implement and manage version-controlled database changes using Liquibase, maintaining schema integrity and stability.
• Collaborate with data scientists, analysts, and software engineers to understand advanced data requirements and assure high data quality standards.
• Develop and enforce data quality assurance frameworks and methodologies, enhancing data integrity across projects.
• Document and establish standard operating procedures for the data infrastructure, ensuring consistency and accuracy.

 

Required Skills & Experience

• Advanced proficiency in using PostgreSQL for ETL processes, including table creation, indexing, joins, and SELECT queries.
• Leadership experience in managing ETL orchestration tools such as Apache Airflow.
• Intermediate knowledge of core AWS services including EC2, Lambda, and S3.
• Advanced skills in data integration with AWS Glue.
• Beginner-level experience with Liquibase for version-controlled database schema changes.
• Advanced capability in general programming or scripting using Python or Bash for ETL and orchestration tasks.
• Skilled in using collaboration tools, specifically GitHub.
• Experience in implementing data quality assurance frameworks or methodologies.
• Strong problem-solving skills and a willingness to learn new technologies.
• Proficiency in English at a C1 Advanced level.

 

Nice to Have Skills

• Experience with data visualization tools such as Tableau or Power BI
• Understanding of big data technologies, including Hadoop or Spark
• Familiarity with DevOps practices and CI/CD pipelines
• Knowledge of data privacy laws and regulations
• Certified AWS Solutions Architect or other cloud certifications
• Advanced - C1 English level 

 

Soft Skills

• Leadership: Demonstrated ability to manage ETL orchestration tools such as Apache Airflow, guiding the team in implementing efficient workflows.
• Collaboration: Effective teamwork with data scientists, analysts, and software engineers, ensuring alignment on data requirements and quality standards, while leveraging tools like GitHub for version control.
• Problem-solving: Strong analytical skills to tackle complex data challenges, fostering a culture of continuous learning and adaptation to new technologies.
• Communication: Proficient in English at C1 level, capable of articulating complex data processes and strategies to both technical and non-technical stakeholders.
• Quality Assurance: Experience in implementing data quality assurance frameworks to maintain high data integrity and trustworthiness across systems.

 

Why You Will Love Working with UsJoin a powerful tech workforce and help us change the world through technology Professional development opportunities with international customers Collaborative work environment Career path and mentorship programs that will lead to new levels. Join Lean Tech and contribute to shaping the data landscape within a dynamic and growing organization. Your skills will be honed, and your contributions will play a vital role in our continued success. Lean Tech is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.