Data Engineer

We are seeking a highly skilled Data Engineer with 4-5 years of experience to join our team. The ideal candidate will have extensive experience with AWS services such as EC2, S3, EKS, and Redshift and a strong understanding of open-source tools like Airbyte, Dagster, Superset, and n8n. The role involves building robust data pipelines, supporting product development, and contributing to the growth of the business.

Responsibilities

1. Data Engineering and AWS Expertise:

• Design, build, and maintain scalable and reliable data pipelines and ETL processes.

• Leverage AWS services (EC2, S3, EKS, Redshift) to implement and optimize data solutions.

• Ensure high availability, performance, and scalability of data systems.

• Monitor and troubleshoot data pipelines, ensuring minimal downtime.

2. Open-Source Tools Integration:

• Implement and manage workflows using tools like Airbyte for data extraction, Dagster for orchestration, and Superset for visualization.

• Automate workflows and integrations using n8n or similar tools (e.g., Zapier or Make.com).

3. Product Building & Business Growth:

• Collaborate with product managers and business stakeholders to understand data needs and align with business objectives.

• Support the development of data-driven features and functionalities to enhance product offerings.

• Analyze data to derive insights that drive growth strategies and decision-making.

4. Data Governance & Optimization:

• Ensure data security, privacy, and compliance with relevant regulations.

• Optimize query performance in Redshift and other data storage systems.

• Implement best practices for version control, documentation, and process automation.

Requirements

Must-Have Skills:

• 4-5 years of hands-on experience in data engineering or related roles.

• Proficiency with AWS services: EC2, S3, EKS, and Redshift.

• Strong expertise in open-source tools like Airbyte, Dagster, Superset, and n8n.

• Solid understanding of database management, data modeling, and query optimization.

• Experience building and maintaining scalable data pipelines and ETL workflows.

Good-to-have Skills:

• Familiarity with additional AWS services like Lambda, Glue, or Athena.

• Knowledge of Python, SQL, and infrastructure-as-code tools like Terraform or CloudFormation.

• Experience with API integrations and webhook setups.

• Understanding of CI/CD pipelines and version control systems (e.g., Git).