Data Engineer

Job Title: Data Engineer 

Location: Karachi, Lahore , Islamabad (Hybrid)
Experience: 5+ Years
Job Type: Full-Time
 

Job Overview:

We are looking for a highly skilled and experienced Data Engineer with a strong foundation in Big Data, distributed computing, and cloud-based data solutions. This role demands a strong understanding of end-to-end Data pipelines, data modeling, and advanced data engineering practices across diverse data sources and environments. You will play a pivotal role in building, deploying, and optimizing data infrastructure and pipelines in a scalable cloud-based architecture.

Key Responsibilities:

• Design, develop, and maintain large-scale Data pipelines using modern big data technologies and cloud-native tools.

• Build scalable and efficient distributed data processing systems using Hadoop, Spark, Hive, and Kafka.

• Work extensively with cloud platforms (preferably AWS) and services like EMR, Glue, Lambda, Athena, S3.

• Design and implement data integration solutions pulling from multiple sources into a centralized data warehouse or data lake.

• Develop pipelines using DBT (Data Build Tool) and manage workflows with Apache Airflow or Step Functions.

• Write clean, maintainable, and efficient code using Python, PySpark, or Scala for data transformation and processing.

• Build and manage relational and columnar data stores such as PostgreSQL, MySQL, Redshift, Snowflake, HBase, ClickHouse.

• Implement CI/CD pipelines using Docker, Jenkins, and other DevOps tools.

• Collaborate with data scientists, analysts, and other engineering teams to deploy data models into production.

• Drive data quality, integrity, and consistency across systems.

• Participate in Agile/Scrum ceremonies and utilize JIRA for task management.

• Provide mentorship and technical guidance to junior team members.

• Contribute to continuous improvement by making recommendations to enhance data engineering processes and architecture.

Required Skills & Experience:

• 5+ years of hands-on experience as a Data Engineer 

• Deep knowledge of Big Data technologies – Hadoop, Spark, Hive, Kafka.

• Expertise in Python, PySpark and/or Scala.

• Proficient with data modeling, SQL scripting, and working with large-scale datasets.

• Experience with distributed storage like HDFS and cloud storage (e.g., AWS S3).

• Hands-on with data orchestration tools like Apache Airflow or StepFunction.

• Experience working in AWS environments with services such as EMR, Glue, Lambda, Athena.

• Familiarity with data warehousing concepts and experience with tools like Redshift, Snowflake (preferred).

• Exposure to tools like Informatica, AbInitio, Apache Iceberg is a plus.

• Knowledge of Docker, Jenkins, and other CI/CD tools.

• Strong problem-solving skills, initiative, and a continuous learning mindset.

Preferred Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.

• Experience with open table formats such as Apache Iceberg.

• Hands-on with AbInitio (GDE, Collect > IT) or Informatica tools.

• Knowledge of Agile methodology, working experience in JIRA.

Soft Skills:

• Self-driven, proactive, and a strong team player.

• Excellent communication and interpersonal skills.

• Passion for data and technology innovation.

• Ability to work independently and manage multiple priorities in a fast-paced environment.