Tech Lead/ Senior Architect - Big Data

Jakarta, Jakarta

NorthBay Solutions

Get access to all of our AWS best practices and insights.     Subscribe Now AWS EXPERT We are a fast-growing, AWS focused onshore/offshore AWS Premier Partner NorthBay Solutions is proud to have earned the AWS competencies through...

View all jobs at NorthBay Solutions

Apply now Apply later

NorthBay Solutions, a premier US-based AWS partner with over a decade of experience serving clients globally, is seeking a passionate and experienced Tech Lead/Senior Architect to join our growing Data Engineering team.

About the Role: In this role, you will play a pivotal role in leading and architecting innovative solutions leveraging AWS Big Data technologies (e.g. S3, Redshift, EMR, Glue). You will collaborate closely with a global, multicultural team of Data Engineers and guiding project execution, ensuring technical excellence, and fostering a culture of continuous learning and supporting sales growth.

Key Qualifications & Required Knowledge:

  • 10+ years of experience in technology, with a minimum of 6 years specifically focused on Data Engineering.
  • Proven experience developing and implementing ETL pipelines on AWS that extract, transform, and load data for strategic business goals.
  • Expertise in ETL Data Engineering using Python, Java, and Big Data technologies (Hadoop, PySpark, Spark SQL, Hive).
  • Effective time management skills with a proven ability to manage and prioritize multiple project tasks.
  • Experience creating and driving large-scale ETL pipelines in an AWS environment.
  • Experience integrating data from diverse data sources.
  • Strong software development and programming skills with a focus on data using Python/PySpark or Scala.

Preferred Qualifications (Nice to have):

  • Experience with DevOps and CI/CD tools (Bitbucket, Bamboo, Ansible, Sonar, Atlassian suite).
  • In-depth knowledge of core AWS services (IAM, CloudFormation, EC2, S3, EMR/Spark, Glue, Lambda, Athena, Redshift).
  • Familiarity with popular data management tools (Data Lake, Databricks, Snowflake).
  • Understanding of core data science concepts (statistics, modelling, algorithms).
  • Experience with AWS AI/ML Services (SageMaker, Ground Truth) for building, training, and deploying machine learning models.
  • Working knowledge of AWS infrastructure services (ECS, Cognito, CloudWatch, RDS: Postgres, API Gateway, Lambda) to support data engineering workflows and integrations.
  • Acquainted with the value and evolution of Generative AI (GenAI) and its potential applications within the Big Data landscape. 
  • Basic familiarity with Amazon Q, a managed generative AI powered chat assistant, and Amazon Bedrock, a serverless service that hosts foundation models.

Technical Responsibilities:

  • Design and develop innovative solutions leveraging Big Data technologies (Hadoop, Spark, etc.) to address complex data challenges.
  • Architect and build highly scalable and distributed data systems on the AWS cloud platform.
  • Design, develop, and manage robust ETL jobs and pipelines to ensure efficient data movement and transformation.
  • Deploy production-ready data models, including data ingestion, configuration, resource management, and monitoring.

Leadership and Collaboration:

  • Partner with sales to qualify opportunities, develop compelling technical solutions, and respond to RFPs.
  • Serve as a thought leader, providing strategic guidance and mentoring junior team members as they leverage new data-driven solutions.
  • Foster a culture of continuous learning and improvement by mentoring and training colleagues.
  • Collaborate effectively to troubleshoot complex issues and provide timely guidance to team members.

Business Impact:

  • Drive impactful business improvements by analysing data, identifying trends, and recommending process optimizations. 

Additional qualities:

  • Exceptional communication and English skills.
  • Proven leadership.
  • Client-facing expertise.
  • Technological agility.
  • Thrives in a fast-paced environment.
  • Analytical & Detail-oriented.
  • Self-driven leader.
  • Team player.
Apply now Apply later

* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰

Job stats:  0  0  0

Tags: Ansible APIs Athena AWS Big Data Bitbucket CI/CD CloudFormation Databricks Data management DevOps EC2 ECS Engineering ETL Generative AI Hadoop Java Lambda Machine Learning ML models Pipelines PostgreSQL PySpark Python Redshift SageMaker Scala Snowflake Spark SQL Statistics

Perks/benefits: Career development

Region: Asia/Pacific
Country: Indonesia

More jobs like this