Technical Lead-Data Engg

Area(s) of responsibility

Sr Data Engineer:

Essential Duties and Responsibilities: 
•    This section contains a list of five to eight primary responsibilities of this role that account for 5% or more of the work. The incumbent will perform other duties assigned.
•    Development of new ETL/data transformation jobs, using PySpark or Python in AWS.
•    Enhancement and support on existing ETL/data transformation jobs.
•    Can explain technical solutions and resolutions with internal customers and communicate feedback to the ETL team.
•    Perform technical code reviews for peers moving code into production.
•    Perform and review integration testing before production migrations.
•    Provide high level of technical support and perform root cause analysis for problems experienced within area of functional responsibility.
•    Can document technical specs from business communications.

Qualifications: 
To perform this job successfully, an individual must be able to perform each essential duty satisfactorily. List knowledge, skills, and/or abilities required. Reasonable accommodations may be made to enable individuals with disabilities to perform essential functions.
•    6+ years of ETL experience.
•    Experience with core Python programming for data transformation.
•    Intermediate-level Python and PySpark skills. Can write, read, understand and debug existing code Python and ySpark code from scratch.
•    Strong knowledge of SQL fundamentals and snowflake experience, understanding of subqueries, can tune queries with execution hints to improve performance.
•    Able to write SQL code sufficient for most business requirements for pulling data from sources, applying rules to the data, and stocking target data
•    Proven track record in troubleshooting ETL jobs and addressing production issues like performance tuning, reject handling, and ad-hoc reloads.
•    Proficient in developing optimization strategies for ETL processes.
•    Basic AWS technical support skills. Has ability to log in, find existing jobs and check run status and logs
•    Will run and monitor jobs running via Control-M
•    Can create clear and concise documentation and communications.
•    Can document technical specs from business communications.