Data Engineer Sr Software Engineer

About the Role

As a Senior Data Engineer, you will work closely with engineering, analytics, data science staff across various Gap Development Center to deliver the goals and objectives of the business.

What You'll Do

• Partner with Product Managers, and Solution Architects to define and analyze customer’s technical requirements
• Provides technical input to Statement of Work for External Service Providers
• Ability to break down product management stories into detailed design specifications for medium to complex business problems
• Experience with building data pipeline from several business applications
• Ability to collaborate with business stakeholders to identify and meet data requirements.
• Guides team members on the use of design patterns and adherence to code quality metrics
• Uses software code management tools and processes to ensure code traceability and good governance.
• Conducts code reviews with other team members (either internal or external service providers)
• Makes continuous improvement to our coding and test practices across applications
• Works in DevOps team which uses continuous development, integration and testing with automated releases to lower and higher environments with application monitoring plus supporting the product
• Experience performing root cause analysis on all data and processes to answer specific questions and identify opportunities for improvement.
• Participates in recommending emerging technologies and application solutions

Who You Are

• Candidate with 5+ years of experience in a Data Engineer role
• Graduate degree in Computer Science or equivalent
• High Problem-Solving Skills with analytical thinking and logical reasoning
• Having experience working on spark platform, spark sql, Python and common python libraries.
• Working experience on cloud-based environments such as Databricks on Azure. ADF, PySpark etc.
• Hands on experience writing notebooks using Python/Spark
• Well versed with relational databases, nonrelational databases, data streams, and file stores
• Experience with source control systems such as GitHub and Jenkins build and continuous integration tools.
• Automation using Linux shell & Python scripting.

Development experience with Enterprise integration patterns