Senior Data Engineer (Remote)

Job Summary

As a Senior Data Engineer, L3 for the Information Technology organization, you’ll be responsible for acquiring, curating, and publishing data for analytical or operational uses. You will prepare data for use by data scientists, business users, and technology platforms by creating a single version of the truth for all data consumers. You will work with streaming and batch-loading data sources from technical systems and solutions across the enterprise.  Successful data engineers have the skills to design, build, and maintain reliable data pipelines and ETL processes to feed databases and data warehouses using a variety of tools and techniques.  You will have the opportunity to work with various programming languages, technologies, and both structured and unstructured data.

A qualified candidate:

• Is a Lifelong Learner and Passionate about Technology
• Derives joy from tackling complex problems and working through solution tradeoffs
• Can learn on the fly and fill knowledge gaps on demand 
• Can work with a variety of people at various levels
• Has excellent data management and QA skills – Process Oriented
• Can debug problems to their root cause, especially when the path leads through multiple systems or environments
• Has interest in working with data at the protocol level
• Has aptitude for data presentation and ability to transform raw data into meaningful, actionable reports
• Has significant experience creating data pipelines and ETL processes
• Is experienced with Google Cloud Composer / Apache Airflow or similar data orchestration services
• Is experienced with BigQuery or other data warehouse products
• Has excellent communication ability

Essential Functions

• Consults on complex data product projects by analyzing moderate to complex end to end data product requirements and existing business processes to lead in the design, development and implementation of data products.
• Builds data cleansing, imputation, and common data meaning and standardization routines from source systems by understanding business and source system data practices and by using data profiling and source data change monitoring, extraction, ingestion, and curation of data flows.
• Responsible for producing data views and data flows for varying client demands such as dimensional data, standard and ad hoc reporting, data feeds, dashboard reporting, and data science research & exploration.
• Translates business data stories into a technical story breakdown structure and work estimate so value and fit for a schedule or sprint.
• Creates business user access methods to structured and unstructured data by such techniques as mapping data to a common data model, transforming data as necessary to satisfy business rules and validation of data content.
• Collaborates with enterprise teams and other internal organizations on CI/CD best practices experience using Google Tables, JIRA, Jenkins, Confluence etc.
• Implements production processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.
• Develops and maintains scalable data pipelines for both streaming and batch requirements and builds out new API integrations to support continuing increases in data volume and complexity
• Writes and performs data unit/integration tests for data quality with input from a business requirements/story, creates and executes testing data and scripts to validate that quality and completeness criteria are satisfied. Can create automated testing programs and data that are reusable for future code changes.
• Practices code management and integration with engineering Git principle and practice repositories.
• Participates as an expert and learner in team tasks for data analysis, architecture, application design, coding, and testing practices.

Qualifications

• Bachelor's Degree in computer science, management information systems, business, statistics, math, a related field or comparable experience and education 
• Three to five years of relevant experience with data quality rules, data management organization/standards and practices.  
• At least three years’ experience in data warehousing and queries. 
• Experience with Cloud technology and infrastructure.
• Data application and practice knowledge. 
• Strong problem solving, oral and written communication skills. 
• Ability to influence, build relationships, negotiate, and present to senior leaders.
• Experience manipulating, processing, and extracting value from large, disconnected datasets
• Advanced query authoring (SQL)
• Advanced Python scripting
• Advanced working knowledge of a variety of databases
• Working experience with Git, and GitHub or GitLab.
• Advanced skills with Python and SQL required