Senior Data Engineer - Customer Data and Insights

More than a career - a chance to make a difference in people's lives.

Build an exciting, rewarding career with us – help us make a difference for millions of people every day. Consider joining the Duke Energy team, where you'll find a friendly work environment, opportunities for growth and development, recognition for your work, and competitive pay and benefits.

Position Summary:

This position is responsible for development and consulting activities related to analytics within various Duke Energy IT and business units.

• Support or collaborate with application developers, database architects, data analysts and data scientists to ensure optimal data delivery architecture throughout ongoing projects/operations.

• Design, build, and manage analytics infrastructure that can be utilized by data analysts, data scientists, and non-technical data consumers, which enables functions of the big data platform for Analytics.

• Develop, construct, test, and maintain architectures, such as databases and large-scale processing systems that help analyze and process data in the way the Analytics organization requires.

• Develop highly scalable data management interfaces, as well as software components by employing programming languages and tools.

• Work closely with data subject matter experts to determine what data management systems are appropriate and data scientists to determine which data is needed.

• Work with stakeholders to understand the information needs and translate these into technical solutions.

• Work closely with a team of Data Science staff to take existing or new models and convert them into scalable analytical solutions.

• Design, document, build, test and deploy data pipelines that assemble large complex datasets from various sources and integrate them into a unified view.

• Identify, design, and implement operational improvements: automating manual processes, data quality checks, error handling and recovery, re-designing infrastructure as needed.

• Create data models that will allow analytics and business teams to derive insights about customer behaviors

• Build new data pipelines, identify existing data gaps and provide automated solutions to deliver analytical capabilities and enriched data to applications. 

• Responsible for obtaining data from the System of Record and establishing batch or real-time data feed to provide analysis in an automated fashion.

• Develop techniques supporting trending and analytic decision making processes 

• Apply technologies for responsive front-end experience 

• Ensure systems meet business requirements and industry practices

• Research opportunities for data acquisition and new uses for existing data

• Develop data set processes for data modeling, mining and production

• Integrate data management technologies and software engineering tools into existing structures

• Employ a variety of languages and tools (e.g. scripting languages) for integration

• Recommend ways to improve data reliability, efficiency and quality

Responsibilities:

• Develop components required for optimal extraction, transformation, and loading of data from a wide variety of data sources using a set of tools or languages (e.g. SQL) 

• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

• Implement big data analytics solutions, as well as integrate new data management technologies and software engineering tools into existing structures

• Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs.

• Solve problems for Stakeholder,  construct problem frames in order to understand the analysis context and scope that will provide timely, useful results

• Create data tools for analytics and data scientist team members that assist them in building and optimizing products.

• Work in Project Teams and participate in multidisciplinary analytics project teams, translating complex functional and technical requirements into detailed architecture, design, and high performing software/techincal components

• Work with data and analytics experts to strive for greater functionality in data systems.

• Partner with business clients, establish professional relationship, and communicate with analytics clients in order to understand business needs.

Employees in this role will also:

• Translate complex functional and technical requirements into detailed architecture, design, and high performing software.

•  Drive user ideation analysis and elaboration, design and development of software applications, and build automation tools.

• Select data solution software and define hardware requirements.

• Help build a next-generation Big Data analytics framework with a set of core technologies.

• Code, test, and document new or modified data systems to create robust and scalable applications for data analytics.

• Ensure all automated processes preserve data by managing the alignment of data availability and integration processes.

• Collaborate with data architects, modelers and IT team members on project goals.

• Assist with extracting standard sets of features from important datasets that will be leveraged by Data Science teams.

• Convert code from applications.

• Build processes supporting data transformation, data structures, metadata, dependency and workload management.

Required/Basic Qualifications

• Bachelors degree in Management of Information Systems, Engineering, Mathematics, Computer Science or Other Related Degree

• In addition to required degree, three (3) or more years related work experience

• In lieu of Bachelors degree(s) AND three (3) or more years related work experience listed above, High School/GED AND five (5) or more years related work experience

Desired Qualifications:

• Architected, modeled and implemented a solution for multiple work management systems, including SS9, Maximo and Field Collection System in AWS Redshift

Additional Preferred Qualifications

• Knowledge of code writing.

• Experience with relational databases, query authoring (SQL) as well as familiarity with a variety of RDMS.

• Knowledge of building and optimizing ‘big data’ data pipelines, architectures and data sets.

• Knowledge of Open Analytics Platforms (such as Hadoop ecosystem) 

• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.

• Excellent verbal and written communication skills

• Self-starter who works with minimal supervision and the ability to work in a team of diverse skill sets

• Ability to comprehend customer requests and provide the correct solution

• Strong analytical mind to help take on complicated problems

• Desire to resolve issues and drive into potential issues

• Familiarity with the Agile Methodology

• Strong analytic skills related to working with unstructured datasets. Excellent verbal and written communication skills.

• Experience supporting and working with cross-functional teams in a dynamic environment.

• Experience with big data tools: Hadoop, Spark, Kafka, etc.

• Experience with relational SQL and NoSQL databases, including SQL Server and Cassandra.

• Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

• Understanding of distributed computing principles.

Working Conditions

• Hybrid – Work will be performed from both remote and onsite locations after the onboarding period. However, hybrid employees should live within a reasonable commute to the designated Duke Energy facility.

Travel Requirements

Relocation Assistance Provided (as applicable)

Represented/Union Position

Visa Sponsored Position

Posting Expiration Date

All job postings expire at 12:01 AM on the posting expiration date.

Please note that in order to be considered for this position, you must possess all of the basic/required qualifications.

Privacy

Do Not Sell My Personal Information (CA)

Terms of Use

Accessibility