IT Analyst - Data Engineer

At Duke Health, we're driven by a commitment to compassionate care that changes the lives of patients, their loved ones, and the greater community. No matter where your talents lie, join us and discover how we can advance health together.
About Duke Health Technology Solutions
Pursue your passion for caring and innovation with Duke Heath Technology Solutions, which is dedicated to the transformation, development, and management of enterprise information technology solutions across Duke Health. By harnessing the power of innovative technologies like cloud computing and artificial intelligence — and pairing them with a forward-thinking approach — Duke Health Technology Solutions is revolutionizing the future of health care at Duke Health and beyond.
 

POSITION DESCRIPTION
Job Title:  Senior Data Engineer

Primary Role:  This position reports to the DHTS-Data Partnerships, Director of Data and Analytics Platforms. This individual will be primarily responsible for the development of data integration and delivery pipelines while also expanding the FHIR-based content stored within Duke’s data lake (the Duke Data Hub). These solutions will capitalize on technologies to improve the value of analytical data, improve effectiveness of information stewardship, and streamline the flow of data in the organization. Solutions will focus on using state of the art data and analytics tools including traditional and near real-time integrations, big-data, and delta lake architecture using both extract, load, transform (ELT) toolsets as well as REST APIs and FHIR. The ideal candidate will also be comfortable with data science platforms with proven experience leveraging DevOps and Automation/Orchestration tools. 

Essential Tasks/Responsibilities
•    Create and maintain optimal data pipeline architecture
•    Develop a data lake on Microsoft Azure using the medallion architecture leveraging a delta lake format for the silver layer
•    Assemble large, complex data sets that meet functional / non-functional business requirements
•    Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
•    Recommend design of analytics solutions which improves data integration, data quality, and data delivery with an eye towards re-useable components
•    Articulate differences, advantages, and disadvantages between architectural solution methods
•    Work with Agile team members to document and execute test plans and data validation scripts. Support the code promotion process through development and production as required by using standard CI/CD processes
•    Develop monitoring, logging, and error notification processes to ensure data is updated as expected and processing metrics reported
•    Participate in the creation and maintenance of standards for coding, documentation, error handling, error notification, logging, etc. 
•    Accountable for conforming to established architectural, developmental, and operational standards and practices
•    Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics
•    Work with stakeholders including the Executive, Product, Data, and Design teams to assist with data-related technical issues and support their data infrastructure needs
•    Evaluate and recommend development tools
•    Assist in application and data operations performance tuning
•    Participate in system architecture design
•    Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader
•    Work with data and analytics experts to strive for greater functionality in our data systems
•    Share troubleshooting and maintenance duties

Education: Bachelor's degree in a related field, or four years of equivalent technical experience required

Required Experience:  We are looking for a candidate with 5+ years of experience in a Data Engineer role:

•    Experience implementing data lakes on Microsoft Azure
•    Experience with relational SQL and NoSQL databases
•    Experience with data pipeline and workflow management tools such as Azure Data Factory, Synapse Analytics pipelines
•    Experience with object-oriented/object function scripting languages such as Python or Java.
•    Experience with Cloud-based analytics platforms such as Azure Synapse Analytics

Required Skills:
•    Advanced SQL skills and experience working with a variety of relational database management systems
•    Intermediate to Advance skills in python development
•    Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement
•    Strong analytical skills related to working with unstructured datasets
•    Build processes supporting data transformation, data structures, metadata, dependency and workload management
•    A successful history of manipulating, processing and extracting value from large disconnected datasets
•    Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores
•    Experience supporting and working with cross-functional teams in a dynamic environment

Desired Skills: 
•    Experience with Microsoft Fabric
•    Working experience with the FHIR specification; including implementation.
•    Knowledge of APIs, API Integration, and API Management 
•    Working knowledge of DevOps & Automation/Orchestration 
•    Knowledge of open-source software solutions and open source as a business model 
•    Technical breadth across application development, enterprise architecture, or application integration
•    Understanding of Agile methodology
•    Prior experience in health care related field is a plus

The information above describes the general nature and level of work assigned to this position. It is not intended to be an exhaustive list of all duties and responsibilities required of position incumbents.

Duke is an Affirmative Action/Equal Opportunity Employer committed to providing employment opportunity without regard to an individual's age, color, disability, gender, gender expression, gender identity, genetic information, national origin, race, religion, sex, sexual orientation, or veteran status.

Duke aspires to create a community built on collaboration, innovation, creativity, and belonging. Our collective success depends on the robust exchange of ideas—an exchange that is best when the rich diversity of our perspectives, backgrounds, and experiences flourishes. To achieve this exchange, it is essential that all members of the community feel secure and welcome, that the contributions of all individuals are respected, and that all voices are heard. All members of our community have a responsibility to uphold these values.

Essential Physical Job Functions: Certain jobs at Duke University and Duke University Health System may include essentialjob functions that require specific physical and/or mental abilities. Additional information and provision for requests for reasonable accommodation will be provided by each hiring department.