Data Engineer (GDC)

My Company

We’re Hitachi Digital Services, a global digital solutions and transformation business with a bold vision of our world’s potential. We’re people-centric and here to power good. Every day, we futureproof urban spaces, conserve natural resources, protect rainforests, and save lives. This is a world where innovation, technology, and deep expertise come together to take our company and customers from what’s now to what’s next. We make it happen through the power of acceleration.

Imagine the sheer breadth of talent it takes to bring a better tomorrow closer to today. We don’t expect you to ‘fit’ every requirement – your life experience, character, perspective, and passion for achieving great things in the world are equally as important to us.

 

Job Description

Data Engineer

The Data Engineer for Historian Integration, Aggregations, and Development is responsible for designing, implementing, and maintaining data pipelines that integrate operational data from Historian systems into enterprise data platforms. This role involves working with time-series data, developing efficient aggregation processes, and ensuring seamless data integration to support analytics and reporting needs.

Key Responsibilities:

Historian Data Integration:

Design and implement data pipelines to extract, transform, and load (ETL) data from Historian systems into centralized data warehouses or cloud platforms.

Ensure data accuracy, consistency, and reliability during the integration process.

Collaborate with OT (Operational Technology) teams to understand data sources and requirements.

Data Aggregation and Transformation:

Develop and optimize data aggregation processes to create summary tables, views, and reports for time-series data.

Implement data transformations to support advanced analytics, including data cleansing, normalization, and enrichment.

Design and maintain data models that support both real-time and batch processing.

Data Pipeline Development:

Build scalable, efficient, and resilient data pipelines using tools and technologies such as Python, SQL, Apache Kafka, Apache NiFi, or similar ETL frameworks.

Monitor and maintain data pipelines to ensure high availability and performance.

Implement automation for data ingestion and processing workflows.

Performance Optimization:

Analyze and optimize the performance of data integration and aggregation processes, ensuring low latency and high throughput.

Fine-tune Historian queries and data extraction processes to improve efficiency.

Identify and resolve data bottlenecks and performance issues.

Collaboration and Communication:

Work closely with data scientists, analysts, and other stakeholders to understand data needs and deliver solutions that meet business requirements.

Collaborate with IT and OT teams to ensure data security and compliance with industry standards.

Provide technical guidance and support to junior data engineers and developers.

Documentation and Reporting:

Maintain detailed documentation of data integration processes, data models, and system configurations.

Generate and deliver reports on data integration performance, data quality, and system health.

Required Qualifications:

Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.

3-5 years of experience in data engineering, with a focus on time-series data and Historian systems.

Proficiency in ETL tools and frameworks (e.g., Apache NiFi, Talend, Informatica).

Strong experience with SQL and Python for data processing and analysis.

Preferred Qualifications:

Experience with industrial Historian systems (e.g., OSIsoft PI, Wonderware, GE Historian).

Familiarity with cloud platforms (e.g., AWS, Azure) and their data integration services.

Understanding of OT and SCADA systems, and their data characteristics.

Knowledge of big data technologies (e.g., Apache Hadoop, Spark) and time-series databases.

Key Skills:

Technical Skills:

ETL development and data pipeline orchestration.

Time-series data management and processing.

SQL and Python for data manipulation.

Data modeling and database design.

About us

We’re a global team of innovators. Together, we harness engineering excellence and passion to cocreate meaningful solutions to complex challenges. We turn organizations into data-driven leaders that can make a positive impact on their industries and society. If you believe that innovation can bring a better tomorrow closer to today, this is the place for you.

#LI-KH1

Championing diversity, equity, and inclusion

Diversity, equity, and inclusion (DEI) are integral to our culture and identity. Diverse thinking, a commitment to allyship, and a culture of empowerment help us achieve powerful results. We want you to be you, with all the ideas, lived experience, and fresh perspective that brings. We support your uniqueness and encourage people from all backgrounds to apply and realize their full potential as part of our team.

How we look after you

We help take care of your today and tomorrow with industry-leading benefits, support, and services that look after your holistic health and wellbeing. We’re also champions of life balance and offer flexible arrangements that work for you (role and location dependent). We’re always looking for new ways of working that bring out our best, which leads to unexpected ideas. So here, you’ll experience a sense of belonging, and discover autonomy, freedom, and ownership as you work alongside talented people you enjoy sharing knowledge with.

We’re proud to say we’re an equal opportunity employer and welcome all applicants for employment without attention to race, colour, religion, sex, sexual orientation, gender identity, national origin, veteran, age, disability status or any other protected characteristic. Should you need reasonable accommodations during the recruitment process, please let us know so that we can do our best to set you up for success.