Data Engineer

Do you want to boost your career and collaborate with expert, talented colleagues to solve and deliver against our clients' most important challenges? We are growing and are looking for people to join our team. You'll be part of an entrepreneurial, high-growth environment of 300.000 employees. Our dynamic organization allows you to work across functional business pillars, contributing your ideas, experiences, diverse thinking, and a strong mindset. Are you ready?

Job Overview: 

We are looking for a skilled and experienced Data Engineer to join our data team. The Data Engineer will be responsible for designing, building, and maintaining scalable and efficient data pipelines and architectures that support various data-driven initiatives. The ideal candidate will have strong technical expertise in data processing, data storage, and data integration, and will work closely with data scientists, analysts, and other stakeholders to ensure the integrity, availability, and reliability of the data. 

Key Responsibilities: 

1. Design and Build Data Pipelines: 

• Develop, construct, test, and maintain scalable data pipelines for ingestion, processing, and storage of large data sets from multiple sources. 

• Optimise data architectures for both structured and unstructured data to support batch and real-time data processing. 

1. Data Integration and Management: 

• Integrate data from different source systems (APIs, databases, cloud services) into the data lake or data warehouse. 

• Implement and manage ETL (Extract, Transform, Load) processes for efficient data movement, cleaning, and transformation. 

• Ensure data quality, consistency, and integrity through proper validation and testing. 

1. Database and Data Warehousing Solutions: 

• Design and implement database solutions that support data storage, transformation, and querying. 

• Build and maintain data warehouse solutions to facilitate business intelligence and analytics needs. 

1. Cloud Infrastructure: 

• Work with cloud platforms such as AWS, Azure, or Google Cloud to implement data storage, processing, and streaming architectures. 

• Set up and manage cloud-based data infrastructure, ensuring scalability, security, and performance optimisation. 

1. Collaboration and Stakeholder Management: 

• Collaborate with data scientists, analysts, software engineers, and business teams to understand data requirements and deliver solutions that meet their needs. 

• Document data flows, schemas, and pipeline designs to ensure transparency and understanding across the team. 

1. Data Governance and Security: 

• Implement data governance standards, data security, and compliance measures to protect sensitive data. 

• Ensure compliance with data privacy regulations such as GDPR, CCPA, etc. 

• Define and maintain data retention policies and ensure adherence to best practices for data management. 

1. Performance Monitoring and Optimisation: 

• Monitor data pipeline performance, identify bottlenecks, and troubleshoot issues as they arise. 

• Continuously improve data processes to ensure they are efficient and scalable for growing data sets. 

1. Automation and Tools Development: 

• Automate repetitive tasks and processes to streamline data engineering workflows. 

• Develop tools and frameworks for data access, management, and analysis, enabling the business to easily work with data. 

Requirements

Skills and Qualifications: 

Essential Skills: 

• Bachelor’s degree in Computer Science, Engineering, or a related field. 

• Proven experience as a Data Engineer, Software Engineer, or similar role in large scale data platform implementation (project team size of >= 10 FTEs and/or project duration 6 months or more). 

• Strong experience in SQL and relational database systems (e.g., MySQL, PostgreSQL, Oracle). 

• Proficiency in programming languages such as Python, Java, or Scala for data manipulation and ETL tasks. 

• Minimum 5 years of hands-on experience with ETL tools like Apache Nifi, Talend, Informatica, or equivalent. 

• Familiarity with big data technologies such as Hadoop, Spark, Kafka, and HBase. 

• Minimum 3 years of hands-on experience with cloud-based data services (e.g., AWS S3, Azure Data Lake, Google BigQuery, Redshift). 

• Knowledge of data modelling, database design, and data architecture best practices. 

• Experience with version control (e.g., Git) and agile development practices. 

Preferred Skills: 

• Experience with NoSQL databases like MongoDB, Cassandra, or DynamoDB. 

• Experience with Databricks and Dataiku. 

• Familiarity with machine learning pipelines and integration with data systems. 

• Knowledge of CI/CD pipelines and automation tools like Jenkins, Docker, or Kubernetes. 

• Exposure to real-time data processing tools such as Apache Flink or Storm. 

• Understanding of data privacy regulations (GDPR, CCPA, HIPAA) and how they impact data storage and processing. 

Personal Attributes: 

• Strong problem-solving skills and ability to work independently or as part of a team. 

• Excellent communication skills, with the ability to explain technical concepts to non-technical stakeholders. 

• Attention to detail and a focus on data accuracy and quality. 

• Proactive mindset with a passion for innovation and staying up-to-date with emerging technologies. 

Given that this is just a short snapshot of the role we encourage you to apply even if you don't meet all the requirements listed above. We are looking for individuals who strive to make an impact and are eager to learn. If this sounds like you and you feel you have the skills and experience required, then please apply now.

Benefits

About your team

Join our growing Data & Analytics practice and make a difference.

In this practice you will be utilizing the most innovative technological solutions in modern data ecosystem.  In this role you’ll be able to see your own ideas transform into breakthrough results in the areas of Data & Analytics strategy, Management & Governance, Data Integration & engineering, Analytics & Data science. 

About Infosys Consulting

Be part of a globally renowned management consulting firm on the front-line of industry disruption and at the cutting edge of technology.  We work with market leading brands across sectors. Our culture is inclusive and entrepreneurial. Being a mid-size consultancy within the scale of Infosys gives us the global reach to partner with our clients throughout their transformation journey.

Our core values, IC-LIFE, form a common code that helps us move forward. IC-LIFE stands for Inclusion, Equity and Diversity, Client, Leadership, Integrity, Fairness, and Excellence. To learn more about Infosys Consulting and our values, please visit our careers page.

Within Europe, we are recognized as one of the UK’s top firms by the Financial Times and Forbes due to our client innovations, our cultural diversity and dedicated training and career paths. Infosys is on the Germany’s top employers list for 2023. Management Consulting Magazine named us on their list of Best Firms to Work for. Furthermore, Infosys has been recognized by the Top Employers Institute, a global certification company, for its exceptional standards in employee conditions across Europe for five years in a row.

We offer industry-leading compensation and benefits, along with top training and development opportunities so that you can grow your career and achieve your personal ambitions. Curious to learn more? We’d love to hear from you.... Apply today!