Data Engineer (Data & BI)

Company Description

Lely, founded in 1948, focuses on a sustainable, profitable, and enjoyable future in the agricultural sector. With the cow at the center, the company develops high-quality robots and data systems that enhance animal welfare, flexibility, and production on dairy farms.

For more than 25 years, Lely has been a global leader in the sales and service of automated systems to successive generations of dairy farmers. Every day, Lely inspires its employees to offer customers innovative solutions and be a trusted partner for long-term advice and support. With its headquarters in Maassluis and a worldwide network of dedicated Lely Centers for tailored sales and service, the Lely Group operates in more than sixty countries.

Job Description

At Lely, we call ourselves 'farming innovators' and not without reason, because we lead the market with technical revolutions. Digital innovation is a crucial factor in achieving our strategic plan "Impact30" and Data & BI is of great importance in this regard. With our enthusiastic Data & BI team, we enable the organization to make increasingly data-driven decisions. By joining our team, you actively contribute to the development of Lely.

We focus on all non-robot related domains ( like sales, finance, service and operations). Our Data & BI team consists of three disciplines: Business Intelligence, Data Governance, and Data Warehouse Management (DWHM).

We are looking for a Data Engineer. This position is part of the DWHM team, consisting of a data engineer and an analytics engineer. You will also work closely with analysts within the BI department. Additionally, you will be part of a growing data & AI community within Lely, where we increasingly share knowledge

Our team is responsible for ingesting, transforming and presenting data from (external) sources and creating foundational data products intended for business reporting and analytics. Together with our internal Databricks platform team, we strive to continuously improve our data warehouse to be more cost-efficient, secure and reliable for all users and applications within Lely.  

As a Data Engineer you are responsible for building scalable, maintainable and cost-efficient data ingestion pipelines.

Your main responsibilities include:  

• Building, maintaining and optimizing ingestion jobs from various type of data sources using Python into Databricks.

• Managing and configuring Databricks components such as Workflows, Unity Catalog, and Compute clusters within our Databricks workspace.

• Deploying and managing our infrastructure components on Azure using OpenTofu/Terraform.

• Supporting and collaborating with users and our colleagues from the BI team.

• Monitoring costs to ensure efficient resource usage. 

• Ensuring quality control based on design, code reviews, unit and integration testing, and addressing technical debt.  

• Contributing to the Agile work process by setting targets, reporting on progress, and managing risks and threats. 

Technologies we use:

• Databricks as our data warehouse.
• Python with Databricks Workflows as our ingestion and orchestration framework.
• DBT as our transformation framework.
• Microsoft Azure as our cloud platform.
• OpenTofu/Terraform as our IaC tool.
• Gitlab as our version control & CI/CD platform.

Qualifications

• Software or Data Engineering using Python or other object-oriented languages (e.g., Java, C#, Scala, Go, Rust).
• SQL with a solid understanding of query optimization and database design.
• Data Ingestion from diverse sources, including APIs, SQL databases, and FTP servers. 
• Working with APIs, with a deep understanding of API complexities and best practices.
• Cloud technologies, preferably within the Azure ecosystem.
• Unit and Integration Testing to ensure robust and reliable code.
• Data Design Patterns in Medallion Architecture.

Additional Information

• Working in an international environment in one of the most sustainable office/production locations in Europe.
• You will be working at the best family business in the Netherlands.
• Freedom in organizing your own work.
• Lely has excellent working conditions, such as 27 vacation days and 13 ADV days.
• A good salary fitting the weight of the role in relation to your knowledge and skills.
• An open company culture.
• Flexible working hours; Working from home is possible.