Senior Data Engineer
London, Greater London, United Kingdom
- Remote-first
- Website
- @opencorporates 𝕏
- GitHub
- Search
OpenCorporates
About OpenCorporates
OpenCorporates is a certified B Corp, revolutionizing access to company data. It has built the world's largest open database of companies, and is trusted by regulators, financial institutions, investigative journalists, and businesses for its unparalleled breadth of data, covering over 200 million companies across 140+ jurisdictions, including the entirety of the USA.
Legal entities are at the heart of the modern world – and as we move into a world of digitalized commerce and AI-generated uncertainty, OpenCorporates will be there to provide the trusted foundational data the world needs.
Why join us?
We’re at an exciting stage of our journey, recently appointing Wojtek Kokoszka as our new Chief Executive Officer. Wojtek is a seasoned entrepreneur and co-founder of Zappi, the leading SaaS Consumer Insights platform. Wojtek played a pivotal role in scaling Zappi to $100 million in revenue.
Wojtek joins OpenCorporates with the clear mission to scale the world's authoritative source of legal-entity data, driving performance, fostering innovation, and executing strategic vision to lead OpenCorporates into its next phase of fast growth.
Requirements
About the Senior Data Engineer role
Data is our product. Our data is used to support a plethora of client solutions from KYC to analytics and data management. Hundreds of organisations rely on our data at scale to power their critical operations, including the US government, the OECD, Oracle and many others.
As the senior data engineer you will be instrumental in building and optimizing our data pipelines, ensuring scalability, efficiency, and reliability. You will need to possess a deep understanding of Snowflake, AWS data tooling, data warehousing, and best practices in cloud-based data platforms.
You will be responsible for:
- Designing, developing and maintaining robust data ingestion pipelines using AWS services (S3, Lambda, ECR, ECS, DynamoDB).
- Developing and optimising data transformation pipelines using a combination of SQL and Python.
- Working extensively with Snowflake as a data platform, ensuring efficient and scalable structures and patterns.
- Implementing best practices for data engineering, including performance tuning, security, and data governance.
- Collaborating with data analysts, scientists, and other stakeholders to deliver high-quality data solutions.
- Monitoring, troubleshooting, and optimising data pipelines for performance and reliability.
- Ensuring data quality, consistency, and integrity across multiple data sources
We would like you will need to demonstrate solid experience in:
- Data warehousing concepts.
- Expert-level proficiency in SQL and Python, with a focus on simplicity and maintainability.
- Hands-on experience with AWS services, specifically S3, Lambda, ECR, ECS, and DynamoDB.
- Deep understanding of data modeling, including dimensional modeling and best practices for cloud-based data platforms.
- Strong problem-solving skills with the ability to optimize queries and improve data performance.
- Experience in handling large-scale datasets and implementing scalable data solutions.
Bonus points if you have 1 or 2 of the following:
- Experience with utilising AI in the data engineering space.
- Experience with Terraform for infrastructure as code.
- Knowledge of CI/CD practices for automating deployment processes.
- Familiarity with Data Vault modeling for data warehousing.
Benefits:
- We're flexible – We support a culture of flexibility which allows our employees to achieve a strong work-life balance.
- Offices – We are remote first but not remote only. We have a strong in person collaboration culture and are looking for people who can work in a remote and hybrid setting.
- Holiday – 28 days of annual leave per year, plus bank holidays & an extra day for every year of service.
- Work abroad – You have the flexibility to work outside of the UK for up to 4 weeks in a rolling 12-month period.
- Regular company socials.
- For those special moments in your life, such as a big birthday or your wedding, we offer discretionary time off.
- Healthy living – Private healthcare with Aviva, life assurance, plus annual budget for your wellbeing.
- Personal development – We offer personal development budgets and learning days.
- Competitive package - alongside your base salary we offer share options and a bonus scheme.
- Cycle to work scheme.
- £500 home office set-up budget.
Diversity Matters
Don’t meet every single requirement? At OpenCorporates we're assembling a diverse innovative team that defies our industry's norms. Think this role could suit you? We encourage you to apply even if your past experience doesn’t align perfectly with every qualification. We welcome applicants with a curious growth mindset.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: AWS CI/CD Data governance Data management Data pipelines Data quality Data Warehousing DynamoDB ECS Engineering Lambda Oracle Pipelines Python Security Snowflake SQL Terraform
Perks/benefits: Career development Competitive pay Equity / stock options Flex hours Flex vacation Home office stipend Salary bonus Startup environment
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.