Data Engineer

About Netcraft

Netcraft is the global leader in cybercrime detection and disruption. We’re a trusted partner for three of the four largest companies in the world and many large country governments. We’ve blocked more than 200 million malicious sites and perform takedowns for around one-third of the world’s phishing sites.  

Our purpose, passion, and expertise are focused on just one thing: protecting the world from cybercrime.   

Our passion doesn’t stop at what we do—it shapes how we work, too. We’re proud of our talented team and the value each person brings. That’s why we’ve created a workplace where people feel supported and inspired, from great benefits and wellness programs to fun social events. 

The role 

Reporting to one of Netcraft’s Engineering Directors, you will work cross-functionally to help build, improve, maintain Netcraft’s vast data estate, as well as support teams to leverage that data to solve a range of problems within their respective services. As a Data Engineer you will have a deep understanding of modern relational DB and NoSQL technologies, plus hands-on data pipeline/ETL development experience, and strong focus on automation, security, performance, and observability.  

You’ll be: 

• Deploying, configuring, and maintaining OpenSearch/Elasticsearch clusters in high-volume environments
  
• Tuning performance to improve indexing and search latency
  
• Managing upgrades and security controls
  
• Monitoring and troubleshooting distributed search infrastructure
  
• Designing and building pipelines for ingesting structured and unstructured data
  
• Designing and optimising index strategies including sharding, partitioning, and improving data locality to ensure scalable, high-performance querying across large datasets
  
• Working with other teams to formulate ETL approaches for their respective data stores
  
• Optimising index mappings for search and analytics use cases
  
• Balancing storage efficiency vs. query speed at scale, and able to advise on alternative solutions and migration approaches
• Cost-aware, and able to articulate business benefits against the often-high run cost of at-scale data services

What you’ll need to be successful: 

• Experience designing and deploying webscale SaaS solutions that make use of OpenSearch or Elasticseach based search engines
• Strong background in cloud engineering and containerisation (Docker, Kubernetes), with experience deploying services at scale, particularly on AWS
• AWS managed data service experience, such as S3, RDS, Aurora, DynamoDB, etc.
• A strong focus on automation and operational efficiency, minimising manual intervention wherever possible to enhance system reliability
• Strong troubleshooting and problem-solving skills, especially in complex, cloud-based infrastructure
• Experience with AWS cost management and optimisation, using tools like AWS Cost Explorer and AWS Budgets to monitor, manage, and explain AWS spend

Bonus points if you have: 

• Experience in managing MongoDB clusters for high-availability and scalability
  
• Comfortable working with SQL-based databases (MariaDB, MySQL, Postgres), including schema design, query optimisation, and debugging scaling issues in high-load environments
• Streaming/eventing technologies like Kafka
  
• Infrastructure-as-code experience, especially Terraform
  
• Setting up dashboards and alerts with observability tooling
  
• Proficiency in a major back-end language, like Golang 

The reward package 

An excellent range of benefits including:   

• Hybrid and flexible working options 
  
• Minimum of 33 days holiday per annum (incl. public holidays)  
• Pension scheme membership with 4% employer contributions + NI savings  
• Private health cover, including access to a private GP service  
• Equity tracking scheme, so you can share in the rewards of Netcraft's long-term success (eligibility criteria apply)  
• Comprehensive wellness and support provisions  
• Enhanced family leave provisions  
• Life Assurance  
• Two days paid Volunteering Leave per year  
• Free meals, drinks and snacks provided daily in the offices  
• Regular social events such as board game nights, big summer party and annual kick-off  
• Inclusive culture and environment, where you’ll feel genuinely valued and respected; and  
• A tax-efficient cycle to work scheme.   

Diversity, Equity and Inclusion
 

This is very important to us and through our ally network we support under-represented groups. We seek to maintain a working environment that is free from bias, harassment or discrimination, and we encourage candidates from any background to apply, regardless of their gender, gender identity, sexual orientation, race/ethnicity, ability/disability, age, religion, or any other specific characteristics.  

We’re happy to make adjustments to our hiring process to ensure that all candidates can participate fully and comfortably. 

Please note Netcraft does not accept any unsolicited approaches from external recruiters.