Senior Data Consultant (Databricks)

Senior Data Consultant (Databricks)

Application Deadline: 26 May 2025

Department: Managed Services & Support

Employment Type: Full Time

Location: South Africa

Compensation: R900,000 - R1,350,000 / year

Description

We’re on the lookout for seasoned Senior Data Consultants who thrive in dynamic, fast-paced environments to join our global Managed Services team based across South Africa, the UK and India.

In this role, you'll take the lead in managing and optimising live Databricks cloud data platforms, ensuring pipelines are resilient, performant, and cost-efficient. As a trusted guardian of our clients’ data solutions, you’ll anticipate and resolve issues before they escalate, balancing immediate fixes with long-term platform health.

Operating within our premium support model, you'll collaborate closely with C-level stakeholders to provide architecture reviews, system performance enhancements, and FinOps strategies to keep cloud costs in check. You’ll also lead regular client touchpoints, offering strategic guidance and roadmap planning to drive platform maturity.

If you love working with clients and want to be at the forefront of cloud data performance, this role is for you.

What you'll be doing...

• Monitoring and managing Databricks environments, ensuring streaming and batch jobs run reliably and performance remains high.
  
  
• Proactively troubleshooting and resolving issues in Spark jobs, Delta Lake processes, and integrations with other Azure/AWS services.
  
  
• Running regular architecture reviews and optimising notebooks, clusters, and pipelines for efficiency and scalability.
  
  
• Acting as the escalation point for complex support issues and production incidents.
  
  
• Advising on best practices for data governance, CI/CD, and platform observability within Databricks.
  
  
• Leading monthly service reviews with clients and contributing to long-term strategy and cost forecasting.
  
  
• Collaborating with vendor support (Databricks, Azure, AWS) and client-side engineering teams to resolve issues seamlessly.

What you'll need to succeed...

• Deep expertise in Databricks, including Spark performance tuning, Delta Lake, and workspace administration.
  
  
• Strong Python and SQL skills, with experience building and supporting ETL/ELT pipelines in a cloud-native ecosystem.
  
  
• Understanding of data modelling principles and performance optimisation for analytics workloads.
  
  
• Experience handling high-pressure incidents, especially in live production environments.
  
  
• Excellent stakeholder management skills, especially when interacting with C-level clients.
  
  
• FinOps experience – able to analyse and reduce Databricks costs (cluster optimisation, job scheduling, etc.).
  
  
• Familiarity with support tooling such as ServiceNow, JIRA, or similar.

So, what's in it for you?

• The chance to work on cutting-edge Databricks platforms that underpin mission-critical business operations.
  
  
• The opportunity to influence architectural decisions and play a key role in scaling cloud-native data solutions.
  
  
• Clear progression into project delivery or platform consulting, with leadership opportunities along the way.
  
  
• Ongoing training in the latest tools and practices, and a fast-paced, high-growth environment.