Databricks Practice Lead

Company Description

At our core, Data Intellect is a data and technology consultancy firm. Our key area of expertise is financial and capital markets technology solutions. However, the utility of these solutions allow us to apply fintech data expertise to other industries such as smart energy and healthcare.

This proprietary offering is complemented by a wealth of experience in data engineering, electronic trading systems, data capture applications, regulatory and compliance systems and middle and back-office enterprise web solutions.

Our mission is to help organisations unlock the full potential of their data, enabling strategic decision-making and transformative business growth. We are known for our deep technical expertise, collaborative approach, and commitment to innovation.

Simply put – we turn big data problems into smart data solutions.

Job Description

We are seeking a highly skilled and motivated Databricks Lead to spearhead our Databricks practice. This is a leadership role responsible for driving the growth, innovation, and successful delivery of cloud-based data solutions for our clients leveraging Databricks. The Practice Lead will collaborate with internal teams and clients to design, implement, and optimize big data and machine learning workflows, driving value through scalable and efficient data architectures.

The ideal candidate has a deep understanding of cloud-based data processing, particularly within a Databricks environment, and a passion for leading and mentoring a team of data engineers. This role requires both technical expertise and strategic leadership, making it a unique opportunity for someone who is looking to shape the future of data-driven solutions within a fast-growing, innovative company.

Key Responsibilities

The role will be working with the CTO on the following:

1. Practice Leadership

• Develop and lead the Databricks practice, establishing it as a key offering within Data Intellect’s services.
• Define the strategic vision for the practice, including service offerings, go-to-market strategies, and best practices.
• Stay up to date with current with Databricks advancements, industry trends, and competitor strategies to ensure continuous innovation.
• Collaborate the Business Development Team to drive new business opportunities and expand the client base.

2. Team Management & Development

• Build, mentor, and manage a team of data engineers and consultants within the Databricks practice.
• Foster a culture of innovation, continuous learning, and knowledge sharing within the team.
• Develop training programs and professional development opportunities to ensure team members are skilled in the latest technologies and best practices.

3. Project Delivery & Execution

• Lead the design, development, and implementation of complex data pipelines, ETL processes, and data lakes using Databricks and other services
• Provide oversight and guidance to project teams ensuring high-quality delivery of data solutions, including scalability, performance, and security.
• Partner with clients to understand their data needs, architect tailored solutions, and provide strategic guidance on leveraging Databricks to meet business objectives.

4. Client Engagement & Relationship Building

• Act as the primary point of contact for Databricks expertise in discussions with key clients, offering thought leadership and driving long-term relationships.
• Lead client discussions around technical architecture, project scope, and implementation plans.
• Ensure client satisfaction through proactive communication, project transparency, and post-deployment support.

5. Technical Excellence & Innovation

• Establish best practices for Databricks deployments, including performance tuning, cost optimisation, and security management.
• Develop reusable frameworks, assets, and accelerators to improve delivery efficiency and consistency across projects.
• Advocate for the adoption of advanced analytics, AI, and machine learning capabilities within Databricks to drive additional business value.

Qualifications

Key Qualifications

• Strong experience in data engineering, data architecture, or big data analytics.
• Proven experience designing and deploying large-scale data pipelines, ETL processes, and data lakes using Databricks.
• Demonstrable experience with cloud services, all three public clouds (AWS, GCP, Azure) is beneficial.
• Solid understanding of Spark architecture, distributed computing, and cloud-based data engineering principles.
• Proficiency in programming languages such as Python or SQL.
• Strong leadership skills with experience managing and growing high-performing technical teams.
• Excellent client-facing communication skills, including the ability to translate technical solutions into business value.
• Experience in Consulting / Financial Services / Capital Markets, delivering solutions for multiple clients is advantageous.

Key Competencies

• Leadership: Ability to inspire, lead, and develop a high-performing team.
• Strategic Thinking: Proactively identifies opportunities to innovate and expand service offerings.
• Problem-Solving: Demonstrates strong analytical skills to solve complex technical problems.
• Communication: Excellent interpersonal and presentation skills to engage with clients and stakeholders.
• Collaboration: Works effectively across departments, teams, and client organizations.

Additional Information

Why Join Data Intellect?

• Be part of a rapidly growing, forward-thinking data consultancy that prioritises innovation and client success.
• Work alongside a talented, passionate, and collaborative team of data experts.
• Access to continuous learning and professional development opportunities.
• Competitive salary and benefits package.
• Opportunities to lead high-impact projects with top-tier clients.

Fair employment and equal opportunities

Data Intellect is an equal opportunity employer, committed to inclusion and diversity.

If you have a disability, accommodations are available on request throughout the assessment and selection processes.

Ready to accept the challenge?

Apply now.