Data Manager

Allata is a fast-growing technology strategy and data development consulting firm delivering scalable solutions to our enterprise clients. Our mission is to inspire our clients to achieve their most strategic goals through uncompromised delivery, active listening, and personal accountability. We are a group who thrive in fast-paced environments, working on complex problems, continually learning, and working alongside colleagues to be better together.
We are actively seeking a Data Manager with strong expertise in DataBricks to guide and drive the evolution of our client's data ecosystem. 
The ideal candidate will combine technical leadership with hands-on execution, leading the design, migration, and implementation of robust data solutions while mentoring team members and collaborating with stakeholders to achieve enterprise-wide analytics goals. 

Role & Responsibilities:

• Provide technical leadership in designing, building, and maintaining reusable data products using Data Bricks, Python, and SQL. 
• Lead the migration of existing data pipelines to modern frameworks and ensure scalability and efficiency. 
• Establish and enforce data quality standards, leveraging automated testing and reporting to ensure data accuracy, consistency, and compliance. 
• Develop and oversee the data infrastructure, pipeline architecture, and integration solutions while actively contributing to hands-on implementation. 
• Collaborate with business and technical stakeholders to define requirements and ensure data solutions align with business goals. 
• Build and maintain scalable, efficient data processing pipelines and solutions for data-driven applications. 
• Develop comprehensive documentation of data engineering processes and provide mentorship to junior team members. 
• Monitor and ensure adherence to data security, privacy regulations, and compliance standards. 
• Troubleshoot and resolve complex data-related challenges and incidents in a timely manner. 
• Stay at the forefront of emerging trends and technologies in data engineering and advocate for their integration when relevant. 

Hard Skills - Must have:

• Proven expertise in DataBricks, SQL, and Python for processing and managing large data volumes. 
• Strong experience in desiging and migrating existing databricks workspaces to unity catalog workspaces.  
• Familiarity with batch and streaming data processing techniques. 

Hard Skills - Nice to have/It's a plus:

• Familiarity with architectural best practices for building data lakes. 
• Knowledge of SSIS/DacPacs. 
• Demonstrated experience Familiarity with applying DevOps principles (e.g., Git, CI/CD) to data engineering projects, Pulumi in Typescript. 
• Experience with Azure Functions. 
• Knowledge of ASP.Net and .Net Core. 
• Knowledge of observability and monitoring frameworks for data solutions. 
• Experience with Databricks Unity Catalog, i.e. configuring and managing data governance and access controls in a Lakehouse environment 

Soft Skills / Business Specific Skills:

• Demonstrated ability to lead technical discussions and guide teams in solving complex data challenges. 
• Strong collaboration and stakeholder management skills in distributed teams. 
• Proactive problem-solving attitude with excellent organizational and analytical capabilities. 
• Fluent English communication skills, both written and verbal. 
• Familiarity with Agile methodologies and SDLC practices. 

At Allata, we value differences.
Allata is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.
Allata makes employment decisions without regard to race, color, creed, religion, age, ancestry, national origin, veteran status, sex, sexual orientation, gender, gender identity, gender expression, marital status, disability or any other legally protected category.
This policy applies to all terms and conditions of employment, including but not limited to, recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training.