Data Engineering Manager

It's fun to work in a company where people truly BELIEVE in what they're doing!
We're committed to bringing passion and customer focus to the business.

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!

Job Description:

We are seeking an experienced Data Engineering Manager with a strong background in Databricks, Apache Spark, and AWS EMR to both lead and contribute to the design, development, and optimization of our data infrastructure. This hybrid role combines hands-on technical responsibilities with leadership, mentoring, and team management. You will play a crucial role in guiding the data engineering team, collaborating with stakeholders, and ensuring that our data solutions are scalable, reliable, and aligned with business objectives.

Key Responsibilities:

• Leadership & Management: Lead and mentor a team of data engineers, providing technical guidance, performance feedback, and fostering a culture of continuous improvement.
• Define and manage the data engineering roadmap, aligning team goals with business objectives and ensuring timely delivery of high-impact projects.
• Collaborate with cross-functional teams, including data scientists, product managers, and business stakeholders, to define and prioritize data projects.
• Conduct regular reviews of data engineering processes and architectures, recommending improvements for performance, scalability, and cost-effectiveness.
• Design and implement scalable data pipelines using Databricks and Apache Spark for both real-time and batch processing.
• Manage and optimize large-scale data workflows on AWS EMR and ensure that all cloud-based infrastructure is cost-efficient and scalable.
• Establish best practices for data management, quality control, security, and governance across all data engineering activities.
• Oversee the development and maintenance of ETL processes, data models, and data lakes to support advanced analytics and reporting.
• Use Tableau and Power BI to develop key business reports and dashboards, facilitating data-driven decision-making at all levels of the organization.
• Act as a point of escalation for complex technical issues and help the team troubleshoot and resolve data pipeline failures or performance bottlenecks.

Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or a related field.
• 7+ years of experience in data engineering, with at least 2+ years in a leadership or management role.
• Extensive hands-on experience with Databricks and Apache Spark for building and optimizing large-scale data pipelines.
• Proven expertise in working with AWS EMR and other AWS services (e.g., S3, Redshift, Lambda, Glue) to manage distributed data systems.
• Proficiency in Python and SQL for data engineering and automation.
• Strong SQL skills and experience with relational and NoSQL databases (e.g., MySQL, Oracle, Cassandra, MongoDB).
• Experience with data visualization tools like Tableau, Quicksight and Power BI for delivering business insights.
• Familiarity with best practices for ETL, data warehousing, and data architecture design.
• Demonstrated ability to manage multiple projects and priorities in a fast-paced environment.

Preferred Skills:

• Experience with DevOps practices, including CI/CD pipelines for automating infrastructure deployments.
• Knowledge of machine learning workflows and MLOps integration.
• Familiarity with data governance frameworks and regulatory compliance (e.g., SOX).
• Excellent communication skills with the ability to present technical concepts to both technical and non-technical stakeholders.
• Proven leadership skills, with experience in recruiting, mentoring, and developing high-performing teams.

Pay Range:

P80-USA-1 :$138,600.00 - $198,000.00

Disclaimer:

Final base salary for the successful candidate will depend on multiple factors, including but not limited to, job location, where work will be performed, qualifications, work history and relevant experience. With our continual goal of making Lumentum a best place to work for our employees, we strive to offer employees competitive total compensation packages, which may include annual bonus, commission for certain sales roles, equity, and health and welfare benefits.