Lead Data Engineer
Bethesda, MD, United States
Full Time Senior-level / Expert USD 146K - 197K
Total Wine & More
- In partnership with the Data Services leadership team, develop the vision for the next evolution of our Enterprise Data & Analytics Platform
- Partner with Data Services product owners, data warehouse engineers, data scientists, and BI developers to support and deliver best-in-class reports, predictive models, datasets, and solutions.
- Create and manage batch and real time data pipelines.
- Manage and enhance our Kafka and Kubernetes implementations.
- Stay informed of new technologies and trends to guide the optimization of and continuous advancement of our platforms.
- Conduct exploratory data analysis to understand the patterns and potential business insights exhibited in the data.
- Be proactive and take ownership of the projects and tasks assigned.
- Coach junior team members and share knowledge with the broader team.
- Engage in implementing good security practices to handle sensitive information.
- Bachelor's degree from four-year College or university in Computer Science, Technology, Business, or related field. Master’s or PhD preferred.
- 5+ years of experience in data engineering, data architecture, data warehousing or related fields.
- Proven track record in building impactful platforms and solutions using cloud-native tools.
- Experience using ETL/ELT tools – Airflow, Data Fusion, Dataprep, Informatica, Fivetran.
- Experience using Apache Kafka or Confluent Kafka or Google Pub/Sub.
- Experienced with general programing languages, such as Python, SQL, and SQL-like query languages for NoSQL databases.
- Experience using MapReduce or distributed computing technologies like PySpark, Apache Beam.
- Experience in leveraging the cloud in the development of data & analytics platforms.
- Familiarity with Dev-ops, Kubernetes, Docker.
- Experience with CI/CD including GitHub, Cloud Build, Jenkins.
- Passionate about data, data engineering, and platform development.
- Strong analytic skills, attention to detail, ability to multi-task, troubleshoot and problem solve along with the ability to make recommendations based on analysis are extremely important.
- Must possess excellent verbal, written, and presentation communication skills.
- Experience with cloud-based analytical platforms (GCP, Azure) is preferred.
- Experience with workflow management (Airflow) and data warehouse tools (dbt) is preferred.
- Knowledge of data lifecycle management (DLM) is preferred.
- Familiar with retail operations is preferred.
- Paid Time Off (PTO)
- Generous store discounts
- Health care plans (medical, prescription, dental, vision)
- 401(k), HSA, FSA, Pre-tax commuter benefits
- Disability & life insurance coverage
- Paid parental leave
- Pet insurance
- Critical illness and accident insurance
- Discounted home and auto insurance
- College tuition assistance
- Career development & product training
- Consumer classes
- & More!
Tags: Airflow Architecture Azure Business Intelligence CI/CD Computer Science CX Data analysis Data management Data Mining Data pipelines Data warehouse Data Warehousing dbt Docker EDA ELT Engineering ETL FiveTran GCP GitHub Informatica Jenkins Kafka Kubernetes NoSQL PhD Pipelines PySpark Python Security SQL
Perks/benefits: Career development Health care Insurance Medical leave Parental leave
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.