Big Data Lead

Snowflake Development - Responsibilities: Primary Skillset:Data Modeling & ETL: Design, develop, and optimize data models and ETL processes using Snowflake for efficient data storage and analytics. Design and implement end-to-end ETL pipelines for loading data from various sources into Snowflake. Utilize Snowflake’s built-in features such as tasks, streams, and Snowpipe to automate the ETL process for continuous and batch data loads. Implement data transformation logic using SQL, Snowflake Stored Procedures (SQL and Python), and ETL tool to ensure the integrity, accuracy, and consistency of data. Optimize data loads and transformations for scalability and performance using Snowflake’s micro-partitioning and clustering features. Optimize and tune Snowflake queries, Data Storage, Warehouse for performance and efficiency. Migration & Integration: Experience with data migration, particularly from SQL Server to Snowflake, and integrating Snowflake with other data sources. Collaborate with data architects and business stakeholders to understand data requirements. Develop and maintain documentation for data processes and solutions. Ensure data quality, governance, and security within the Snowflake environment. Troubleshoot and resolve issues related to the Snowflake data warehouse. Stay updated with the latest Snowflake features and best practices. API Integrations: Work on integrating Snowflake with external APIs for seamless data extraction and transfer. DevOps Integration - Secondary Azure DevOps Experience: Proficiency with Azure DevOps for: Source control management Automating deployments Deploying Snowflake scripts across different environments Managing ETL pipeline deployments Proficiency in DevOps tools and best practices, with experience deploying Snowflake and ETL tool services.