Cloud Data Engineer (Azure)

Cloud Data Engineer will be responsible for developing on the data lake platform and all applications on Azure cloud. Good data Engineering, data modeling is a must with Python programming background. The Data Engineer will be responsible for providing design and development solutions for applications in the cloud.

Essential Job Functions:

• Understand requirements and engage with team to design and deliver projects.
• Design and implement data lake house projects within azure.
• Design and develop application lifecycle utilizing Microsoft Azure technologies
• Participate in design and planning and necessary documentation
• Participate in Agile ceremonies including daily standups, scrum, retrospectives,demos, code reviews.
• Hands on with Python development and Azure data pipelines
• Engage with team to develop and deliver cross functional products
• Postgres SQL knowledge is a plus

Minimum Qualifications and Job Requirements:

• Bachelor's degree in CS.
• 7 years of hands-on experience in designing and developing distributed data pipelines.
• 5 years of hands-on experience in Azure data service technologies.
• 5 years of hands-on experience in Python, SQL, Object oriented programming, ETL and unit testing
• Experience with data integration with APIs, Web services, Queues
• Experience with Azure DevOps and CI/CD as well as agile tools and processes including JIRA, confluence.
• Working experience with writing SQL queries from scratch

Key Skills

a. Data Engineering and SQL

b. Python

c. PySpark

d. Azure Data lake and ADF

Other Responsibilities:

• Document and maintain project artifacts.
• Maintain comprehensive knowledge of industry standards, methodologies, processes, and best practices.
• Complete training as required for Privacy, Code of Conduct etc.
• Promptly report any known or suspected loss, theft or unauthorized disclosure or use of PI to the General Counsel/Chief Compliance Officer or Chief Information Officer.
• Adhere to the company's compliance program.
• Safeguard the company's intellectual property, information, and assets.