Lead Data Engineer - Azure Data Lake Job

YASH Technologies is a leading technology integrator specializing in helping clients reimagine operating models, enhance competitiveness, optimize costs, foster exceptional stakeholder experiences, and drive business transformation.

At YASH, we’re a cluster of the brightest stars working with cutting-edge technologies. Our purpose is anchored in a single truth – bringing real positive changes in an increasingly virtual world and it drives us beyond generational gaps and disruptions of the future.

We are looking forward to hire PySpark Professionals in the following areas :

Experience

8+ years

Job Description

8+ years of professional experience in development and support
4+ years of experience in hands on work on PySpark 
Areas of expertise include Big Data Technologies like Hadoop, Spark, Impala, HDFS, Hive, Sqoop, Yarn, Java, Oracle database.
Good knowledge on Cloud technologies like Azure and AWS
Worked on Azure data factory, blobs, Azure SQL, and Azure Data Lake Storage.
Worked with Data frames and Datasets in Spark for Transforming data
Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform, and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse
Migrating and recreating existing application logic and functionality in the Azure Data Lake, Data Factory, SQL Database and SQL Datawarehouse environment
A logical, analytical thinker with excellent team player skills and possesses good experience as a Data Engineer
Excellent interpersonal skills, confident and poised in interaction with individuals at all levels
Focus on following a structured approach and Quality check methodology to generate error free deliverables
 

Required Technical/ Functional Competencies

• Technical Skills: Spark, Impala, Hadoop, HDFS, Hive, Sqoop, AWS, Azure, Databricks

Functional Skills: Banking and Finance, Insurance

Required Behavioral Competencies

Accountability: Takes responsibility for and ensures accuracy of own work, as well as the work and deadlines of the team.

Collaboration: Shares information within team, participates in team activities, asks questions to understand other points of view.

Agility: Demonstrates readiness for change, asking questions and determining how changes could impact own work.

Customer Focus: Identifies trends and patterns emerging from customer preferences and works towards customizing/ refining existing services to exceed customer needs and expectations.

Communication: Targets communications for the appropriate audience, clearly articulating and presenting his/her position or decision.

Drives Results: Sets realistic stretch goals for self & others to achieve and exceed defined goals/targets.

Resolves Conflict: Displays sensitivity in interactions and strives to understand others’ views and concerns.

Certifications

At YASH, you are empowered to create a career that will take you to where you want to go while working in an inclusive team environment. We leverage career-oriented skilling models and optimize our collective intelligence aided with technology for continuous learning, unlearning, and relearning at a rapid pace and scale.

Our Hyperlearning workplace is grounded upon four principles

• Flexible work arrangements, Free spirit, and emotional positivity
• Agile self-determination, trust, transparency, and open collaboration
• All Support needed for the realization of business goals,
• Stable employment with a great atmosphere and ethical corporate culture