Data Engg - Sr Technical Lead

About Birlasoft:
Birlasoft, a global leader at the forefront of Cloud, AI, and Digital technologies, seamlessly blends domain expertise with enterprise solutions. The company’s consultative and design-thinking approach empowers societies worldwide, enhancing the efficiency and productivity of businesses. As part of the multibillion-dollar diversified CKA Birla Group, Birlasoft with its 12,000+ professionals, is committed to continuing the Group’s 170-year heritage of building sustainable communities.

Title: Data Engineer

Key Responsibilities:
•    Experience building maintainable, testable, and scalable data pipelines for big data using PySpark according to best practices
•    Deep understanding of Spark's execution engine for processing billion of records
•    Strong skills in performance optimization as well as experience configuring Spark clusters for efficient resource usage at scale.
•    Advanced Python skills
•    Basic knowledge in SQL
•    Basic understanding of Git and Git workflow
•    Collaborate with business teams to gather requirements and translate them into technical solutions.
•    Build and optimize data models and workflows for performance and scalability.
•    Ensure data quality, integrity, and governance across all solutions.
•    Create clear and concise documentation and deliver presentations to stakeholders.
•    Work independently and take ownership of projects from inception to deployment.

Required Skills:
•    Strong experience with PySpark and Python.
•    Solid understanding of data engineering principles and ETL processes.
•    Experience in working with large-scale data systems.
•    Excellent communication, documentation, and presentation skills.
•    Ability to work independently and manage multiple priorities.

Nice to Have:
•    Familiarity with Palantir Foundry or similar platforms.
•    Experience with cloud platforms (AWS, Azure, GCP).
•    Knowledge of data warehousing and SQL.