Python Pyspark Data Engineer

Job Description:

Python Pyspark Data Engineer

Job Location: Hyderabad / Bangalore / Chennai / Kolkata / Noida/ Gurgaon / Pune / Indore / Mumbai

We are seeking a skilled Lead Data Engineer with strong programming and SQL skills to join our team. The ideal candidate will have hands-on experience with Python and Pyspark Data Analytics services and a basic understanding of general AWS services.

Key Responsibilities:

• Design, develop, and optimize data pipelines using Python, Pyspark, AWS Data Analytics services such as RDS, DMS, Glue, Lambda, Redshift, and Athena.
• Implement data migration and transformation processes using AWS DMS and Glue.
• Work with SQL (Oracle & Postgres) to query, manipulate, and analyse large datasets.
• Develop and maintain ETL/ELT workflows for data ingestion and transformation.
• Utilize AWS services like S3, IAM, CloudWatch, and VPC to ensure secure and efficient data operations.
• Write clean and efficient Python scripts for automation and data processing.
• Collaborate with DevOps teams using Azure DevOps for CI/CD pipelines and infrastructure management.
• Monitor and troubleshoot data workflows to ensure high availability and performance.

At DXC Technology, we believe strong connections and community are key to our success. Our work model prioritizes in-person collaboration while offering flexibility to support wellbeing, productivity, individual work styles, and life circumstances. We’re committed to fostering an inclusive environment where everyone can thrive.

Recruitment fraud is a scheme in which fictitious job opportunities are offered to job seekers typically through online services, such as false websites, or through unsolicited emails claiming to be from the company. These emails may request recipients to provide personal information or to make payments as part of their illegitimate recruiting process. DXC does not make offers of employment via social media networks and DXC never asks for any money or payments from applicants at any point in the recruitment process, nor ask a job seeker to purchase IT or other equipment on our behalf. More information on employment scams is available here.