Sr. PySpark Developer Data Engineer
Tasks
- Apply data cleansing and data linking
- Assess data quality and validate data
- Build and maintain ETL workflows
- Build data pipelines in production
- Collaborate with teams on structured semi structured and unstructured data
- Communicate with technical and non-technical stakeholders
- Develop and execute testing and validation
- Develop streaming and batch data pipelines
- Implement machine learning pipelines
- Perform feature engineering and imputation
- Write clean, maintainable Python code
Perks/Benefits
Skills/Tech-stack
Anomaly Detection | Apache Airflow | Apache Hadoop | Apache Spark | CI/CD | Data Analysis | Data Cleansing | Data Quality | Exploratory Data Analysis | Feature Engineering | Git | GitHub Actions | Hive | Imputation Techniques | Jenkins | Jupyter | MapReduce | NoSQL | Pandas | PySpark | Python | SQL
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Data Engineer | Developer | Engineer | PySpark Developer | Senior Data Engineer
Related jobs
-
BI Developer - Snowflake & Salesforce Data INR 2000K-2500KAlerting | DBT | Data Quality | Data Validation | Data WarehousingChallenging projects | Great work-life balance | Professional developmentMid-level Full TimePune, MH, IN5h ago
-
Senior-level Full TimeMaharashtra, Mumbai, India6h ago
-
AWS | AWS Glue | AWS Lambda | Amazon Athena | Amazon EMRAI certifications | Ethical AI focus | Mentorship | World-class trainingSenior-level Full TimeIndia-Hyderabad7h ago
-
Manager, Tableau Developer INR 1400K-2000KAWS | Azure | CRM | Data Lakes | Data ModelingAI certifications | Mentorship | World-class trainingMid-level Full TimeIndia-Bangalore7h ago
-
Senior-level Full TimeMaharashtra, Mumbai, India7h ago
-
Senior-level Full TimeIN-TN-Chennai7h ago
-
Senior-level Full TimeIN-TN-Chennai7h ago
-
Senior-level Full TimeIN-TN-Chennai7h ago
-
ETL Test Engineer INR 1200K-1500KAgile | Amazon Redshift | Data Integrity | Data Transformation | Data ValidationMid-level Full TimeIN-TN-Chennai7h ago
-
AWS Cloud & DevOps Engineer INR 2156K-2541KAWS | Ansible | Configuration Management | DevOps | Infrastructure as CodeSenior-level Full TimeIN-AP-Hyderabad7h ago
-
Lead/Manager Software Engineer INR 2500K-3200KAgent Orchestration | Agentic AI | Amazon ECS | Amazon EMR | Amazon Elastic Kubernetes ServiceSenior-level Full TimeBengaluru, Karnataka, India8h ago
-
Sr.Lead Software Engineer INR 2500K-3500KAWS ECS | AWS EKS | AWS EMR | Agent Development | Amazon Web ServicesSenior-level Full TimeHyderabad, Telangana, India8h ago
-
Forward Deployed Engineer, Generative AI, Google Cloud INR 2500K-4500KAPI Integration | Agent systems | Cloud platform | Cost Per Request | CrewAISenior-level Full TimeGurgaon, Haryana, India; Bengaluru, Karnataka, India9h ago
-
DAX | Data Modeling | Data Visualization | Data Warehouse | Data integrationMid-level Full TimeGurgaon, Haryana, India9h ago
-
Mid-level Full TimePune, Maharashtra, India9h ago
-
Principal Data Operations Analyst - SQL & Data Management INR 1800K-2146KData Governance | Data Migration | Data Modeling | Data Quality | Data SecurityEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeGurgaon, HR, India10h ago
-
Senior-level Full TimeBangalore, Karnataka, India10h ago
-
Senior-level Full TimeBangalore, Karnataka, India10h ago
-
Mid-level Full TimeGurugram, Haryana, India10h ago
-
AWS | AWS Lambda | Agile | Apache Spark | Data ModelingSenior-level Full TimeBengaluru, Karnataka, India11h ago
-
None Full TimePune, Maharashtra, India12h ago
-
Generative AI Engineer INR 2500K-4500KAgentic Frameworks | Autogen | Embeddings | Fine Tuning | LanggraphSenior-level Full TimePune, Maharashtra, India12h ago
-
Software Engineer (6months to 2 Years of experience - JAVA/Python Backend, CICD, APIs, GEN AI, Microservices, Cloud) INR 2000K-2156KAgile | Algorithms | Apache Kafka | Automated testing | C++Continued learning opportunities | Hybrid work arrangement | MentorshipEntry-level Full TimeBengaluru, INDIA, India12h ago
-
Data Management AI Lead INR 2500K-3380KAWS | Agent Orchestration | Agent systems | Azure | Cloud DataSenior-level Full TimeBangalore, Karnataka, IN12h ago
-
API Integration | AWS | Azure | Cloud platform | Data PreprocessingMid-level Full TimeDelhi, India12h ago