Sr. PySpark Developer Data Engineer
Tasks
- Apply data cleansing and data linking
- Assess data quality and validate data
- Build and maintain ETL workflows
- Build data pipelines in production
- Collaborate with teams on structured semi structured and unstructured data
- Communicate with technical and non-technical stakeholders
- Develop and execute testing and validation
- Develop streaming and batch data pipelines
- Implement machine learning pipelines
- Perform feature engineering and imputation
- Write clean, maintainable Python code
Perks/Benefits
Skills/Tech-stack
Anomaly Detection | Apache Airflow | Apache Hadoop | Apache Spark | CI/CD | Data Analysis | Data Cleansing | Data Quality | Exploratory Data Analysis | Feature Engineering | Git | GitHub Actions | Hive | Imputation Techniques | Jenkins | Jupyter | MapReduce | NoSQL | Pandas | PySpark | Python | SQL
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Data Engineer | Developer | Engineer | PySpark Developer | Senior Data Engineer
Related jobs
-
Convolutional Neural Networks | Deep learning | Generative AI | Google Gemini | LLM PromptsFlexibility programs | Inclusive benefits | Mentorship | Wellbeing supportMid-level Full TimeKolkata DN 57, India-1d ago
-
API | Amazon Web Services | Apache Airflow | Apache Hadoop | Azure DataFlexibility programs | Inclusive benefits | MentorshipMid-level Full TimeKolkata DN 57, India-1d ago
-
Apache Airflow | Apache Hadoop | Apache Spark | App Service | Azure AppFlexibility programmes | Inclusive benefits | MentorshipSenior-level Full TimeKolkata DN 57, India-1d ago
-
Apache Airflow | Apache Spark | Azure Cosmos | Azure Cosmos DB | Azure DataSenior-level Full TimeKolkata DN 57, India-1d ago
-
Apache Airflow | Apache Hadoop | Apache Kafka | Apache Spark | App ServiceSenior-level Full TimeKolkata DN 57, India-1d ago
-
IN_Manager_Data Analyst_Data and Analytics_Advisory_Bangalore INR 1500K-2000KApache Airflow | Apache Flink | Azure | Azure DevOps | CI/CDMid-level Full TimeBengaluru Millenia, India-1d ago
-
API Integration | Agile | Apache Airflow | Apache Hadoop | Apache KafkaFlexibility programs | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeKolkata DN 57, India-1d ago
-
Amazon Web Services | Apache Airflow | Apache Databricks | Apache Hadoop | Apache SparkSenior-level Full TimeKolkata DN 57, India-1d ago
-
API ingestion | Agile | Apache Airflow | Apache Hadoop | Apache KafkaFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeKolkata DN 57, India-1d ago
-
Ab Initio | Apache Airflow | BigQuery | Cloud Storage | Cloud platformFlexible work arrangements | Inclusive workplace culture | MentorshipSenior-level Full TimeBengaluru Millenia, India-1d ago
-
Ab Initio | BigQuery | Cloud Composer | Cloud Storage | Cloud platformFlexible work programs | Inclusive workplace culture | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India-1d ago
-
Ab Initio | Apache Airflow | BigQuery | Cloud Storage | Cloud platformFlexible work arrangements | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India-1d ago
-
Ab Initio | Apache Dataflow | BigQuery | Cloud Composer | Cloud StorageCompetitive compensation | Flexibility programmes | Hybrid work environment | Inclusive benefits | MentorshipSenior-level Full TimeBengaluru Millenia, India-1d ago
-
BI Developer - Snowflake & Salesforce Data INR 2000K-2500KAlerting | DBT | Data Quality | Data Validation | Data WarehousingChallenging projects | Great work-life balance | Professional developmentMid-level Full TimePune, MH, IN7h ago
-
Senior-level Full TimeMaharashtra, Mumbai, India7h ago
-
AWS | AWS Glue | AWS Lambda | Amazon Athena | Amazon EMRAI certifications | Ethical AI focus | Mentorship | World-class trainingSenior-level Full TimeIndia-Hyderabad8h ago
-
Manager, Tableau Developer INR 1400K-2000KAWS | Azure | CRM | Data Lakes | Data ModelingAI certifications | Mentorship | World-class trainingMid-level Full TimeIndia-Bangalore8h ago
-
Senior-level Full TimeMaharashtra, Mumbai, India9h ago
-
Senior-level Full TimeIN-TN-Chennai9h ago
-
Senior-level Full TimeIN-TN-Chennai9h ago
-
Senior-level Full TimeIN-TN-Chennai9h ago
-
ETL Test Engineer INR 1200K-1500KAgile | Amazon Redshift | Data Integrity | Data Transformation | Data ValidationMid-level Full TimeIN-TN-Chennai9h ago
-
AWS Cloud & DevOps Engineer INR 2156K-2541KAWS | Ansible | Configuration Management | DevOps | Infrastructure as CodeSenior-level Full TimeIN-AP-Hyderabad9h ago
-
Lead/Manager Software Engineer INR 2500K-3200KAgent Orchestration | Agentic AI | Amazon ECS | Amazon EMR | Amazon Elastic Kubernetes ServiceSenior-level Full TimeBengaluru, Karnataka, India10h ago
-
Sr.Lead Software Engineer INR 2500K-3500KAWS ECS | AWS EKS | AWS EMR | Agent Development | Amazon Web ServicesSenior-level Full TimeHyderabad, Telangana, India10h ago