PySpark Data Engineer with Cloudera and Cloud Expertise
Bengaluru - BCIT, India
INR 2400K-3500K (estimate) Senior-level Full Time
Tasks
- Automate workflow orchestration
- Design data pipelines
- Develop PySpark transformations
- Implement data quality checks
- Maintain data governance documentation
- Manage data ingestion
- Monitor and troubleshoot pipelines
- Optimize spark job performance
- Support security and compliance
Perks/Benefits
Skills/Tech-stack
AWS EMR | Apache Airflow | Apache Atlas | Apache Oozie | Apache Spark | Azure HDInsight | Bash | CI/CD | Cloud Dataproc | Cloudera Data Platform | Collibra | Data platform | GitLab CI | Google Cloud | Google Cloud Dataproc | HBase | HDFS | Hadoop | Hive | Impala | Jenkins | Kafka | Linux | PySpark | Python | SQL
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | CI/CDLocal connection | Periodic meet ups | Remote work | Structured secure work environment | Support for education and wellnessSenior-level Full TimeJalalabad, India5h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | Cloud Composer | Cloud Dataproc | Cloud MonitoringBudgets for personal growth education and wellness | Legal Compliance Support | Local connection meetups | Remote workSenior-level Full TimeMumbai, India5h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | Bitemporal data modeling | CI/CD | Change Data CaptureDedicated budgets for education | Dedicated budgets for wellness | Innovative projects | Local connection | Periodic meetupsSenior-level Full TimeChennai, India5h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | BigQuery | Change Data Capture | ClickHouseBudget for education | Budget for wellness | Local compliance in India | Periodic team meetups | Personal growth supportSenior-level Full TimeGurgaon Kty., India5h ago
-
Data Engineer ID56374 INR 1500K-2300KAnomaly Detection | Azure Data | Azure Data Factory | Azure Key Vault | Azure SQLEducation budget | Innovative projects | Local connection | Periodic meet ups | Personal growth budgetSenior-level Full TimeCoimbatore South, India5h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | BigQuery | Bitemporal data modeling | CI/CDEducation budget | Innovative projects | Local connection | Periodic meet ups | Personal growth budgetSenior-level Full TimeJalalabad, India5h ago
-
Data Engineer ID56374 INR 1500K-2300KAgentic AI | Azure Data | Azure Data Factory | Azure Key Vault | Azure SQLEducation budget | Growth budget | Local meetups | Remote work | Wellness budgetSenior-level Full TimeGurgaon Kty., India5h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | BigQuery | CI/CD | Change Data CaptureEducation budget | Innovative projects | Local compliance | Local connection | Periodic meetupsSenior-level Full TimeAhmedabad, India5h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure | Azure Data | Azure Data Factory | Azure Key Vault | Azure SQLEducation budget | Innovative projects | Local connection | Periodic meetups | Personal growth budgetSenior-level Full TimeChennai, India5h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | CI/CD | Change Data Capture | Change Data Capture (CDC)Education budget | Local compliance in India | Periodic team meetups | Personal growth budget | Remote workSenior-level Full TimeNew Delhi, India5h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | BigQuery | CI/CD | Change Data CaptureEducation budget | Local compliance support | Periodic team meetups | Personal growth budget | Remote workSenior-level Full TimeKochi, India5h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | CI/CD | Change Data Capture | ClickHouseEducation budget | Innovative projects | Legal presence in India compliance | Local connection meetups | Personal growth budgetSenior-level Full TimeJaipur, India5h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | CD pipelinesEducation budget | Innovative projects | Local connection periodic meetups | Personal growth budget | Remote workSenior-level Full TimeTrivandrum, India5h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | BigQuery | CI/CD | Change Data CaptureEducation budget | Local connection meetups | Personal growth budget | Remote work | Structured secure work environmentSenior-level Full TimeIndore, India5h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | Apache pandas | BigQuery | CI/CDInnovative projects | Legal compliance in India | Local connection meetups | Personal growth education and wellness budget | Remote workSenior-level Full TimeKolkata, India5h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | CI/CDGrowth education budget | Local connection meetups | Remote work | Secure compliant work environment | Wellness budgetSenior-level Full TimeKolkata, India5h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure | Azure Data | Azure Data Factory | Azure Key Vault | Azure SQLEducation budget | Innovative projects | Local connection meetups | Personal growth budget | Remote workSenior-level Full TimeAhmedabad, India5h ago
-
Data Engineer ID56374 INR 1500K-2300KAgentic AI | Azure Data | Azure Data Factory | Azure Key Vault | Azure SQLEducation budget | Innovative projects | Legal presence in India | Local connection | Periodic team meetupsSenior-level Full TimeJaipur, India5h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | CI/CDEducation budget | Local connection meetups | Professional growth budget | Remote work | Wellness budgetSenior-level Full TimeKochi, India5h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | CI/CD | Cloud Composer | Cloud MonitoringEducation budget | Local compliance support | Local meetups | Personal growth budget | Remote workSenior-level Full TimeTrivandrum, India5h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | Bitemporal data modelingEducation budget | Legal Compliance Support | Local connection | Meetups | Professional growth budgetSenior-level Full TimeNew Delhi, India5h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | Bitemporal data modelingEducation budget | Innovative projects | Local connection meetups | Personal growth budget | Remote workSenior-level Full TimeMumbai, India5h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | CI/CDEducation budget | Innovation projects | Local connection meetups | Personal growth budget | Remote workSenior-level Full TimeHyderabad, India5h ago
-
Data Engineer ID56374 INR 1500K-2300KAnomaly Detection | Azure Data | Azure Data Factory | Azure Key Vault | Azure SQLEducation budget | Local connection | Meetups | Personal growth budget | Remote workSenior-level Full TimePune City, India5h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | CI/CDEducation budget | Innovative projects | Local connection | Periodic meetups | Personal growth budgetSenior-level Full TimeBangalore North, India5h ago