Data Engineer, II
Tasks
- Automate data pipelines
- Build data pipeline workflows
- Cleanse data
- Conduct performance testing
- Create data validation tests
- Design data pipelines
- Develop PySpark data transformations
- Maintain technical documentation
- Minimize pipeline cost
- Optimize pipeline performance
- Perform code profiling
- Transform data
Perks/Benefits
Skills/Tech-stack
AWS | Agile | Airflow | Apache Airflow | Azure | Dask | Data Validation | Data profiling | Databricks | GCP | Google Cloud | Linux | MongoDB | MySQL | NoSQL | Performance Testing | PostgreSQL | PySpark | Python | Redis | Relational databases | SQL | Spark
Education
Bachelor of Engineering | Bachelor of Science | Master of Science | PhD
Roles
Related jobs
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | CI/CDLocal connection | Periodic meet ups | Remote work | Structured secure work environment | Support for education and wellnessSenior-level Full TimeJalalabad, India9h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | Cloud Composer | Cloud Dataproc | Cloud MonitoringBudgets for personal growth education and wellness | Legal Compliance Support | Local connection meetups | Remote workSenior-level Full TimeMumbai, India9h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | Bitemporal data modeling | CI/CD | Change Data CaptureDedicated budgets for education | Dedicated budgets for wellness | Innovative projects | Local connection | Periodic meetupsSenior-level Full TimeChennai, India9h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | BigQuery | Change Data Capture | ClickHouseBudget for education | Budget for wellness | Local compliance in India | Periodic team meetups | Personal growth supportSenior-level Full TimeGurgaon Kty., India9h ago
-
Data Engineer ID56374 INR 1500K-2300KAnomaly Detection | Azure Data | Azure Data Factory | Azure Key Vault | Azure SQLEducation budget | Innovative projects | Local connection | Periodic meet ups | Personal growth budgetSenior-level Full TimeCoimbatore South, India9h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | BigQuery | Bitemporal data modeling | CI/CDEducation budget | Innovative projects | Local connection | Periodic meet ups | Personal growth budgetSenior-level Full TimeJalalabad, India9h ago
-
Data Engineer ID56374 INR 1500K-2300KAgentic AI | Azure Data | Azure Data Factory | Azure Key Vault | Azure SQLEducation budget | Growth budget | Local meetups | Remote work | Wellness budgetSenior-level Full TimeGurgaon Kty., India9h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | BigQuery | CI/CD | Change Data CaptureEducation budget | Innovative projects | Local compliance | Local connection | Periodic meetupsSenior-level Full TimeAhmedabad, India9h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure | Azure Data | Azure Data Factory | Azure Key Vault | Azure SQLEducation budget | Innovative projects | Local connection | Periodic meetups | Personal growth budgetSenior-level Full TimeChennai, India9h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | CI/CD | Change Data Capture | Change Data Capture (CDC)Education budget | Local compliance in India | Periodic team meetups | Personal growth budget | Remote workSenior-level Full TimeNew Delhi, India9h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | BigQuery | CI/CD | Change Data CaptureEducation budget | Local compliance support | Periodic team meetups | Personal growth budget | Remote workSenior-level Full TimeKochi, India9h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | CI/CD | Change Data Capture | ClickHouseEducation budget | Innovative projects | Legal presence in India compliance | Local connection meetups | Personal growth budgetSenior-level Full TimeJaipur, India9h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | CD pipelinesEducation budget | Innovative projects | Local connection periodic meetups | Personal growth budget | Remote workSenior-level Full TimeTrivandrum, India9h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | BigQuery | CI/CD | Change Data CaptureEducation budget | Local connection meetups | Personal growth budget | Remote work | Structured secure work environmentSenior-level Full TimeIndore, India9h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | Apache pandas | BigQuery | CI/CDInnovative projects | Legal compliance in India | Local connection meetups | Personal growth education and wellness budget | Remote workSenior-level Full TimeKolkata, India9h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | CI/CDGrowth education budget | Local connection meetups | Remote work | Secure compliant work environment | Wellness budgetSenior-level Full TimeKolkata, India9h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure | Azure Data | Azure Data Factory | Azure Key Vault | Azure SQLEducation budget | Innovative projects | Local connection meetups | Personal growth budget | Remote workSenior-level Full TimeAhmedabad, India9h ago
-
Data Engineer ID56374 INR 1500K-2300KAgentic AI | Azure Data | Azure Data Factory | Azure Key Vault | Azure SQLEducation budget | Innovative projects | Legal presence in India | Local connection | Periodic team meetupsSenior-level Full TimeJaipur, India9h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | CI/CDEducation budget | Local connection meetups | Professional growth budget | Remote work | Wellness budgetSenior-level Full TimeKochi, India9h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | CI/CD | Cloud Composer | Cloud MonitoringEducation budget | Local compliance support | Local meetups | Personal growth budget | Remote workSenior-level Full TimeTrivandrum, India9h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | Bitemporal data modelingEducation budget | Legal Compliance Support | Local connection | Meetups | Professional growth budgetSenior-level Full TimeNew Delhi, India9h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | Bitemporal data modelingEducation budget | Innovative projects | Local connection meetups | Personal growth budget | Remote workSenior-level Full TimeMumbai, India9h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | CI/CDEducation budget | Innovation projects | Local connection meetups | Personal growth budget | Remote workSenior-level Full TimeHyderabad, India9h ago
-
Data Engineer ID56374 INR 1500K-2300KAnomaly Detection | Azure Data | Azure Data Factory | Azure Key Vault | Azure SQLEducation budget | Local connection | Meetups | Personal growth budget | Remote workSenior-level Full TimePune City, India9h ago
-
Data Engineer ID56374 INR 1500K-2300KAzure Data | Azure Data Factory | Azure Key Vault | Azure SQL | CI/CDEducation budget | Innovative projects | Local connection | Periodic meetups | Personal growth budgetSenior-level Full TimeBangalore North, India9h ago