Pyspark Data Engineer
Tasks
- Design data pipelines
- Develop ETL/ELT workflows
- Ensure data governance and compliance
- Implement data quality and validation
- Integrate data from multiple sources
- Optimize data processing jobs
- Participate in code reviews, testing, and deployment
- Troubleshoot pipeline issues
Perks/Benefits
- N/A
Skills/Tech-stack
Agile | Apache Spark | Data Governance | Data Lake | Data Modeling | Data Quality | Data Validation | Data Warehouse | Data integration | ELT | ETL | Hadoop | Informatica BDM | PySpark | Python | SQL | Workflow Orchestration
Education
N/A
Roles
Related jobs
-
Software Engineer (AI/DevOps) ID53179 INR 2829K-3500KCI/CD | Docker | GitHub | Incident Management | JavaEducation budget | Local team meetups | Professional growth budget | Remote work | Structured secure work environmentSenior-level Full TimeNoida, India4h ago
-
Software Engineer (Python, AI/ML) ID47296 INR 2535K-4225KCI/CD | Data Modeling | Git | Microservices | PytestLocal legal compliance | Periodic team meetups | Remote workSenior-level Full TimeTrivandrum, India4h ago
-
Software Engineer (Python, AI/ML) ID47296 INR 2535K-4225KCI/CD | Clean Code | Data Modeling | Git | MicroservicesEducation support | Innovative projects | Legal compliance in India | Local team meetups | Professional development budgetSenior-level Full TimeChennai, India4h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2829K-3500KAI Agents | AI Assisted Development | CI/CD | Docker | GitHub ActionsEducation budget | Innovative projects | Legal Compliance Support | Local team meetups | Personal growth budgetSenior-level Full TimeKochi, India4h ago
-
Software Engineer (Python, AI/ML) ID47296 INR 2535K-4225KCI/CD | Cloud Native | Cloud native design | Data Modeling | Engineering PracticesEducation budget | Local compliance | Periodic team meetups | Personal growth budget | Remote workSenior-level Full TimeKochi, India4h ago
-
Cloud deployment | FastAPI | GenAI | PostgreSQL | Power BIMid-level Full TimeHyderabad, India4h ago
-
Lead Consultant-SSIS Developer! INR 2500K-3380KData Warehousing | Data integration | Design Patterns | Dimensional Modeling | ETLAI certifications | Mentorship | TrainingSenior-level Full TimeIndia-Bangalore5h ago
-
Lead Consultant - Site Reliability Engineers/L2 INR 2535K-4225KAWS | Angular | Azure | C# | Chaos EngineeringSenior-level Full TimeIndia-Bangalore5h ago
-
Senior-level Full TimeIN-AP-Hyderabad5h ago
-
002AZM - Data Integration Engineer INR 2000K-2040KAWS | AWS Glue | Amazon Redshift | Amazon S3 | Apache AirflowContract work | Remote workSenior-level ContractTamil Nadu, Coimbatore, India5h ago
-
Gen AI - Consultant INR 2800K-4225KAPI Development | AWS SageMaker | AgentOps | Apache Airflow | Apache SparkAccess to AI research labs | Certifications | Cloud innovation platforms | Lead POCs | Work on next generation AI platformsSenior-level Full TimeBangalore, Karnataka, India6h ago
-
Azure Data Engineer - AM- MFT KGS CH INR 2000K-2000KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksSenior-level Full TimeBangalore, Karnataka, India6h ago
-
Azure Data Engineer - Senior- MFT KGS CH INR 2000K-2000KAccess Control | Azure Data | Azure Data Factory | Azure Data Lake | Azure DatabricksSenior-level Full TimeBangalore, Karnataka, India6h ago
-
Azure Data Engineer - AM - MFT KGS CH INR 2000K-2000KAgile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeBangalore, Karnataka, India6h ago
-
Azure Platform Operations Eng - Senior- MFT KGS CH INR 2000K-2125KAlerting | Azure Data | Azure Data Factory | Azure Data Lake | Azure MonitorSenior-level Full TimeBangalore, Karnataka, India6h ago
-
AWS Data Engineering - Consultant INR 1200K-1500KAWS CloudFormation | AWS Glue | AWS Lambda | AWS Step Functions | Amazon AthenaMid-level Full TimeBangalore, Karnataka, India6h ago
-
Assistant Manager - Azure Data Engineering INR 1500K-3000KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOpsMid-level Full TimeBangalore, Karnataka, India6h ago
-
Assistant Manager - Azure Data Engineering INR 1500K-3000KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOpsMid-level Full TimeBangalore, Karnataka, India6h ago
-
AWS Data Engineering - Consultant INR 1200K-1500KAWS CloudFormation | AWS Glue | AWS Lambda | AWS Step Functions | Amazon AthenaMid-level Full TimeBangalore, Karnataka, India7h ago
-
Senior Software Engineer, Google Cloud Storage INR 2500K-3500KBenchmarking | C++ | Connectors | Debugging | Distributed SystemsSenior-level Full TimeHyderabad, Telangana, India7h ago
-
Senior Manager-Business Analysis-Business Analyst INR 1800K-2800KCSI | Data Analysis | Data Processing | Deep learning | Distributed ComputingSenior-level Full TimeGurugram, Haryana, India8h ago
-
Senior BI Engineer #3033 INR 800K-1500KCalculated Fields | DBT | Dashboard performance | Dashboard performance tuning | Data LineageSenior-level Full TimeChennai, India9h ago
-
Senior AI/ML Engineer I INR 2500K-4500KAzure | C++ | Dataset Processing | Deep learning | Generative AICareer advancement programs | Corporate National Pension Scheme | Dental insurance | Education reimbursement programs | Employee assistance programSenior-level Full TimeBangalore, Karnataka, India9h ago
-
Senior Software Engineer, Data Science INR 3125K-3880KA/B | A/B Testing | B testing | Causal analysis | D3Senior-level Full TimeBengaluru, KA, India10h ago
-
Staff Software Engineer, Data Science INR 2520K-3380KApache Spark | D3 | Data Governance | Data Modeling | Data VisualizationSenior-level Full TimeBengaluru, KA, India10h ago