Pyspark Databricks AWS
Tasks
- Apply data governance
- Build scalable data pipelines
- Develop and maintain Delta Lake solutions
- Ensure data quality
- Implement CI/CD pipelines
- Integrate data from multiple sources
- Manage data access with Unity Catalog
- Optimize workflows
- Perform ETL transformations
- Tune data and pipeline performance
Perks/Benefits
- N/A
Skills/Tech-stack
AWS Glue | Agile | Airflow | Amazon Web Services | Apache Kafka | Apache Spark | CI/CD | Data Governance | Data Quality | Databricks | Delta Lake | Delta Live Tables | Delta Live) | ETL | Performance Tuning | PySpark | SQL | Scala | Unity Catalog | Web Services
Education
N/A
Roles
Big Data Engineer | Data Engineer | Databricks Developer | Developer | Engineer
Related jobs
-
Senior Principal Consultant, Gen_AI Engineer INR 2500K-4000KAWS | Agentic AI | Azure OpenAI | Cloud Native | Cloud platformSenior-level Full TimeIndia-Bangalore3h ago
-
Consultant_ GCP Data Engineer INR 1200K-2000KAirflow | BigQuery | Cloud Composer | Cloud Functions | Cloud StorageAI certifications | Training and mentorshipEntry-level Full TimeIndia-Gurugram3h ago
-
Lead Consultant, Gen_AI Engineer with MCP server expertise! INR 2500K-4000KAWS | Agentic AI | Amazon SageMaker | Azure | Azure OpenAISenior-level Full TimeIndia-Bangalore3h ago
-
Mid-level Full TimeIN-MH-Pune4h ago
-
AI/ML Engineer INR 2500K-4000KAzure Cognitive | Azure Cognitive Search | Azure Data | Azure Data Factory | Azure Data LakeSenior-level Full TimeIN-HR-Gurgaon4h ago
-
AIML Engineer INR 2500K-4000KAzure Data | Azure Data Factory | Azure Data Lake | Cognitive Search | Data FactorySenior-level Full TimeIN-HR-Gurgaon4h ago
-
Senior-level Full TimeIN-TN-Chennai4h ago
-
AI/ML-Data Scientist INR 2500K-3500KAzure Data | Azure Data Factory | Azure Data Lake | Azure OpenAI | Cognitive SearchSenior-level Full TimeIN-HR-Gurgaon4h ago
-
Mid-level Full TimePune, Maharashtra, India5h ago
-
AWS Lead Data Engineer INR 2000K-2040KAWS | AWS Glue | AWS architecture | Agile methodology | Amazon AuroraSenior-level Full TimeGurugram, Haryana, India5h ago
-
Senior-level Full TimeNew Delhi, DL, India6h ago
-
DE&A - Core - Cloud Data Engineering - Azure Data Engineering INR 1500K-2000KAggregation pipelines | Azure Blob | Azure Blob Storage | Azure Cosmos | Azure Cosmos DBMid-level Full TimePune, Maharashtra, India7h ago
-
Lead Software Engineer - AWS, PySpark, BigData INR 2000K-2450KAWS | AWS CDK | AWS Lambda | Agile | AirflowSenior-level Full TimeHyderabad, Telangana, India7h ago
-
API Gateway | AWS Bedrock | AWS Lambda | AWS Step Functions | Amazon APIEntry-level Full TimeMumbai, MH, India7h ago
-
Software Engineer III - AWS, PySpark, ETL INR 2000K-2450KAWS EMR | Agile | Apache Spark | CI/CD | Data ModelingSenior-level Full TimeHyderabad, Telangana, India7h ago
-
Software Engineer III - PySpark, BigData, AWS INR 2050K-2600KAWS | Agile | Amazon Web Services | Application Resiliency | Application SecuritySenior-level Full TimeHyderabad, Telangana, India7h ago
-
Senior Embedded DevOps Engineer INR 2229K-2500KAutomotive Software | Build Tools | CI/CD | Change Management | Cloud ComputingSenior-level Full TimeBangalore, IN, 5601008h ago
-
Consultant - Python Developer with Gen AI INR 1500K-2000KAI Search | Agile | Azure | Azure OpenAI | CSSMid-level Full TimeBangalore, Karnataka, India8h ago
-
Data Scientist INR 2500K-3000KAPI Integration | AWS | Asynchronous Architecture | Azure | CI/CDContinuous learning culture | Flexible work environment | Innovation cultureSenior-level Full TimeNoida, Uttar Pradesh, India8h ago
-
Azure Data Engineer - Associate Consultant INR 1500K-2000KAzure Data | Azure Data Factory | Azure Data Flow | Azure Data Lake | Azure DatabricksMid-level Full TimeBangalore, Karnataka, India9h ago
-
Data Engineer- AWS+MS Fabric+ Purview INR 1628K-2040K.NET | AWS | AWS CloudFormation | AWS Lambda | Amazon EC2Senior-level Full TimeGurugram, Haryana, India9h ago
-
Senior Data Analyst INR 2516K-2829KAgile | Business Intelligence | Data Modeling | Data integration | Database DesignEmployee assistance program | Flexible time off | Medical insurance plans | Paid parental leave | Vacation and holiday leaveSenior-level Full TimeCHANDIGARH, India10h ago
-
Data Engineer INR 960K-1400KADO | Automated testing | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimeMumbai, Maharashtra, India10h ago
-
Mid-level Full TimePune, India10h ago
-
Senior-level Full TimeChennai, TN, India10h ago