DE&A - Core - Big Data Engineering - ETL
Bangalore, Karnataka, India
INR 1500K-2000K (estimate) Mid-level Full Time
Tasks
- Architect and develop PySpark applications
- Define migration strategies
- Design target state data architectures
- Implement batch and streaming workloads on GCP
- Lead big data platform migrations to GCP
- Modernize ETL pipelines
- Optimize data pipelines for performance, scalability, and cost
- Refactor Spark Hive and MapReduce jobs for GCP
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Hadoop | Apache Hive | Apache Spark | Batch Processing | Cloud Dataflow | Cloud Dataproc | Cloud Storage | Data Migration | ETL | GCP | Google BigQuery | Google Cloud | Google Cloud Dataflow | Google Cloud Dataproc | Google Cloud Storage | MapReduce | PySpark | Python | Streaming processing
Education
N/A
Related jobs
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KAI Agents | AI Assisted Development | CI/CD | Docker | GitHub ActionsBudget for education | Budget for wellness | Local connection meetups | Remote workSenior-level Full TimeChennai, India7h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KCI/CD | Docker | GitHub Actions | Java | KubernetesLocal team connection | Remote workSenior-level Full TimePune City, India7h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KAutomated testing | CI/CD | Docker | GitHub Actions | JavaEducation budget | Local team meetups | Personal growth budget | Remote work | Secure local complianceSenior-level Full TimeAhmedabad, India7h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KAI Agents | CI/CD | Cloud Native | Docker | GitHubEducation budget | Local team connection | Periodic meetups | Personal growth budget | Remote workSenior-level Full TimeJalalabad, India7h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KCI/CD | Docker | GitHub Actions | Java | JiraEducation budget | Local meetups | Personal growth budget | Remote work | Wellness budgetSenior-level Full TimeKolkata, India7h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KCI/CD | Docker | GitHub Actions | Java | JavaScriptEducation budget | Local compliance | Local connection | Periodic meet ups | Remote workSenior-level Full TimeNoida, India7h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KAI Agents | CI/CD | Docker | GitHub Actions | JavaEducation budget | Local office connectivity | Periodic team meetups | Personal growth budget | Remote workSenior-level Full TimeJaipur, India7h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KCI/CD | Docker | GitHub Actions | Java | JiraEducation budget | Legal compliance support for India | Personal growth budget | Remote work | Team meetupsSenior-level Full TimeTrivandrum, India7h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KAI-assisted coding | Assisted coding | CI/CD | Docker | GitHubEducation budget | Local meet-ups | Professional development budget | Remote work | Wellness budgetSenior-level Full TimeHyderabad, India7h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KCI/CD | Docker | GitHub Actions | Java | KubernetesEducation budget | Local connection | Personal growth budget | Remote work | Wellness budgetSenior-level Full TimeKochi, India7h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KCI/CD | Cloud Native | Docker | GitHub Actions | JavaEducation budget | Local connection | Personal growth budget | Remote work | Wellness budgetSenior-level Full TimeIndore, India7h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KCI/CD | Docker | GitHub | Java | JiraLocal connection meetups | Remote workSenior-level Full TimeCoimbatore South, India7h ago
-
HR TECHNOLOGY - HR Analytics INR 1200K-1500KAI/ML | Artificial Intelligence | Machine Learning | Microsoft Excel | Power BIMid-level Full TimePune, Maharashtra, India9h ago
-
Data Science AI Engineer, Senior Professional INR 2500K-4500KAPI Development | AWS | Agentic AI | Apache Spark | AzureFully remoteSenior-level Full TimeBangalore, KA, IN, 560100 R13h ago
-
Manufacturing Data Analytics Technical Expert INR 1628K-2146KApache Hive | Apache Spark | Data Historian | Data analytics | Data integrationSenior-level Full TimeBangalore, Karnataka, IN, 56010014h ago
-
Agile | Apache Spark | Apache Superset | Big Data | Cloud ArchitectureAnnual holiday leave | Biannual in person events | Career growth opportunities | Employee assistance program | Flexible working arrangementsMid-level Full TimeIndia R1d ago
-
Mid-level Full TimeIndia1d ago
-
Lead Assistant Manager INR 500K-1500KAWS | Apache Airflow | Cloud platform | Data Governance | Data LakehouseSenior-level Full TimeIndia1d ago
-
Senior Data Engineer INR 1300K-2000KAPI Integration | API Security | Agile | Apigee | Asset ManagementSenior-level Full TimeChennai, Tamil Nadu, India1d ago
-
Senior Consultant INR 1800K-2200KAmazon Aurora | Amazon Aurora PostgreSQL | Amazon Relational Database Service | Amazon Web Services | Aurora PostgreSQLSenior-level Full TimeIN-AP-Hyderabad1d ago
-
Associate, Data Analytics & AI INR 1000K-1800KAmazon SageMaker | Anthropic API | Azure Machine Learning | CRM | CrewAIWellness Fridays | Wellness resources | Work-life balance supportMid-level Full TimePune, Maharashtra, India R1d ago
-
Senior-level Full TimeIN-AP-Hyderabad1d ago
-
Data Analytics-Python Developer INR 2000K-3000KAgile | Dagster | Data Modeling | Dimensional Modeling | DockerMid-level Full TimeBengaluru, Karnataka, India1d ago
-
Senior-level Full TimeHybrid, Hyderabad, Telangana, India R1d ago
-
Senior AI Engineer INR 2000K-3800KAWS | Agent Orchestration | Amazon Bedrock | Artificial Intelligence | Confidence scoringComprehensive health benefits | Flexible work environment | Generous time off | Global days of service | Hybrid work modelSenior-level Full TimeBengaluru, Karnataka, India1d ago