DE&A - Core - Big Data Engineering - ETL
Bangalore, Karnataka, India
INR 1500K-2000K (estimate) Mid-level Full Time
Tasks
- Architect and develop PySpark applications
- Define migration strategies
- Design target state data architectures
- Implement batch and streaming workloads on GCP
- Lead big data platform migrations to GCP
- Modernize ETL pipelines
- Optimize data pipelines for performance, scalability, and cost
- Refactor Spark Hive and MapReduce jobs for GCP
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Hadoop | Apache Hive | Apache Spark | Batch Processing | Cloud Dataflow | Cloud Dataproc | Cloud Storage | Data Migration | ETL | GCP | Google BigQuery | Google Cloud | Google Cloud Dataflow | Google Cloud Dataproc | Google Cloud Storage | MapReduce | PySpark | Python | Streaming processing
Education
N/A
Related jobs
-
Senior Data Engineer INR 2800K-4000KAWS Elastic MapReduce | Agile | Apache Spark | Business Intelligence | Data WarehousingAnnual health check eye checkup dental screening OPD reimbursement | Employee assistance program | Free transportation | Leave policy | Medical insuranceSenior-level Full TimeBengaluru, Karnataka, India4h ago
-
Custom Software Engineer INR 1400K-1848KAirflow | CI/CD | Cause analysis | Change Management | Control-MRotational shifts | Stakeholder communicationMid-level Full TimeBengaluru, BDC7A, India9h ago
-
API Gateway | Agile | Apache Kafka | Artifactory | CICSSenior-level Full TimeTOWER B, EON FREE ZONE II, …9h ago
-
Mid-level Full TimePLOT NO-1, S.NO. 77, India9h ago
-
.NET | ASP.NET | Amazon Web Services | Angular | Apache AirflowFlexible work arrangements | Inclusive benefits | MentorshipSenior-level Full TimeKolkata DN 57, India9h ago
-
IN_Senior Associate_SCM_Data and Analytics_Advisory_Bangalore INR 2040K-3380KAgile | Amazon Web Services | Apache Airflow | Apache Hadoop | Azure DataFlexibility programs | Inclusive benefits | MentorshipSenior-level Full TimeBengaluru Millenia, India9h ago
-
IN_Associate_Palantir_ Data and Analytics_Advisory_Bangalore INR 3000K-4000KAnalytics Cloud | Business Intelligence | Data Analysis | Data Mining | Data ModelingSenior-level Full TimeBengaluru Millenia, India9h ago
-
Gen AI Engineer / Applied AI Engineer INR 1800K-2500KAI Agents | Cloud Run | Embeddings | Gemini | Generative AIHands on experience with Vertex AI and Gemini ecosystem | Opportunity to work on production systems | Work on challenging problemsMid-level Full TimeGurugram, Haryana, India9h ago
-
Senior PySpark Data Engineer INR 2000K-2800KApache Airflow | CI/CD | Cloudera | Data Modeling | Data WarehousingSenior-level Full TimePLOT NO-1, S.NO. 77, India9h ago
-
Sr. AI engineer (Remote) USD 50K-120KAWS | AWS CloudWatch | AWS SageMaker | Apache Spark | CI/CDRemote workSenior-level Full TimeIndia, Remote R19h ago
-
Senior-level Full TimeHyderabad, India20h ago
-
AI Engineer INR 2755K-4600KAI Safety | API Integration | Access Controls | Agent Orchestration | Data Leakage PreventionAsynchronous work | Continuous learning | High-performance culture | Knowledge sharing | Remote workSenior-level Full TimeRemote, Bangalore R20h ago
-
Senior-level Full TimeIN-KA-Bangalore20h ago
-
Senior Consultant INR 1900K-2600KAWS Lambda | Agile | Alerting | Amazon Web Services | Apache SparkGlobal team collaborationSenior-level Full TimeIN-KA-Bangalore20h ago
-
Mid-level Full TimeChennai, Tamil Nadu, India21h ago
-
AI Engineer (Manager level) INR 1200K-1800KAI Search | Agile | Apache Spark | Azure AI | Azure AI SearchMid-level Full TimeHyderabad, India21h ago
-
ML Engineer (Data) INR 1500K-2000KAmazon SageMaker | Apache Spark | BERT | CI/CD | CachingCoaching | Hybrid work | Learning opportunities | Mentoring | Paid volunteer dayMid-level Full TimeBengaluru21h ago
-
Senior AI Engineer INR 2500K-4000KAPI Gateway | API Integration | AWS CloudFormation | AWS Lambda | Amazon ECREmployee assistance program | Flexible work | Free Apps Access | Free Economist subscription | Free Podcasts AccessSenior-level Full TimeGurugram R22h ago
-
Senior AI Engineer INR 2500K-4000KAPI Gateway | AWS Lambda | Agile | Airflow | Amazon ECRAnnual leave | Employee assistance program | Flexible working | Free access to Economist content | Moving home supportSenior-level Full TimeBengaluru, Karnataka, India R22h ago
-
APIs | AWS | Automated testing | Azure | ClassificationEducational assistance | Flexible working hours | In-house soft skills training | Medical insuranceSenior-level Full TimeRemote - India R22h ago
-
AI/ML Developer INR 1000K-1890KAWS SageMaker | Accelerate | Adapter Layers | Apache Spark | Azure Machine LearningMid-level Full TimeGurugram, India22h ago
-
Product Data Scientist Manager, Play Apps INR 2000K-2146KCausal Inference | Classification | Ecommerce analytics | Experiment analysis | ForecastingSenior-level Full TimeBengaluru, Karnataka, India22h ago
-
Software Engineering INR 1500K-2000KApache Hadoop | Apache Kafka | Apache Spark | Azure | Azure ComputeMicrosoft cloud background check | On-call rotationMid-level Full TimeBengaluru, KA, IN22h ago
-
Software Engineer III - Python Developer + AWS + LLM / Gen AI INR 2000K-4000KAWS | Agile | Application Resiliency | Azure | CI/CDSenior-level Full TimeBengaluru, Karnataka, India22h ago
-
Senior Azure Data Engineer INR 2250K-3000KAzure Data | Azure Data Factory | Azure DevOps | Azure Functions | Azure SynapseSenior-level Full TimeThane, Maharashtra, India23h ago