DE&A - Core - Big Data Engineering - ETL
Bangalore, Karnataka, India
INR 2040K-2400K (estimate) Senior-level Full Time
Tasks
- Architect and develop large scale PySpark applications
- Define migration strategies for rehost replatform and reengineer
- Design target state architectures using BigQuery Dataproc Dataflow and GCS
- Implement batch and streaming workloads on GCP
- Lead big data platform migrations to GCP
- Modernize ETL pipelines using cloud native or Spark based solutions
- Optimize data pipelines for performance, scalability, and cost
- Refactor Spark Hive and MapReduce jobs for GCP
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Hive | Apache Spark | Batch Processing | BigQuery | Cloud Architecture | Cloud Storage | Cloud platform | Data Migration | Dataflow | Dataproc | ETL | Google Cloud | Google Cloud Platform | Google Cloud Storage | MapReduce | PubSub | PySpark | Streaming processing
Education
N/A
Roles
Related jobs
-
Database Engineer INR 1500K-2400KAWS RDS | Azure SQL | Backup and Recovery | Data Modeling | Data WarehousingSenior-level Full TimeIN-TN-Chennai8h ago
-
Senior-level Full TimeIN-TN-Chennai8h ago
-
Aws Etl INR 1500K-2000KAWS Glue | AWS Lambda | AWS Step Functions | Amazon Athena | Amazon CloudWatchMid-level Full TimeIN-AP-Hyderabad8h ago
-
Aws Etl INR 1500K-2000KAWS Glue | AWS Lambda | AWS Step Functions | Amazon Athena | Amazon CloudWatchMid-level Full TimeIN-AP-Hyderabad8h ago
-
Aws Etl INR 1500K-2000KAWS Glue | AWS Lambda | AWS Step Functions | Amazon Athena | Amazon CloudWatchMid-level Full TimeIN-AP-Hyderabad8h ago
-
DE&A - AWS - Data Engineer + Data Platform Support INR 3000K-4000KAmazon Bedrock | Amazon EC2 | Amazon EMR | Amazon S3 | Amazon SageMakerSenior-level Full TimePune, Maharashtra, India10h ago
-
Mid-level Full TimeMumbai, Maharashtra, India10h ago
-
Lead Software Engineer - Big Data INR 2040K-3380KAmazon DynamoDB | Apache Avro | Apache Hive | Apache Kafka | Apache ParquetSenior-level Full TimeMumbai, Maharashtra, India13h ago
-
Mid-level Full TimeBangalore, Karnataka, India13h ago
-
Senior-level Full TimeBangalore, Karnataka, India13h ago
-
Sr. Data Engineer INR 2800K-4000KCloud warehousing | DAX | Dashboard Design | Data Quality | Data StorytellingCommunity outreach programs | Employee Resource Group | Employee stock purchase program | Health and wellness incentivesSenior-level Full TimeBangalore, KA, IN, 56003714h ago
-
Senior-level Full TimeGurgaon, Haryana, India14h ago
-
Amazon S3 | C# | C++ | Data Migration | Data replicationSenior-level Full TimeBengaluru, Karnataka, IND20h ago
-
AI Software Engineer INR 1753K-2600KAWS | Agent/tool use | Apache Spark | Azure | ContainersContinuing education program | Continuous learning | Family-friendly perks | Financial wellness programs | Flexible time offEntry-level Full TimeIN - HYDERABAD ORION, India20h ago
-
Apache Spark | Azure Blob | Azure Blob Storage | Azure Data | Azure Data FactoryFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeNoida, India20h ago
-
IN_Manager_SSIS_Data & Analytics_Advisory_Noida INR 1500K-2000KAWS | Azure | CI/CD | Cloud Data | Cloud Data PlatformsFlexibility programmes | Inclusive benefits | MentorshipMid-level Full TimeNoida, India20h ago
-
EY - GDS Consulting - AI And DATA -Snowflake Architect -Manager INR 3000K-4000KAgile | Batch Processing | Cloud Data | Cloud Data Warehouse | DBTCareer development | Coaching and feedback | Flexible work arrangementSenior-level Full TimeHyderabad, TG, IN, 50008120h ago
-
Sr. Data Engineer, Analytics Engineering INR 3000K-4000KAirflow | Data Governance | Data Modeling | Data Pipelines | Data QualitySenior-level Full TimeMumbai, India20h ago
-
Big Data Engineer - (Scala | Spark | Databricks | Cloud) INR 2000K-3000KAWS EMR | Apache Flink | Apache Hadoop | Apache Hive | Apache SparkSenior-level Full TimeIT BUILDING, RAMANUJAN IT SEZ,, India20h ago
-
Mid-level Full TimeChennai (Flexible), India20h ago
-
Senior-level Full Time111443-IND-HYDERABAD-INTL HYD WF CENTRE BLK B8 …20h ago
-
Profisee MDM Developer Senior Specialist (Databricks/AWS) INR 2755K-3380KData Governance | Data Management | Data Matching | Data Modeling | Data QualityEmployee stock purchase program | Flex day program | Flexible working schedule | Health insurance | Hybrid work modelSenior-level Full TimeBangalore-Affluence, India20h ago
-
Profisee MDM Developer Specialist (Databricks/AWS) INR 2755K-3380KData Governance | Data Management | Data Mapping | Data Matching | Data ModelingAnnual bonus | Flex day program | Flexible work schedule | Health insurance | Hybrid workSenior-level Full TimeBangalore-Affluence, India20h ago
-
Senior Software Engineer - Gen AI INR 2000K-2800KAJAX | AWS | Agile | Angular | AnsibleContinuous learningSenior-level Full Time110380-IND-BENGALURU-INTL BLR Twr-1&2 CARNATION, India20h ago
-
AWS | Apache Airflow | Cloud platform | Control-M | Data GovernanceSenior-level Full TimePLOT NO-1, S.NO. 77, India20h ago