IN_Sr Associate_Big Data Developer _Data & Analytics _Advisory _Mumbai
INR 2000K-2400K (estimate) Senior-level Full Time
Tasks
- Apply data warehousing dimensional modeling partitioning bucketing
- Build data pipelines on Hadoop Spark ecosystem
- Conduct data quality checks
- Design and optimize SQL for large scale datasets
- Develop Spark jobs batch and streaming
- Implement batch near real time flows with NiFi
- Implement streaming ingestion with Kafka
- Manage Hadoop components HDFS YARN Tez
- Orchestrate jobs with Oozie or Airflow
- Perform unit and integration testing
- Tune Spark and SQL performance
- Use Git and CI CD for development
- Use Hive or Impala for data querying
- Write technical design documents HLD LLD
Perks/Benefits
Skills/Tech-stack
AWS | AWS Glue | Amazon EMR | Amazon S3 | Apache Airflow | Apache NiFi | Apache Spark | Azure | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure HDInsight | Azure Synapse | Azure Synapse Analytics | Bucketing | CI/CD | Cloud Dataproc | Cloudera Manager | Data Lake Storage | Dimensional Modeling | GCP | Git | GitLab CI | Google BigQuery | Google Cloud | Google Cloud Dataproc | HDFS | Hadoop | Hive | Impala | Integration Testing | Jenkins | Kafka | Linux | Observability | Oozie | Partitioning | PySpark | SQL | Scala | Shell Scripting | Synapse Analytics | Tez | Unit Testing | YARN
Education
N/A
Roles
Big Data Developer | Data Developer | Data Engineer | Developer | Engineer
Related jobs
-
Senior-level Full TimeIndia11h ago
-
Mid-level Full TimeHaryana, Gurugram, India14h ago
-
GCP Cloud Database Specialist INR 1000K-1500KBackup and Restore | Cloud Databases | Cloud platform | Data Reconciliation | Data ValidationMid-level Full TimeIN-AP-Hyderabad15h ago
-
Senior System Development Engineer, Silicon, Google Cloud INR 2125K-3500KAutomation | Best practices | Deployment | Distributed Systems | Incident ManagementSenior-level Full TimeBengaluru, Karnataka, India17h ago
-
Senior Embedded Software Development Engineer INR 2156K-3200KAgile | C# | CI/CD | CppUnit | EmulatorsAnnual bonus | Flexible hours | Health insurance | On-demand training | Paid volunteering leaveSenior-level Full TimeChennai, TN, IN, 60009720h ago
-
Data Engineer II INR 1500K-2000KApache Spark | Azure Data | Azure Data Explorer | Azure Data Factory | Azure SynapseMid-level Full TimeHyderabad, TS, IN22h ago
-
AWS | Agent systems | Agentic AI | Amazon SageMaker | Autonomous AgentsSenior-level Full TimeBengaluru Millenia, India1d ago
-
Azure Databricks | CI/CD | Data Pipelines | Data integration | ETLSenior-level Full TimeGurugram 10 C, India1d ago
-
Azure Databricks | Data integration | ETL | Filtering | JoiningSenior-level Full TimeGurugram 10 C, India1d ago
-
IN_Associate_Python/Gen AI_AI in Delivery_Advisory_Bangalore INR 3000K-4000KAWS | Azure | CI/CD | Data Augmentation | Data PreprocessingFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India1d ago
-
IN_Associate_Python/Gen AI_AI in Delivery_Advisory_Bangalore INR 3000K-4000KAWS | Adversarial Networks | Autoregressive models | CI/CD | Cloud platformFlexible work arrangements | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India1d ago
-
IN_Associate_Python/Gen AI_AI in Delivery_Advisory_Bangalore INR 3000K-4000KAWS | Adversarial Networks | Autoregressive models | Azure | CI/CDFlexibility program | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India1d ago
-
Module Lead - GCP Data Engineer Job INR 1650K-3200KBigQuery | Bigtable | CI/CD | Cloud Build | Cloud NetworkingContinuous learning | Flexible work arrangements | Stable employmentSenior-level Full TimeHyderabad, TG, IN1d ago
-
Senior Principal Engineer - Data Engineering INR 3000K-4000KAWS | Apache Spark | Automation | Azure Data | Azure Data FactoryHybrid work environment | Inclusive culture | Professional development opportunities | Supportive leadersSenior-level Full TimePune - Panchshil, India R1d ago
-
Database (Oracle & Big Data) Programmer Analyst INR 1050K-1500KBackup & Recovery | Hadoop | Hive | Impala | Index ManagementMid-level Full TimeIT BUILDING, RAMANUJAN IT SEZ,, India1d ago
-
Senior Data Engineer INR 3000K-4000KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Machine LearningSenior-level Full TimeIT Centre Bengaluru GDTC, India1d ago
-
API Development | Async Programming | Autogen | CI/CD | CrewAISenior-level Full TimeBengaluru Millenia, India1d ago
-
Lead AI Designer– Agentic AI (L6) INR 2000K-3500KAPIs | AWS | Agent SDK | Asynchronous programming | AutogenSenior-level Full TimePune, India1d ago
-
Senior Data Engineer INR 3000K-4000KAgile | Analysis Services | Azure Analysis | Azure Analysis Services | Azure DataSenior-level Full TimeIT Centre Bengaluru GDTC, India1d ago
-
Associate, Data Engineer INR 1400K-2700KApache Airflow | Azure Blob | Azure Blob Storage | Azure Event | Azure Event GridComprehensive healthcare | Flexible time off | Flexible-hybrid work | Tuition reimbursementMid-level Full TimeM11 - Commers III, 43, 44 … R1d ago
-
A/B | A/B Testing | AWS | Azure | B testingFlexible work arrangements | Hybrid work environment | MentorshipSenior-level Full TimeMumbai Goregaon, India1d ago
-
AWS SageMaker | Agent systems | Autonomous Agents | Azure ML | Computer VisionFlexibility programs | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India1d ago
-
Azure | Azure Databricks | ETL | PySpark | PythonFlexible work arrangements | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeGurugram 10 C, India1d ago
-
Azure Databricks | ETL | PySpark | PythonFlexibility programmes | Inclusive benefits | MentorshipSenior-level Full TimeGurugram 10 C, India1d ago
-
26-2253: Data Engineer, AI/BI: Hyderabad, India INR 1500K-2040KAWS Kinesis | Access Control | Airflow | Amazon Redshift | Apache BeamMid-level Full TimeHyderabad, Telangana1d ago