IN_Sr Associate_Big Data Developer _Data & Analytics _Advisory _Mumbai
Mumbai Shivaji Park, India
INR 2000K-2400K (estimate) Senior-level Full Time
Tasks
- Apply data warehousing dimensional modeling partitioning bucketing
- Build data pipelines on Hadoop Spark ecosystem
- Conduct data quality checks
- Design and optimize SQL for large scale datasets
- Develop Spark jobs batch and streaming
- Implement batch near real time flows with NiFi
- Implement streaming ingestion with Kafka
- Manage Hadoop components HDFS YARN Tez
- Orchestrate jobs with Oozie or Airflow
- Perform unit and integration testing
- Tune Spark and SQL performance
- Use Git and CI CD for development
- Use Hive or Impala for data querying
- Write technical design documents HLD LLD
Perks/Benefits
Skills/Tech-stack
AWS | AWS Glue | Amazon EMR | Amazon S3 | Apache Airflow | Apache NiFi | Apache Spark | Azure | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure HDInsight | Azure Synapse | Azure Synapse Analytics | Bucketing | CI/CD | Cloud Dataproc | Cloudera Manager | Data Lake Storage | Dimensional Modeling | GCP | Git | GitLab CI | Google BigQuery | Google Cloud | Google Cloud Dataproc | HDFS | Hadoop | Hive | Impala | Integration Testing | Jenkins | Kafka | Linux | Observability | Oozie | Partitioning | PySpark | SQL | Scala | Shell Scripting | Synapse Analytics | Tez | Unit Testing | YARN
Education
N/A
Roles
Big Data Developer | Data Developer | Data Engineer | Developer | Engineer
Related jobs
-
Senior Data Engineer INR 1300K-2000KAPI Integration | API Security | Agile | Apigee | Asset ManagementSenior-level Full TimeChennai, Tamil Nadu, India10h ago
-
Senior-level Full TimeIN-TN-Chennai10h ago
-
Senior Consultant INR 1800K-2200KAmazon Aurora | Amazon Aurora PostgreSQL | Amazon Relational Database Service | Amazon Web Services | Aurora PostgreSQLSenior-level Full TimeIN-AP-Hyderabad10h ago
-
Senior-level Full TimeIN-AP-Hyderabad10h ago
-
Senior-level Full TimeHybrid, Hyderabad, Telangana, India R10h ago
-
Technical Program Manager, AI Agent Prototyping INR 1875K-3000KA/B | A/B Testing | B testing | Cloud Computing | Data QualityMid-level Full TimeBengaluru, Karnataka, India11h ago
-
GCP and Databricks Assistant Manager Hiring INR 1500K-2000KApache Spark | BigQuery | Bitbucket | Cloud Composer | Cloud FunctionsMid-level Full TimeBangalore, Karnataka, India12h ago
-
Manager - Fraud Analytics & Monitoring (Detection) INR 618K-618KAnomaly Detection | Artificial Intelligence | Behavioral analytics | Decisioning systems | Early WarningAnnual health checkup | Dental benefits | Gender neutral policy | Group term life insurance | Inclusive and diverse team cultureMid-level Full TimeMumbai, India13h ago
-
DE&A - Core - Big Data Engineering - ETL INR 2040K-2400KApache Hive | Apache Spark | Batch Processing | BigQuery | Cloud ArchitectureSenior-level Full TimeBangalore, Karnataka, India15h ago
-
Senior-level Full TimeIndia16h ago
-
AI Engineer INR 2000K-2600KAPI-driven | API-driven Development | Agent Development | Agent Development Framework | Agent Development KitMid-level Full TimePune, Maharashtra, India16h ago
-
Mid-level Full TimeHyderabad, India18h ago
-
Sr. Data Engineer INR 3200K-4500KAWS | Apache Airflow | Apache Spark | Cloud platform | Data ModelingCommuter benefits | Dental insurance | Disability insurance | Financial wellness support | Health insuranceSenior-level Full TimeBengaluru, India18h ago
-
Abinitio | Agile | Autosys | Cloud platform | Code CoverageSenior-level Full Time110380-IND-BENGALURU-INTL BLR Twr-1&2 CARNATION, India23h ago
-
Senior Software Engineer - Linux, Filesystems & Storage INR 3375K-4590KAgile Scrum | C# | Debugging | Device Drivers | Dm-cryptSenior-level Full TimeBangalore - Carina, India R23h ago
-
Principal Software Engineer - Linux, Filesystems & Storage INR 3375K-4590KAgile Scrum | C# | Device Drivers | Dm-crypt | File SystemSenior-level Full TimeBangalore - Carina, India R23h ago
-
Senior-level Full TimeBangalore, KA, IN23h ago
-
Manager, Software Engineering - ML, NLP, Gen AI INR 1100K-1680KAWS | AWS Bedrock | Agile | Azure | Azure AIEqual employment opportunity | Flexible work schedule | Hybrid work environmentMid-level Full TimeGurgaon - Cyber Park, India23h ago
-
Software Engineer INR 1500K-1700KAWS | Apache Flink | Apache Spark | Data Modeling | ETLEmployee wellness programs | Hybrid work | Remote work | Work-life balanceMid-level Full TimeRemote (IND), India R23h ago
-
Data Engineer II - Decision Analytics INR 800K-1050KAWS Lambda | Agile | Amazon EC2 | Amazon EMR | Amazon KinesisAccess to learning courses | Adoption Assistance | Flexible benefits | Health and welfare insurance | Parental leaveMid-level Full TimeGurgaon, India23h ago
-
Intern - Full stack Software Engineer with AI/ML INR 175K-175KAngular | Database Design | Django | Flask | GitFlexible working arrangements | Inclusive work environmentEntry-level Full Time InternshipBangalore (Airbus), India23h ago
-
Artificial Intelligence | Business Intelligence | Data Quality | Data Science | Data VisualizationFlexibility programme | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India23h ago
-
IN_ Associate_Palantir_ Data and Analytics_Advisory_Pan India INR 1800K-2400KIICS | Palantir | Project Management | SQL | ScrumInclusive benefits | Mentorship | Work flexibilitySenior-level Full TimeHyderabad - Salarpuria, India23h ago
-
IN_Senior Associate_GEN AI_AI-50_Advisory_Bangalore INR 2520K-4500KAWS | Amazon Redshift | Apache Airflow | Apache Hadoop | AzureSenior-level Full TimeBengaluru Millenia, India23h ago
-
IN_Senior Associate_AWS_OC - D&A_Advisory_ Bhubaneshwar INR 2000K-2400KAWS Glue | AWS Lambda | AWS Step Functions | Amazon AppFlow | Amazon AthenaSenior-level Full TimeBhubaneswar - Ihub, India23h ago