IN_Sr Associate_Big Data Developer _Data & Analytics _Advisory _Mumbai
Mumbai Shivaji Park, India
INR 2000K-2400K (estimate) Senior-level Full Time
Tasks
- Build data pipelines on Hadoop Spark ecosystem
- Design data models and dimensional modeling
- Develop Spark jobs batch and streaming
- Develop technical design documents HLD LLD
- Implement batch and near real time flows with NiFi
- Implement data quality checks
- Implement streaming ingestion with Kafka
- Perform unit testing and integration testing
- Schedule and orchestrate data jobs with Airflow or Oozie
- Tune SQL and Spark performance for large datasets
- Use observability for Spark jobs
- Write Hive and Impala SQL queries
Perks/Benefits
- N/A
Skills/Tech-stack
AWS EMR | AWS Glue | Amazon S3 | Apache Airflow | Apache NiFi | Apache Spark | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure HDInsight | Azure Synapse | Azure Synapse Analytics | Bucketing | CI/CD | Cloud Dataproc | Cloudera Manager | Data Lake Storage | Data Warehousing | Dimensional Modeling | Git | GitLab CI | Google BigQuery | Google Cloud | Google Cloud Dataproc | HDFS | Hadoop | Hive | Impala | Jenkins | Kafka | Linux | Oozie | Partitioning | PySpark | SQL | Scala | Shell Scripting | Synapse Analytics | Tez | YARN
Education
Bachelor of Engineering | Master of Business Administration | Master of Computer Applications
Roles
Big Data Engineer | Data Engineer | Engineer | Software Engineer
Related jobs
-
Senior Data Engineer INR 1300K-2000KAPI Integration | API Security | Agile | Apigee | Asset ManagementSenior-level Full TimeChennai, Tamil Nadu, India10h ago
-
Senior Consultant INR 1800K-2200KAmazon Aurora | Amazon Aurora PostgreSQL | Amazon Relational Database Service | Amazon Web Services | Aurora PostgreSQLSenior-level Full TimeIN-AP-Hyderabad10h ago
-
Senior-level Full TimeIN-AP-Hyderabad10h ago
-
Technical Program Manager, AI Agent Prototyping INR 1875K-3000KA/B | A/B Testing | B testing | Cloud Computing | Data QualityMid-level Full TimeBengaluru, Karnataka, India12h ago
-
GCP and Databricks Assistant Manager Hiring INR 1500K-2000KApache Spark | BigQuery | Bitbucket | Cloud Composer | Cloud FunctionsMid-level Full TimeBangalore, Karnataka, India13h ago
-
Manager - Fraud Analytics & Monitoring (Detection) INR 618K-618KAnomaly Detection | Artificial Intelligence | Behavioral analytics | Decisioning systems | Early WarningAnnual health checkup | Dental benefits | Gender neutral policy | Group term life insurance | Inclusive and diverse team cultureMid-level Full TimeMumbai, India13h ago
-
DE&A - Core - Big Data Engineering - ETL INR 2040K-2400KApache Hive | Apache Spark | Batch Processing | BigQuery | Cloud ArchitectureSenior-level Full TimeBangalore, Karnataka, India15h ago
-
Senior-level Full TimeIndia16h ago
-
AI Engineer INR 2000K-2600KAPI-driven | API-driven Development | Agent Development | Agent Development Framework | Agent Development KitMid-level Full TimePune, Maharashtra, India16h ago
-
Mid-level Full TimeHyderabad, India18h ago
-
Sr. Data Engineer INR 3200K-4500KAWS | Apache Airflow | Apache Spark | Cloud platform | Data ModelingCommuter benefits | Dental insurance | Disability insurance | Financial wellness support | Health insuranceSenior-level Full TimeBengaluru, India19h ago
-
Abinitio | Agile | Autosys | Cloud platform | Code CoverageSenior-level Full Time110380-IND-BENGALURU-INTL BLR Twr-1&2 CARNATION, India23h ago
-
Senior Software Engineer - Linux, Filesystems & Storage INR 3375K-4590KAgile Scrum | C# | Debugging | Device Drivers | Dm-cryptSenior-level Full TimeBangalore - Carina, India R23h ago
-
Principal Software Engineer - Linux, Filesystems & Storage INR 3375K-4590KAgile Scrum | C# | Device Drivers | Dm-crypt | File SystemSenior-level Full TimeBangalore - Carina, India R23h ago
-
Software Engineer INR 1500K-1700KAWS | Apache Flink | Apache Spark | Data Modeling | ETLEmployee wellness programs | Hybrid work | Remote work | Work-life balanceMid-level Full TimeRemote (IND), India R23h ago
-
Data Engineer II - Decision Analytics INR 800K-1050KAWS Lambda | Agile | Amazon EC2 | Amazon EMR | Amazon KinesisAccess to learning courses | Adoption Assistance | Flexible benefits | Health and welfare insurance | Parental leaveMid-level Full TimeGurgaon, India23h ago
-
Intern - Full stack Software Engineer with AI/ML INR 175K-175KAngular | Database Design | Django | Flask | GitFlexible working arrangements | Inclusive work environmentEntry-level Full Time InternshipBangalore (Airbus), India23h ago
-
Artificial Intelligence | Business Intelligence | Data Quality | Data Science | Data VisualizationFlexibility programme | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India23h ago
-
IN_ Associate_Palantir_ Data and Analytics_Advisory_Pan India INR 1800K-2400KIICS | Palantir | Project Management | SQL | ScrumInclusive benefits | Mentorship | Work flexibilitySenior-level Full TimeHyderabad - Salarpuria, India23h ago
-
IN_Senior Associate_GEN AI_AI-50_Advisory_Bangalore INR 2520K-4500KAWS | Amazon Redshift | Apache Airflow | Apache Hadoop | AzureSenior-level Full TimeBengaluru Millenia, India23h ago
-
IN_Senior Associate_AWS_OC - D&A_Advisory_ Bhubaneshwar INR 2000K-2400KAWS Glue | AWS Lambda | AWS Step Functions | Amazon AppFlow | Amazon AthenaSenior-level Full TimeBhubaneswar - Ihub, India23h ago
-
Lead AI Engineer – Lead Engineer , VP INR 2500K-4500KAIOps | API Design | Agentic AI | BigQuery | Cloud ComposerAccident insurance | Best in class leave policy | Childcare assistance reimbursement | Comprehensive hospitalization insurance | Employee assistance programSenior-level Full TimePune - Margarpatta, India23h ago
-
Senior Data Analytics Engineer INR 2800K-4200KCloud Data | Cloud data sources | Data Analysis | Data Engineering | Data SourcesSenior-level Full TimeIndia - Hyderabad23h ago
-
AWS | Agile | Azure | Big Data | Data ComplianceFlexible workplace arrangements | Internal mobility | Internal training | Learning and development programs | MentoringSenior-level Full TimeHyderabad, India23h ago
-
AWS | AWS Glue | Access Control | Amazon Redshift | Amazon S3Flexible workplace arrangements | Internal mobility | Learning and development programs | MentoringSenior-level Full TimeBengaluru - GTP, India23h ago