Senior Big Data Engineer
Tasks
- Architect and manage cloud based data solutions
- Build data ingestion transformation and storage solutions for structured and unstructured data
- Collaborate with data scientists and analytics teams to enable advanced analytics and ML
- Design scalable big data pipelines for batch and real time processing
- Develop ETL ELT workflows with data quality reliability and monitoring
- Ensure data security, compliance, and governance
- Implement data modeling partitioning and performance tuning
- Lead technical design and mentor engineers
- Perform code reviews and promote engineering best practices
- Work with distributed data processing frameworks
Perks/Benefits
- N/A
Skills/Tech-stack
Amazon EMR | Amazon Kinesis | Amazon S3 | Apache Airflow | Apache Flink | Apache HBase | Apache Kafka | Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Event | Azure Event Hubs | Azure Synapse | Azure Synapse Analytics | CI/CD | Cassandra | Data Factory | Data Governance | Data Lake | Data Modeling | Data Partitioning | Delta Lake | Docker | ELT | ETL | Event Hubs | Git | Google BigQuery | HDFS | Hadoop | Java | Kubernetes | NoSQL | Performance Tuning | Python | SQL | Scala | Synapse Analytics
Related jobs
-
Senior-level Full TimeBengaluru, Karnataka, India3h ago
-
Forward Deployed Engineer, Generative AI, Google Cloud INR 2500K-4500KAPIs | Batch Processing | BigQuery | CI/CD | Cloud platformSenior-level Full TimeBengaluru, Karnataka, India; Gurgaon, Haryana, India3h ago
-
Business Intelligence Engineer INR 1400K-1700KAgile | Jira | SQL | SnowflakeFlexible hybrid work model | Health insurance | Life insurance | Paid time off | Pension/retirement benefitsMid-level Full TimeHyderabad, India R4h ago
-
Mid-level Full TimeBengaluru, KA, IN4h ago
-
Manager INR 300K-370KApache Airflow | Data Modeling | Data Pipeline Architecture | Data Processing | Data pipelineSenior-level Full TimeIndia5h ago
-
Software Engineer II INR 950K-1250KApache Kafka | Apache Spark | BFS | Batch Processing | Cloud ComputingMid-level Full TimeHyderabad, TS, IN; Bengaluru, KA, IN5h ago
-
Senior-level Full TimeWhitefield, Bangalore, India6h ago
-
Senior-level Full TimeWhitefield, Bangalore, India6h ago
-
Senior-level Full TimeWhitefield, Bangalore, India6h ago
-
Senior-level Full TimeWhitefield, Bangalore, India6h ago
-
Senior-level Full TimeWhitefield, Bangalore, India6h ago
-
Azure Data Engineer INR 1000K-2000KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeIndia6h ago
-
Data Engineer Senior Consultant INR 2000K-2146KAzure Data | Azure Data Factory | CI/CD | DAX | Data FactoryOn-call support | Shift workSenior-level Full TimeHyderabad, TG, IN7h ago
-
Scientist 5, Data Science (12+ years experience . Generative AI, interpretable ML, Statistics) INR 2500K-5000KA/B | A/B Testing | APIs | AWS | Agnostic RAGSenior-level Full TimeBengaluru, KA, India7h ago
-
Associate Staff Engineer, Generative AI INR 2500K-3199KAI Foundry | AI Search | AWS Bedrock | Agentic AI | Azure AIMid-level Full TimeChennai, India7h ago
-
Senior-level Full TimeBANGALORE, KA, IN, 5600488h ago
-
Staff - Back End Developer INR 2040K-3500KAPI Design | AWS | Asynchronous processing | Azure | CachingSenior-level Full TimeBANGALORE, KA, IN, 5600488h ago
-
Sr.EM Data Engineering INR 2800K-3500KAPI Development | AWS | Automated testing | CI/CD | Data ProcessingDental insurance | Flexible time off | Medical insurance | Paid parental leave | SabbaticalSenior-level Full TimeIndia (Remote-First) R8h ago
-
Mid-level Full TimePune, Maharashtra, India8h ago
-
Sr. Staff Software Engineer, Machine Learning INR 2500K-4500KC++ | Data Pipelines | Deep learning | Explainability | FairnessHealth and wellness programs | Time away from workSenior-level Full TimeBengaluru, KA, India9h ago
-
Lead Software Engineer INR 3375K-4590KAWS | Distributed Systems | GitHub Copilot | Java | LLMContinuous learning | Cross-functional projects | Online learning campus | Training coursesSenior-level Full TimeMumbai, India10h ago
-
Software Engineer II INR 3050K-4590KAWS | Agile | Algorithms | Cloud Computing | Code reviewCareer growth | Collaborative team culture | Flexible work arrangements | Inclusive workplace | Learning opportunitiesSenior-level Full TimeMumbai, India10h ago
-
AI/ML Engineer II INR 950K-1250KAWS | Azure | Containerization | Data Preprocessing | Deep learningMid-level Full TimeRemote, India R11h ago
-
AI/ML Engineer II INR 950K-1250KAmazon Web Services | Azure | Cloud platform | Data Preprocessing | Deep learningMid-level Full TimeRemote, India R11h ago
-
Software Engineer (Backend - SDE 2) INR 2500K-3000KC++ | Cloud Computing | Distributed Systems | Go | JavaMid-level Full TimeBengaluru, India12h ago