Sr. Data Engineer
Bengaluru, Karnataka, India
Weekday
At Weekday, we help companies hire engineers who are vouched by other software engineers. We are enabling engineers to earn passive income by leveraging & monetizing the unused information in their head about the best people they have worked...This role is for one of the Weekday's clients
Salary range: Rs 1500000 - Rs 4000000 (ie INR 15-40 LPA)
Min Experience: 5 years
Location: Bangalore
JobType: full-time
We are looking for a Senior Data Engineer to design and implement scalable data management services for data trust, compliance, access, and metadata management. In this role, you will be responsible for developing robust data pipelines, integrating Big Data tools and frameworks, and optimizing ELT processes to ensure seamless data processing and infrastructure scalability.
Requirements
Key Responsibilities:
- Design and implement scalable and configurable data management services.
- Develop ELT processes for efficient data transformation and integration.
- Select and integrate Big Data tools and frameworks to enhance capabilities.
- Monitor and optimize data performance, recommending infrastructure improvements.
- Build data platforms and services for real-time and batch processing of large datasets.
- Take end-to-end ownership of core product functionality and key technical initiatives.
Requirements:
- 5+ years of experience in data engineering within a fast-paced software-driven environment.
- Strong expertise in designing and developing scalable Data Warehouses and Data Pipelines.
- Proficiency in SQL, Java, and Scala.
- Deep understanding of Apache Spark, including Spark tuning, RDDs, and data frames.
- Hands-on experience in distributed environments using Kafka, Spark, Hive, Hadoop, etc.
- Familiarity with NoSQL databases and experience working with file formats like Parquet, Avro, etc.
- Strong understanding of distributed database systems architecture.
- Knowledge of engineering best practices and design principles.
Skills:
- Apache Spark
- Java / Scala
- Data Engineering
- ETL
- Distributed Systems
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Architecture Avro Big Data Data management Data pipelines Distributed Systems ELT Engineering ETL Hadoop Java Kafka NoSQL Parquet Pipelines Scala Spark SQL
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.