Senior Software Engineer – Cloud Data Pipeline
Bangalore
Calix
Calix is a leading provider of cloud and software platforms, systems, and services for internet service providers. Partner with Calix and grow your business.Calix is leading a service provider transformation to deliver a differentiated subscriber experience around the Smart Home and Business, while monetizing their network using Role based Cloud Services, Telemetry, Analytics, Automation, and the deployment of Software Driven Adaptive networks.
As part of a high performing global team, the right candidate will play a significant role as Calix Cloud Data Engineer involved in architecture design, implementation, technical leadership in data ingestion, extraction, and transformation domain.
Responsibilities and Duties:
- Work closely with Cloud product owners to understand, analyze product requirements and provide feedback.
- Architecture design and review of Cloud data pipeline, including data ingestion, extraction, and transformation services.
- Implement and enhance support tools for monitoring and acting on data pipeline issues and interpret trends and patterns.
- Technical leadership of software design in meeting requirements of service stability, reliability, scalability, and security
- Guiding technical discussions within engineer group and making technical recommendations
- Design review and code review with peer engineers
- Guiding testing architecture for large scale data ingestion and transformations.
- Customer facing engineering role in debugging and resolving field issues.
Qualifications:
- 7-10 years of software engineering experience
- 4+ years of development experience performing ETL and/or data pipeline implementations.
- Organised and goal-focused, ability to deliver in a fast-paced environment.
- Strong understanding of distributed systems and Restful APIs.
- Experience in cloud-based big data projects (preferably deployed in GCP)
- Hands on experience implementing data pipeline infrastructure for data ingestion and transformation near real time availability of data for applications, BI analytics, and ML pipelines.
- Working knowledge of Data Lake technologies, data storage formats (Parquet, ORC, Avro) and query engines (BI Engine, Athena, Presto etc) and associated concepts for building optimised solutions at scale.
- Experience in designing data streaming and event-based data solutions (GCP Pub/Sub, Kafka, Kinesis, or like)
- Experience building data pipelines (Pub/Sub, Spark or like)
- Working experience with the cloud-based data warehouse like (BigQuery, RedShift, Azure SQL Data Warehouse, etc.)
- Experience designing cost optimised solutions for large datasets using open-source frameworks.
- Knowledge and experience designing solutions with cloud-native GCP Cloud services as well as deploying alternative solutions for appropriate use cases.
- Expert level in one of the following programming languages or similar- Python, Java.
- BS degree in Computer Science, engineering, or mathematics or equivalent experience.
Location:
- Bangalore, India
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: APIs Architecture Athena Avro Azure Big Data BigQuery Computer Science Data pipelines Data warehouse Distributed Systems Engineering ETL GCP Java Kafka Kinesis Machine Learning Mathematics Open Source Parquet Pipelines Python Redshift Security Spark SQL Streaming Testing
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.