Senior Data Engineer
IN KA Bengaluru, India
Quantiphi
While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!
Requirement Description:
Experienced in designing, implementation & testing of highly scalable and cost-efficient AWS based big data ETL/ELT pipeline using PySpark and SQL
Well-experienced in data warehousing
Well versed in working with transactional databases
Must have
Big data engineering technologies like Spark (PySpark)
Good knowledge of working with Python Programming
Good knowledge of advanced SQL concepts, Optimization techniques
Good knowledge of data warehousing, data modeling and concepts like OLAP and OLTP
Working knowledge with SQL databases like Postgresql, etc. is mandatory
Good knowledge of Data Migration like bringing on premise database to cloud or one cloud to another
Understand Architecture Requirements and ensure effective design, development, validation, and support activities.
Understanding of data based AWS services like Glue, EMR, Lambda, DMS, RDS, S3 etc and basic AWS architecture best practices.
Knowledge of data pipeline orchestration using Step function or Airflow
Ability to use the AWS service APIs, AWS CLI, and SDKs to build applications
Good to have
Knowledge of NoSQL databases like Dynamo can be add on
Knowledge of BI tools
Knowledge of working with streaming pipelines
If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Airflow APIs Architecture AWS Big Data Data Warehousing ELT Engineering ETL Excel Lambda NoSQL OLAP Pipelines PostgreSQL PySpark Python Spark SQL Streaming Testing
Perks/benefits: Career development
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.