Senior Software Engineer RR/1312/2024
Bangalore, India
Emids
Committed to bettering healthcare and empowering wellness, we focus on what matters. Using technology and insights to create transformative outcomes for patients, providers and partners.Exp: 5 to 12 Years
Job Description:
Please consider Python as ETL tool to read data from source perform all datatype & data transformation, error handling ,logging within python and load to data bricks managed delta tables.
Anyone we are hiring should be very good at python along with very good knowledge in Databricks & PySpark so that they can evaluate and perform transformation within Databricks as well. Candidate should be able to do almost everything in python including class ,objects, methods, dictionaries ,loops, Wheel files ,packages, should be able to connect to any source data including any DBs, streaming data.
Expertise in Azure services such as Azure SQL Database, Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake
• The ideal candidate has extensive experience in data engineering, working with large-scale data infrastructures, and optimizing ETL processes
• Expertise in Azure services such as Azure SQL Database, Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake
• At least 6+ years of experience in data engineering, with a focus on building data pipelines
• Proven experience with Spark, Scala, and other related technologies
• Familiarity with Could Warehouses, data lakes, storage formats.
• Strong knowledge of data modeling and ETL processes
• Experience leading data engineering projects.
• Proven experience with Databricks and Apache Spark
• Strong proficiency in Python
• Experience in Unix shell scripting and scheduling tools
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Azure Databricks Data pipelines Engineering ETL Pipelines PySpark Python Scala Shell scripting Spark SQL Streaming
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.