Lead Data Engineer

Job Description:

The Purpose of the role

Data is at the core of our business. The data engineer is a technical job that requires substantial expertise in a broad range of software development and programming fields. The data engineer should especially have sufficient knowledge of big data solutions to be able to implement those on premises or in the cloud.

A data engineer generally works on implementing complex big data projects with a focus on collecting, parsing, managing, analyzing and visualizing large sets of data to turn information into insights using multiple platforms. He or she should be able to decide on the needed hardware and software design needs and act according to the decisions. The big data engineer should be able to develop prototypes and proof of concepts for the selected solutions.

Ideal candidate for this role is someone with a strong background in computer programming, statistics, and data science who is eager to tackle problems with large, complex datasets using the latest Python, and/or PySpark.  You are a self-starter who will take ownership of your projects and deliver high-quality data-driven analytics solutions.  You are adept at solving diverse business problems by utilizing a variety of different tools, strategies, algorithms and programming languages.   

Location: Bangalore, India

 

Responsibilities

• Utilize the data engineering skills within and outside of the developing Chubb information ecosystem for discovery, analytics and data management.
• Work with data science team to deploy Machine Learning Models.
• You will be using Data wrangling techniques converting one "raw" form into another including data visualization, data aggregation, training a statistical model etc.
• Work with various relational and non-relational data sources with the target being Azure based SQL Data Warehouse & Cosmos DB repositories.
• Clean, unify and organize messy and complex structured/unstructured data sets for easy access and analysis.
• Create different levels of abstractions of data depending on analytics needs.
• Hands on data preparation activities using the Azure technology stack especially Azure Databricks is strongly preferred.
• Implement discovery solutions for high speed data ingestion.
• Work closely with the Data Science team to perform complex analytics and data preparation tasks.
• Work with the Sr. Data Engineers on the team to develop APIs.
• Sourcing data from multiple applications, profiling, cleansing and conforming to create master data sets for analytics use. 
• Utilize state of the art methods for data manning especially unstructured data.
• Experience with Complex Data Parsing (Big Data Parser) and Natural Language Processing (NLP) Transforms on Azure a plus.
• Design solutions for managing highly complex business rules within the Azure ecosystem.
• Performance tune data loads.

 

 

Required knowledge, Skills and qualifications

• Knowledge of Python and Pyspark is an absolute must.
• Knowledge of Azure, Hadoop 2.0 ecosystems, HDFS, MapReduce, Hive, Pig, Sqoop, Mahout, Spark is important for this role.
• Experience with Web Scraping frameworks (Scrapy or Beautiful Soup or similar). 
• Extensive experience working with Data APIs (Working with REST endpoints and/or SOAP).
• Significant programming experience (with above technologies as well as Java, R and Python on Linux).
• Knowledge of any commercial distribution application such as Horton Works, Cloudera, MapR etc.
• Excellent working knowledge of relational databases, MySQL, Oracle etc.
• Experience with Complex Data Parsing (Big Data Parser) a must. Should have worked on XML, JSON and other custom Complex Data Parsing formats.
• Natural Language Processing (NLP) skills with experience in Apache Solr, Python a plus.
• Knowledge of High-Speed Data Ingestion, Real-Time Data Collection and Streaming is a plus.
• Bachelor’s in computer science or related educational background.
• 3-5 years of solid experience in Big Data technologies a must.
• Microsoft Azure certifications a huge plus.
• Data visualization tool experience a plus.

 

Why Chubb Business Services India?

Here are some reasons to join us:

• You will be joining a group of diverse and driven professionals trained to succeed in today’s insurance landscape.
• We provide the opportunity to cultivate a robust career; we accommodate your growth and development even as we expect you to make meaningful contributions to our company.
• We offer competitive compensation and benefit programs. We pride ourselves in being a meritocracy that rewards the best talent.
• We seek to create an inclusive environment where employees of different cultures and backgrounds can thrive.

Our company and culture

Chubb Business Services India focuses on building strong capability in the areas related to technology, CAT modeling, risk management, actuarial services and advanced analytics. The culture we would like to establish would align with the core principles of the organization as outlined below with the desire to transform and grow together.   

Our culture reflects a commitment to craftsmanship and as craftspeople, we adhere to certain ideals:

• A Can-Do Attitude – we are optimistic about our future and we have an enthusiasm to serve our customers and business partners with utmost speed and energy. We approach our work with passion – people want to live their lives with meaning. We have a shared vision to build something great – to serve society and leave the world a little better place because we’ve been here.
• Excellence in All We Do – we hold ourselves to exacting standards of technical proficiency. Customers can count on us to pay claims fairly and quickly. Agents and brokers can rely on us for attention to detail and help them build their businesses
• Constant Improvement – we are a learning organization. We are constantly searching for ways to improve our craft. We relentlessly pursue data-driven insights to drive better, smarter decision-making – our portfolio management efforts, for example.
• Global Reach and Local Expertise – we apply our craft locally all around the world. We appreciate differences in culture, history and geography. And this enables us to adapt accordingly to best serve diverse customers, markets and distribution.

 

 

 

 

Job Description:

The Purpose of the role

Data is at the core of our business. The data engineer is a technical job that requires substantial expertise in a broad range of software development and programming fields. The data engineer should especially have sufficient knowledge of big data solutions to be able to implement those on premises or in the cloud.

A data engineer generally works on implementing complex big data projects with a focus on collecting, parsing, managing, analyzing and visualizing large sets of data to turn information into insights using multiple platforms. He or she should be able to decide on the needed hardware and software design needs and act according to the decisions. The big data engineer should be able to develop prototypes and proof of concepts for the selected solutions.

Ideal candidate for this role is someone with a strong background in computer programming, statistics, and data science who is eager to tackle problems with large, complex datasets using the latest Python, and/or PySpark.  You are a self-starter who will take ownership of your projects and deliver high-quality data-driven analytics solutions.  You are adept at solving diverse business problems by utilizing a variety of different tools, strategies, algorithms and programming languages.   

Location: Bangalore, India

 

Responsibilities

• Utilize the data engineering skills within and outside of the developing Chubb information ecosystem for discovery, analytics and data management.
• Work with data science team to deploy Machine Learning Models.
• You will be using Data wrangling techniques converting one "raw" form into another including data visualization, data aggregation, training a statistical model etc.
• Work with various relational and non-relational data sources with the target being Azure based SQL Data Warehouse & Cosmos DB repositories.
• Clean, unify and organize messy and complex structured/unstructured data sets for easy access and analysis.
• Create different levels of abstractions of data depending on analytics needs.
• Hands on data preparation activities using the Azure technology stack especially Azure Databricks is strongly preferred.
• Implement discovery solutions for high speed data ingestion.
• Work closely with the Data Science team to perform complex analytics and data preparation tasks.
• Work with the Sr. Data Engineers on the team to develop APIs.
• Sourcing data from multiple applications, profiling, cleansing and conforming to create master data sets for analytics use. 
• Utilize state of the art methods for data manning especially unstructured data.
• Experience with Complex Data Parsing (Big Data Parser) and Natural Language Processing (NLP) Transforms on Azure a plus.
• Design solutions for managing highly complex business rules within the Azure ecosystem.
• Performance tune data loads.

 

 

Required knowledge, Skills and qualifications

• Knowledge of Python and Pyspark is an absolute must.
• Knowledge of Azure, Hadoop 2.0 ecosystems, HDFS, MapReduce, Hive, Pig, Sqoop, Mahout, Spark is important for this role.
• Experience with Web Scraping frameworks (Scrapy or Beautiful Soup or similar). 
• Extensive experience working with Data APIs (Working with REST endpoints and/or SOAP).
• Significant programming experience (with above technologies as well as Java, R and Python on Linux).
• Knowledge of any commercial distribution application such as Horton Works, Cloudera, MapR etc.
• Excellent working knowledge of relational databases, MySQL, Oracle etc.
• Experience with Complex Data Parsing (Big Data Parser) a must. Should have worked on XML, JSON and other custom Complex Data Parsing formats.
• Natural Language Processing (NLP) skills with experience in Apache Solr, Python a plus.
• Knowledge of High-Speed Data Ingestion, Real-Time Data Collection and Streaming is a plus.
• Bachelor’s in computer science or related educational background.
• 3-5 years of solid experience in Big Data technologies a must.
• Microsoft Azure certifications a huge plus.
• Data visualization tool experience a plus.

 

Why Chubb Business Services India?

Here are some reasons to join us:

• You will be joining a group of diverse and driven professionals trained to succeed in today’s insurance landscape.
• We provide the opportunity to cultivate a robust career; we accommodate your growth and development even as we expect you to make meaningful contributions to our company.
• We offer competitive compensation and benefit programs. We pride ourselves in being a meritocracy that rewards the best talent.
• We seek to create an inclusive environment where employees of different cultures and backgrounds can thrive.

Our company and culture

Chubb Business Services India focuses on building strong capability in the areas related to technology, CAT modeling, risk management, actuarial services and advanced analytics. The culture we would like to establish would align with the core principles of the organization as outlined below with the desire to transform and grow together.   

Our culture reflects a commitment to craftsmanship and as craftspeople, we adhere to certain ideals:

• A Can-Do Attitude – we are optimistic about our future and we have an enthusiasm to serve our customers and business partners with utmost speed and energy. We approach our work with passion – people want to live their lives with meaning. We have a shared vision to build something great – to serve society and leave the world a little better place because we’ve been here.
• Excellence in All We Do – we hold ourselves to exacting standards of technical proficiency. Customers can count on us to pay claims fairly and quickly. Agents and brokers can rely on us for attention to detail and help them build their businesses
• Constant Improvement – we are a learning organization. We are constantly searching for ways to improve our craft. We relentlessly pursue data-driven insights to drive better, smarter decision-making – our portfolio management efforts, for example.
• Global Reach and Local Expertise – we apply our craft locally all around the world. We appreciate differences in culture, history and geography. And this enables us to adapt accordingly to best serve diverse customers, markets and distribution.