Big Data Engineer Senior Consultant II

At Allstate, great things happen when our people work together to protect families and their belongings from life’s uncertainties. And for more than 90 years our innovative drive has kept us a step ahead of our customers’ evolving needs. From advocating for seat belts, air bags and graduated driving laws, to being an industry leader in pricing sophistication, telematics, and, more recently, device and identity protection. 

Job Description

We are seeking a highly skilled and experienced Senior Big Data Consultant to join our data engineering team and lead the development of high-performance, real-time data solutions. This role is ideal for someone who thrives in a fast-paced environment, has a passion for streaming technologies, and can architect scalable data pipelines that drive business impact.

You will be responsible for designing, developing, and maintaining real-time data processing systems using Apache Flink, AWS Kinesis Data Analytics (KDA), and related AWS services. Your work will directly support critical analytics, reporting, and machine learning workloads by ensuring timely and reliable data availability. As a senior member of the team, you will collaborate closely with data scientists, product managers, and cloud engineers to build end-to-end solutions that support both internal business intelligence and customer-facing applications

Key Responsibilities

• Design and develop real-time data processing applications using Apache Flink.

• Build and optimize Kinesis Data Analytics (KDA) applications for low-latency stream processing.

• Good knowledge on Scala or Java 

• Architect scalable, fault-tolerant solutions leveraging AWS services such as S3, Lambda, EMR, and Glue.

• Work closely with product and data teams to translate business requirements into technical solutions.

• Implement best practices for performance tuning, monitoring, and alerting of streaming pipelines.

• Perform code reviews, mentor junior team members, and contribute to architecture discussions.

• Ensure high availability, scalability, and reliability of data infrastructure.

Education

• Masters Degree (Preferred)

Experience

• 7 or more years of experience (Preferred)

Supervisory Responsibilities

• This job does not have supervisory duties.

Education & Experience (in lieu)

• In lieu of the above education requirements, an equivalent combination of education and experience may be considered.

Primary Skills

Big Data Engineering, Big Data Systems, Big Data Technologies, Data Science, Influencing Others

Shift Time

Shift B (India)

Recruiter Info

Annapurna Jha

ajhat@allstate.com

About Allstate

The Allstate Corporation is one of the largest publicly held insurance providers in the United States. Ranked No. 84 in the 2023 Fortune 500 list of the largest United States corporations by total revenue, The Allstate Corporation owns and operates 18 companies in the United States, Canada, Northern Ireland, and India. Allstate India Private Limited, also known as Allstate India, is a subsidiary of The Allstate Corporation. The India talent center was set up in 2012 and operates under the corporation's Good Hands promise. As it innovates operations and technology, Allstate India has evolved beyond its technology functions to be the critical strategic business services arm of the corporation. With offices in Bengaluru and Pune, the company offers expertise to the parent organization’s business areas including technology and innovation, accounting and imaging services, policy administration, transformation solution design and support services, transformation of property liability service design, global operations and integration, and training and transition.

Learn more about Allstate India here.