Big Data Engineer Senior Consultant I

At Allstate, great things happen when our people work together to protect families and their belongings from life’s uncertainties. And for more than 90 years our innovative drive has kept us a step ahead of our customers’ evolving needs. From advocating for seat belts, air bags and graduated driving laws, to being an industry leader in pricing sophistication, telematics, and, more recently, device and identity protection. 

Job Description

Job Title: Data Engineer

Job Description:

Looking for Data Engineer having 6 to 9 years of total IT work experience.

He/she will be responsible for assisting on building tools and technologies that allow Arity to ingest, normalize, transform, validate and persist trip data at scale in a high performance, reliable and available environment.

Build a state-of-the-art data management platform to execute the data lifecycle management operations within the operational data lake and warehouse and add resiliency in data quality.

Technical Experience Required:

• 3+ years of experience with Spark, Scala/Python and SQL.

• Strong understanding of big data design principles and implementing and managing large data sets along with ETL processing.

• Experience in Test Driven Development including knowledge and experience in testing frameworks.

• Experience with AWS services such as S3, EMR, Step Function, Lambda, SQS.

• Experience with Database such as PostgreSQL.

• Experience with CICD tool such as Jenkins.

• Clear understanding of data management, process automation, system monitoring and maintainability.

• understanding of cloud services, deployment models and distributed systems.

Good to Have:

• Experience with Trino and Starburst.

Job Responsibilities:

• Collaborates with other Engineers, IT Architects and business clients to ensure complete understanding of the platform's technical and business functionality.

• Participates in daily site, cross-site and product team ceremonies associated with scrum principles.

• Designs, develops and tests components of the data ingestion, transformation and delivery functions including full ETL processing.

• Tests, debug and analyzes performance and documents hardware and software technology components.

• Participates in knowledge sharing and contributes to the overall growth of the collective knowledge of the team.

Primary Skills:

• Spark with Scala or Python

• AWS Services (S3, EMR, Step Function, Lambda)

• SQL

• ETL

Primary Skills

Big Data Engineering, Big Data Systems, Big Data Technologies, Data Science, Influencing Others

Shift Time

Recruiter Info

Annapurna Jha

ajhat@allstate.com

About Allstate

Joining our team isn’t just a job — it’s an opportunity. One that takes your skills and pushes them to the next level. One that encourages you to challenge the status quo. And one where you can impact the future for the greater good.  

You’ll do all this in a flexible environment that embraces connection and belonging. And with the recognition of several inclusivity and diversity awards, we’ve proven that Allstate empowers everyone to lead, drive change and give back where they work and live. 

Good Hands. Greater Together.

The Allstate Corporation is one of the largest publicly held insurance providers in the United States. Ranked No. 84 in the 2023 Fortune 500 list of the largest United States corporations by total revenue, The Allstate Corporation owns and operates 18 companies in the United States, Canada, Northern Ireland, and India. Allstate India Private Limited, also known as Allstate India, is a subsidiary of The Allstate Corporation. The India talent center was set up in 2012 and operates under the corporation's Good Hands promise. As it innovates operations and technology, Allstate India has evolved beyond its technology functions to be the critical strategic business services arm of the corporation. With offices in Bengaluru and Pune, the company offers expertise to the parent organization’s business areas including technology and innovation, accounting and imaging services, policy administration, transformation solution design and support services, transformation of property liability service design, global operations and integration, and training and transition.

Learn more about Allstate India here.