AWS Data Engineer

At Lilly, we unite caring with discovery to make life better for people around the world. We are a global healthcare leader headquartered in Indianapolis, Indiana. Our employees around the world work to discover and bring life-changing medicines to those who need them, improve the understanding and management of disease, and give back to our communities through philanthropy and volunteerism. We give our best effort to our work, and we put people first. We’re looking for people who are determined to make life better for people around the world.

Job Description

As a Cloud Data Engineer, you will play a crucial role in designing, implementing, and maintaining our cloud-based data infrastructure. You will collaborate with cross-functional teams to ensure data availability, reliability, and security, enabling data-driven decision-making and insights.

Key Responsibilities

 

• Design and develop data pipelines to extract, transform, and load (ETL) data from various sources into cloud-based data storage solutions.
• Implement data integration solutions using cloud services such as AWS (Amazon Web Service)
• Migrate Data pipelines previously developed in OnPrem to AWS Native data pipelines.
• Incorporate deep data management expertise into solutions, including permissions, recovery, security, and monitoring.
• Validate and qualify solutions.
• Identify, troubleshoot, debug, and resolve technical issues.
• Work with business and IT groups to understand source systems and enterprise infrastructure offerings.
• Work with data from multiple data sources to build integrated views that will drive decisions.
• Collaborate with data scientists, analysts, and software engineers to understand data requirements and ensure the availability of quality data.
• Optimize and tune data pipelines for performance and scalability.
• Implement data security and access controls to protect sensitive information.
• Monitor, troubleshoot, and resolve data pipeline and infrastructure issues.
• Stay current with emerging data engineering technologies and best practices.
• Document data engineering processes and standards.
• Work with stakeholders to define data architecture and modelling requirements.
• Provides technical leadership and consultancy to Global Services in close partnership with Global Services Architect to ensure the delivery of cloud solutions is aligned with the Enterprise IT (Information Technology) architecture.
• Creates and reviews system designs to leverage existing standards, to evaluate alternative technologies and to build a cost-effective solution.
• Develops a deep expertise on cloud platforms/technologies used by Lilly Solution Services to assist system custodians with life cycle management or complex changes.
• Ensures coding standards or technical implementation guides are in place for  cloud platforms or technologies and educates External Service Providers to ensure they are followed.
• Helps to establish a DevOps to improve the quality, security, automation capabilities and stability of our cloud solutions.
• Identifies synergies and reuse opportunities to improve efficiency and cost savings.

Qualifications

 

• Bachelor’s degree in computer science, Information Technology, or a related field (master’s preferred).

• 4 to 7 Years of related experience.

• Proven experience as a data engineer, preferably in a cloud environment.
• Prior experience in migrating from informatica workflow to AWS native processing.
• Strong experience with AWS ETL/File Movement tools (Glue, Athena, Lambda,  and other AWS integration stack)
• Strong experience with Agile Development, CI/CD, Jenkins, GitHub, Automation platforms
• Strong experience with AWS database technologies (Redshift, Aurora, RDS, S3 & other AWS Data Service) covering security, policies, access management.
• Strong programming Experience with Lambda, Python and Spark
• Experience with Apache Airflow, Ansible & other automation stack.
• Familiarity with data modeling, data warehousing, and data architecture best practices.
• Strong problem-solving and analytical skills.
• Excellent communication and collaboration skills.
• Ability to work in a fast-paced and dynamic environment.
• Work independently and guide juniors
  
   

Technical Experience

Must Have

• Strong in Postgres SQL

• Strong in AWS Glue / Lambda

• Python / Py Spark

• AWS S3 / Redshift / Aurora DB / Athena
• AWS Step Functions/Event Bridge

• Orchestration, Airflow
• Data Audit capturing
• Data modelling techniques
• Github Actions
• PL/SQL
• Agile, JIra, Scrum

Good to Have

Power BI / Tableau
Metadata Manager
AWS Glue Catalog
AWS CloudTrail

AWS Developer/Practitioner Certification

Service Management
Oracle / MS SQL

 

Lilly is dedicated to helping individuals with disabilities to actively engage in the workforce, ensuring equal opportunities when vying for positions. If you require accommodation to submit a resume for a position at Lilly, please complete the accommodation request form (https://careers.lilly.com/us/en/workplace-accommodation) for further assistance. Please note this is for individuals to request an accommodation as part of the application process and any other correspondence will not receive a response.

Lilly does not discriminate on the basis of age, race, color, religion, gender, sexual orientation, gender identity, gender expression, national origin, protected veteran status, disability or any other legally protected status.

#WeAreLilly