Sr Data Engineer

Job Description:

We are seeking a highly skilled Senior Data Engineer to support our project team in the development, enhancement, and productionalization of a Dynamic Pricing Solution.

This role will focus on turning an existing functional prototype to a “production quality” solution. This involves designing, developing, and optimizing a scalable data architecture, implementing best practices in SDLC and CI/CD, and facilitating the migration of the solution to AWS and Databricks.

The ideal candidate will possess extensive experience in data engineering, data modeling, cloud platforms, and big data technologies, with a strong background in software development and automation. Knowledge and experience with Revenue Management and Pricing solutions or other machine learning solutions is preferred.

Key Responsibilities:

• System Development & Architecture
• Provide broad support for the creation of a scalable, production-grade system and data model.
• Gather and document requirements for the Dynamic Pricing Solution in collaboration with the project team.
• Support design and RMS solution architecture for the Dynamic Pricing Solution.
• Develop and refine the solution to ensure it aligns with production standards.
• Development
• Support the migration of the existing solution to AWS and Databricks.
• Refactor code to improve modularity, performance, and scalability.
• Conduct regression testing to ensure system stability and accuracy.
• Ensure security and reliability of the data in the warehouse.
• Automation & Best Practices
• Automate data feeds and pipelines to improve efficiency and reliability.
• Implement SDLC and CI/CD best practices in alignment with Project guidelines.
• Ensure compliance with data governance, security, and operational standards.

Required Qualifications:

• Education & Experience:
• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.
• 5+ years of experience in data engineering, big data technologies, and cloud-based platforms.
• Experience with Dynamic Pricing Solutions or Revenue Management Systems (RMS).
• Knowledge of machine learning pipelines and MLOps.
• Technical Skills:
• Strong programming skills in SQL, Python, and PySpark
• Deep expertise in cloud technologies: AWS, Databricks, Snowflake.
• Experience with Github
• Hands-on experience with ETL pipelines, data modeling, and automation.
• Proficiency in CI/CD, DevOps, and SDLC best practices.
• Soft Skills:
• Strong communication and collaboration abilities to work cross-functionally with IT and business teams.
• Excellent problem-solving and analytical skills.
• Ability to manage multiple tasks and work in a fast-paced environment.