Big Data Engineer (Hybrid)

Orion Innovation is a premier, award-winning, global business and technology services firm.  Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering, with a unique combination of agility, scale, and maturity.  We work with a wide range of clients across many industries including financial services, professional services, telecommunications and media, consumer products, automotive, industrial automation, professional sports and entertainment, life sciences, ecommerce, and education.

We are seeking a big data engineer for one of our clients. The successful candidate will be responsible for overseeing the creation and maintenance of our database infrastructure, including collecting and maintaining data, ensuring the integrity of our data, and creating and training data models.

Responsibilities:

Below are some of the responsibilities of a big data engineer:

• Design the architecture of our big data platform
• Build lakehouse PoC and make lakehouse technology decisions (based on our requirements)
• Build a scalable and performant data model satisfying our existing use cases and future plans using object storage (S3-compatible)
• Build code architecture and deployment infrastructure for data pipelines development using orchestrator (Airflow) and Spark
• Prepare documentation, knowledge transfer sessions and run books for data lake and data platform
• Mentor engineering team during work process
• Maintain our data pipeline and data lake platform
• Customize and oversee integration tools, warehouses, databases, and analytical systems
• Configure and provide availability for data-access tools

Job Qualifications and Skill Set:

Below are the qualifications expected of a big data engineer:

• 3 to 5 years of relevant data engineering experience
• Experience designing data model which utilizes object (S3-like) storage and used for massive parallel processing using Spark
• Experience running lake house technologies (DeltaLake, Iceberg) in production on big datasets (TB of data)
• Experience with pipeline orchestrators (Airflow/Dagster).
• Experience writing and operating Spark pipelines at scale (hundreds of cores, TBs or RAM)
• Experience with JVM based languages (Kotlin would be a plus)
• Experience with Python
• Excellent command of English both verbal and written

Orion is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, citizenship status, disability status, genetic information, protected veteran status, or any other characteristic protected by law.

Candidate Privacy Policy

Orion Systems Integrators, LLC and its subsidiaries and its affiliates (collectively, “Orion,” “we” or “us”) are committed to protecting your privacy. This Candidate Privacy Policy (orioninc.com) (“Notice”) explains:

• What information we collect during our application and recruitment process and why we collect it;
• How we handle that information; and
• How to access and update that information.

Your use of Orion services is governed by any applicable terms in this notice and our general Privacy Policy.