Sr. Analytics Data Engineer

About DAT

DAT is an award-winning employer of choice and a next-generation SaaS technology company that has been at the leading edge of innovation in transportation supply chain logistics for 45 years. We continue to transform the industry year over year, by deploying a suite of software solutions to millions of customers every day - customers who depend on DAT for the most relevant data and most accurate insights to help them make smarter business decisions and run their companies more profitably. We operate the largest marketplace of its kind in North America, with 400 million freights posted in 2022, and a database of $150 billion of annual global shipment market transaction data. Our headquarters are in Denver, CO, with additional offices in Missouri, Oregon, and Bangalore, India. For additional information, see www.DAT.com/company.

The Opportunity

DAT is looking for a Sr. Analytics Data Engineer to join our Product team in Bengaluru.

Please note that this role must overlap with Pacific Time hours (IST +12:30) due to the highly collaborative nature of the job. Typical hours are 3:00 pm - 12:00 am IST.

DAT Freight & Analytics (https://careers.dat.com/) is seeking a highly motivated, adaptable, and experienced Sr. Analytics Data Engineer to join our growing Insights Engineering team. In this pivotal role, you will be instrumental in building and maintaining robust, high-availability data infrastructure that powers our customer-facing data products delivered by our Data Analytics Services (DAS) team. You will be responsible for designing, developing, and implementing scalable data pipelines, robust data models, and innovative data delivery solutions that enable our paying customers to leverage valuable insights. This role requires a collaborative individual who thrives on both learning from and teaching others within a dynamic team environment.

Responsibilities:

• Design, develop, and implement resilient and scalable data pipelines to ingest, transform, and load data from various internal sources (Snowflake, Amazon S3, internal APIs) into our data repositories (Snowflake, Amazon S3).
• Develop, maintain, and optimize data models using tools like DBT to ensure consistent, accurate, and performant data representation for customer-facing products.
• Collaborate closely with DAS analytics consultants to understand their data requirements and the analytical needs of our paying customers, translating these into actionable and effective data infrastructure solutions.
• Engineer and manage data delivery mechanisms to various customer endpoints, including FTP, replicated database tables (Snowflake), and future solutions such as APIs and BI dashboards; proactively research and recommend optimal endpoint solutions.
• Implement and manage monitoring and alerting systems to ensure the high availability and reliability of data pipelines and customer-facing data products, proactively addressing any failures or performance issues, and rapidly responding to any customer-facing downtime.
• Optimize data structures, query performance, and data pipeline efficiency to ensure timely and reliable data retrieval for customer products.
• Thoroughly document data processes, models, infrastructure, and endpoint solutions to ensure maintainability, knowledge transfer, and clear understanding for both internal teams and stakeholders.
• Collaborate with data management and governance teams to uphold data quality standards and best practices, particularly concerning data sold to external customers.
• Stay up-to-date with emerging trends and technologies in data engineering, cloud data platforms (particularly AWS services like S3), and data delivery mechanisms, bringing innovative ideas to the team.
• Actively contribute to a collaborative team culture by sharing knowledge, and readily assisting colleagues to achieve common goals.

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, Data Science, or a related field.
• 3+ years of experience as an Analytics Engineer, Data Engineer, or in a similar role with a focus on building data infrastructure for analytical or product purposes.
• Strong proficiency in SQL and data modeling concepts and tools (e.g., DBT).
• Proven experience in designing, building, and maintaining data pipelines and data ingestion processes, including pulling data from APIs (experience with tools like Airflow is a plus).
• Familiarity with cloud data platforms, particularly Snowflake and AWS data ecosystem tools (e.g., S3, and other relevant services).
• Experience in developing solutions for making data available through various endpoints (e.g., FTP, database access, APIs).
• Understanding of and experience with implementing monitoring and alerting for data pipelines.
• Excellent communication and collaboration skills, with the ability to work effectively with technical and non-technical stakeholders.
• Strong problem-solving and analytical skills, with a proactive approach to identifying and resolving data-related challenges.
• Demonstrated ability to work independently, manage priorities in a fast-paced environment, and contribute effectively as part of a highly collaborative team.
• A genuine willingness to learn new technologies and methodologies, and to teach and mentor others.
• Ability to translate business logic into data engineering code.
• Experience moving a legacy data pipeline framework into a new, scalable framework.

Preferred Skills:

• Proficiency with code management tools (e.g., Git).
• Experience with infrastructure as code (especially Terraform)
• Experience working in an agile development environment.

DAT embraces the value of a diverse workforce, and believes it is a core strength of our company that we encourage those values in every DAT employee, at every level of our organization, regardless of tenure or rank. We provide equal employment opportunities (EEO) to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, disability, genetic information, marital status, amnesty, or status as a covered veteran in accordance with applicable federal, state, and local laws.

#LI-SC1