Data Engineer - Databricks

1. Data Collection & Integration:
   • Extract, load, and transform data from various sources into our data platforms.
     
   • Collaborate with source system owners to understand data structures and nuances.
     
2. Database Design & Maintenance:
   • Help design and maintain robust and scalable data models.
     
   • Perform routine maintenance tasks to ensure data integrity and optimal performance.
     
3. Optimization:
   • Implement data pipelines that are efficient, reliable, and scalable.
     
   • Regularly review, optimize, and refactor the existing data infrastructure.
     
4. Collaboration:
   • Work closely with data scientists and analysts to ensure data availability and quality.
     
   • Collaborate with the IT and security teams to ensure compliance with data usage and storage policies.
     
5. Documentation:
   • Maintain clear documentation of data schemas, ETL processes, and related workflows.
     
   • Assist in training sessions or knowledge transfer sessions as necessary.
     
6. Continuous Learning:
   • Stay updated with the latest trends in data engineering and introduce relevant technologies and tools to the team.
     

Requirements

Education: Bachelor’s degree in Computer Science, Engineering, or a related technical field.
Experience: 0-2 years of experience in data engineering or a related discipline.
Technical Skills:
Proficiency in SQL and familiarity with popular data processing frameworks (e.g., Spark, Hadoop).
Experience with cloud platforms like AWS, Azure, or GCP is a plus.
Familiarity with data warehousing solutions (e.g., Snowflake, Redshift) and ETL tools. Deep understanding of Databricks and related components like Delta Live Table (DLT) pipeline implementations.
Familiarity with AWS services (additional cloud platform experience is a plus).

Soft Skills:
Strong analytical and problem-solving abilities.
Effective communication skills with the ability to explain technical concepts to non-technical stakeholders.
A team player who can also work independently when required.Should be proficient in Microsoft Word/Excel/PPT, Google doc/sheet/powerpoint and should know diagram tools like Draw.io

Benefits

We provide a competitive salary and benefits package, a vibrant work environment, and numerous opportunities for professional growth. You'll have the opportunity to work with a team of industry experts on exciting projects that transform businesses and create significant value. Join us to revolutionise the way companies leverage technology for digital transformation.

OnebyZero is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.