Data Engineer - Databricks

As a Data Engineer specializing in Databricks, you will be instrumental in designing, building, and maintaining robust data processing pipelines leveraging the Databricks platform. You will work closely with data scientists and analysts to ensure the seamless integration and transformation of data to facilitate analytical needs and enhance decision-making across the organization.

Key Responsibilities:

• Data Pipeline Development: Design and implement scalable data pipelines using Databricks, ensuring efficient and reliable data processing from various sources.
• Collaboration: Work closely with data science and analytics teams to understand data requirements, providing support for data-driven projects and initiatives.
• Data Transformation: Utilize Spark SQL and Python to transform and clean large datasets, making them ready for analysis and reporting.
• Performance Optimization: Monitor and optimize the performance of data processing workflows within Databricks to ensure high availability and scalability.
• Data Quality Assurance: Implement data quality checks and validation processes to ensure the accuracy and integrity of data ingested into the system.
• Documentation: Create and maintain comprehensive documentation for data pipelines, architecture, and processes to facilitate knowledge sharing within the team.

Requirements

• Bachelor's degree in Computer Science, Data Engineering, or a related field.
• Proven experience as a Data Engineer with a focus on Databricks and Spark.
• Strong proficiency in SQL and Python for data manipulation and analysis.
• Experience with data modeling, ETL processes, and data warehousing solutions.
• Familiarity with cloud platforms (AWS, Azure, or GCP) is preferred.
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration abilities.

Benefits

• Salary: SGD 6000 - 9000
• Great Work Culture
• Includes Standard Annual and Sick Leave