Data Engineer

Job Summary

As part of data science team, you will be taking ownership of data platform along with your fellow data engineers and backend developers. This will involve building and maintaining data pipelines, integrations, performance tuning and setting monitoring tools. You will keep an eye on the future of the data warehouse is a data-driven way. As part of the Insights team you will play a crucial role in making the right data available in the near real-time in the data platform, and change the highly competitive project management landscape one task at a time.

Job Responsibility:

• Setup, operate and maintain data platform and data analytics tools. Manage data integration across different platforms and develop data ingestion from different data sources to data science infrastructure

• Build and maintain batch/real-time data pipelines, integrations, performance tuning and setting monitoring tools. Participate in cluster performance tuning to continually improve the quality of data pipeline and overall data platform

• Maintain analytical data marts and dashboards to ensure the readiness for serving business and operation purposes

• Work with data governance team to sure data integrity and securities across different departments. Serving & maintaining data/ML products.

• Apply software engineering rigor and best practices to data and ML services, including CI/CD, automation, testing, Infrastructure as Code, etc.

Qualifications:

• 3+ of Experience in Data Architecture, Data Modeler, or Data Management with large-scale data in the distributed data platform and overseeing complex data projects in an enterprise environment.

• Hand-on experience in data engineering, working with big data technology (Spark/Hadoop) and cloud platforms such as Azure, AWS, GCP, Docker or Kubernetes.

• Experience with hand-on productionized backend service development (Python – preferred, Java, Node.js).

• Bachelor's degree or equivalent experience in Computer Engineering, Computer Science, Information Technology, or related fields.

• Excellent Proficiency in programing languages (SQL, R, Python) and Experience with business intelligence solutions (Tableau, PowerBI or Advanced Excel)

• Full stack experience in data collection, aggregation, analysis, visualization, productionisation, and monitoring of ML products - MLOps

• Experience in using Git as source code version management tool and using or building CI/CD workflow.

• Hand-on experience in Infrastructure as Code such as Terraform or Pulumi.

• Hands-on knowledge in Agile methodology