Senior Big Data Platform Engineer
Singapore, Singapore
Centre for Strategic Infocomm Technologies
CSIT is a technical agency in the Ministry of Defence that harnesses cutting-edge digital technologies to meet Singapore’s security needs.
Data processing is an essential part of national security in the modern world. As an engineer in our Big Data Platform team, you will build and own a solid data platform as the foundation to support critical national security missions. You will push the limits of technology to ensure that the whole big data pipeline runs optimally and smoothly from ingestion to processing to persistence. You are encouraged to suggest the best way to deliver value and the technologies used. Join the awesome CSIT family and use cutting-edge technologies to protect the nation.
Responsibilities
- Design, implement and maintain real-time high-volume data processing, querying and persistence solutions using Big Data technologies
- Design and implement highly available, resilient, scalable and sustainable Big Data platforms
- Administer (configuration, backup and recovery) Big Data platform services running across multiple clusters
- Deploy and manage the cluster efficiently using DevOps and automation tools
- Build performance and health monitoring tools to ensure optimal system performance and availability
- Work closely with different teams to ensure smooth onboarding and operations of the Big Data platform
- Conduct resource planning and scaling exercises to support increasing workload
- Conduct research and evaluations of new technologies
- Innovate and improve the current system
Requirements (Minimum Qualifications)
- Bachelor’s degree in Computer Engineering, Computer Science or equivalent experience
- Able to communicate clearly
- Positive attitude and excellent teamwork
- Passionate about big data technologies
- Love for open source technologies
- At least 5 years of experience in big data domain
- Deep expertise in big data platform
- Experience in leading engineering teams
- Technical leadership and mentoring/coaching experience
Additional qualifications for senior position:
Nice-to-haves
- Experience in distributed systems
- Experience in supporting search workloads in multi-terabytes and billions of records
- Experience in managing physical and virtual infrastructures
- Experience in using and managing Hadoop cluster services (Spark, Kafka, HBase, Hive, Hadoop, HDFS, Nifi, Kudu, Solr, ElasticSearch)
- Experience in troubleshooting and resolving issues of the Big Data stack, ranging from hardware, networking, OS to software
- Experience with cluster management and DevOps tools (e.g. Ansible, Kubernetes)
- Experience working with application or product teams
- Experience working in an agile and innovative environment
Why join us?
- The work is purposeful and meaningful
- You will work with the best engineers
- We work with modern technologies and tech stacks
- We have excellent engineering culture and work-life balance
- We aspire to engineering and operational excellence
- We empower to innovate
- We grow together as a family
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Job stats:
0
0
0
Categories:
Big Data Jobs
Engineering Jobs
Tags: Agile Ansible Big Data Computer Science DevOps Distributed Systems Elasticsearch Engineering Hadoop HBase HDFS Kafka Kubernetes NiFi Open Source Research Security Spark
Region:
Asia/Pacific
Country:
Singapore
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.
Data Engineer II jobsBI Developer jobsData Scientist II jobsStaff Data Scientist jobsPrincipal Data Engineer jobsData Manager jobsJunior Data Analyst jobsData Science Manager jobsResearch Scientist jobsLead Data Analyst jobsBusiness Data Analyst jobsSenior AI Engineer jobsData Engineer III jobsSr. Data Scientist jobsData Science Intern jobsData Specialist jobsJunior Data Engineer jobsSoftware Engineer, Machine Learning jobsSenior Data Scientist, Performance Marketing jobsSoftware Engineer II jobsData Analyst Intern jobsBI Analyst jobsData Analyst II jobsSr Data Engineer jobsSenior Artificial Intelligence/Machine Learning Engineer - Remote, Latin America jobs
Snowflake jobsEconomics jobsLinux jobsHadoop jobsOpen Source jobsJavaScript jobsPhysics jobsMLOps jobsComputer Vision jobsAirflow jobsKafka jobsRDBMS jobsBanking jobsNoSQL jobsGoogle Cloud jobsData Warehousing jobsScala jobsR&D jobsKPIs jobsData warehouse jobsGitHub jobsOracle jobsClassification jobsPostgreSQL jobsSAS jobs
CX jobsStreaming jobsLooker jobsScikit-learn jobsScrum jobsTerraform jobsPandas jobsPySpark jobsDistributed Systems jobsBigQuery jobsData Mining jobsRobotics jobsJira jobsIndustrial jobsJenkins jobsRedshift jobsdbt jobsReact jobsUnstructured data jobsMicroservices jobsData strategy jobsE-commerce jobsMySQL jobsMatlab jobsNumPy jobs