Senior/Staff Big Data Storage and Computing Engineer, Recommendation Data Ecosystem
San Jose, California, United States
Team Introduction
Our team plays a crucial role in the data ecosystem of the TikTok Recommendation System, focusing on creating offline and real-time data storage solutions for large-scale recommendation, search, and advertising businesses, serving over 1 billion users. The core goals of the team are to ensure high system reliability, uninterrupted service, and smooth data processing. We are committed to building a storage and computing infrastructure that can adapt to various data sources and meet diverse storage requirements, ultimately providing efficient, cost-effective, and user-friendly data storage and management tools for the business.
Responsibilities
1. Architecture Design and Implementation: Design and implement offline and real-time data architectures for large-scale recommendation, search, and advertising systems based on Paimon and Flink. Ensure efficient data processing and storage to meet the strict requirements of the business for data timeliness and accuracy.
2. System Construction and Optimization: Design and implement flexible, scalable, stable, and high-performance storage systems and computing models. Use Paimon as the storage foundation and combine it with the powerful computing capabilities of Flink. Continuously optimize system performance to cope with the challenges brought by business growth.
3. Troubleshooting and Stability Assurance: Be responsible for troubleshooting production systems. For problems that occur in the Paimon-Flink architecture during operation, design and implement necessary mechanisms and tools, such as data consistency assurance and exception recovery, to ensure the overall stability of the production system.
4. Distributed System Construction: Build industry-leading distributed systems, including offline and online storage based on Paimon and batch and stream processing frameworks based on Flink, providing solid and reliable infrastructure support for massive data and large-scale business systems.
Our team plays a crucial role in the data ecosystem of the TikTok Recommendation System, focusing on creating offline and real-time data storage solutions for large-scale recommendation, search, and advertising businesses, serving over 1 billion users. The core goals of the team are to ensure high system reliability, uninterrupted service, and smooth data processing. We are committed to building a storage and computing infrastructure that can adapt to various data sources and meet diverse storage requirements, ultimately providing efficient, cost-effective, and user-friendly data storage and management tools for the business.
Responsibilities
1. Architecture Design and Implementation: Design and implement offline and real-time data architectures for large-scale recommendation, search, and advertising systems based on Paimon and Flink. Ensure efficient data processing and storage to meet the strict requirements of the business for data timeliness and accuracy.
2. System Construction and Optimization: Design and implement flexible, scalable, stable, and high-performance storage systems and computing models. Use Paimon as the storage foundation and combine it with the powerful computing capabilities of Flink. Continuously optimize system performance to cope with the challenges brought by business growth.
3. Troubleshooting and Stability Assurance: Be responsible for troubleshooting production systems. For problems that occur in the Paimon-Flink architecture during operation, design and implement necessary mechanisms and tools, such as data consistency assurance and exception recovery, to ensure the overall stability of the production system.
4. Distributed System Construction: Build industry-leading distributed systems, including offline and online storage based on Paimon and batch and stream processing frameworks based on Flink, providing solid and reliable infrastructure support for massive data and large-scale business systems.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Job stats:
0
0
0
Categories:
Big Data Jobs
Engineering Jobs
Leadership Jobs
Tags: Architecture Big Data Distributed Systems Flink
Region:
North America
Country:
United States
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.
BI Developer jobsSr. Data Engineer jobsData Engineer II jobsBusiness Intelligence Analyst jobsPrincipal Data Engineer jobsStaff Data Scientist jobsStaff Machine Learning Engineer jobsData Manager jobsData Science Manager jobsPrincipal Software Engineer jobsData Science Intern jobsBusiness Data Analyst jobsJunior Data Analyst jobsData Analyst Intern jobsData Specialist jobsSoftware Engineer II jobsLead Data Analyst jobsResearch Scientist jobsSr. Data Scientist jobsDevOps Engineer jobsStaff Software Engineer jobsAI/ML Engineer jobsData Engineer III jobsSenior Backend Engineer jobsBI Analyst jobs
Git jobsAirflow jobsEconomics jobsOpen Source jobsLinux jobsComputer Vision jobsKafka jobsGoogle Cloud jobsJavaScript jobsMLOps jobsNoSQL jobsData Warehousing jobsTerraform jobsPhysics jobsKPIs jobsRDBMS jobsPostgreSQL jobsScikit-learn jobsBanking jobsHadoop jobsScala jobsGitHub jobsData warehouse jobsStreaming jobsPandas jobs
Classification jobsR&D jobsBigQuery jobsDistributed Systems jobsOracle jobsPySpark jobsdbt jobsLooker jobsCX jobsScrum jobsReact jobsRAG jobsMicroservices jobsRobotics jobsJira jobsRedshift jobsIndustrial jobsSAS jobsData Mining jobsNumPy jobsPrompt engineering jobsGPT jobsELT jobsMySQL jobsData strategy jobs