大数据平台工程师
Tasks
- Build lakehouse stream batch unified platform
- Construct vector storage and management architecture
- Deploy and monitor lakehouse and vector search platforms
- Design data metrics system and data quality monitoring
- Develop end to end data engineering from logs to vector engine
- Develop real time data pipelines with exactly once semantics
- Implement metadata lineage and SLA governance
- Integrate Kafka or Pulsar for real time ingestion
- Optimize lakehouse write and vector synchronization workflows
- Standardize lakehouse schema evolution and partition strategy
- Tune resources for performance and stability
Perks/Benefits
- N/A
Skills/Tech-stack
ACID | ANN | Apache Flink | Apache Hadoop | Apache Iceberg | Apache Kafka | Apache Paimon | Apache Pulsar | Apache Spark | Arrow | Checkpoint | Doris | Exactly once | Faiss | HDFS | HNSW | Hive | IVF) | Java | Lance | Milvus | Parquet | Python | Qdrant | Scala | Schema evolution | Trino | Watermark | Weaviate | YARN
Education
Bachelor of Arts | Bachelor of Engineering | Bachelor of Science
Related jobs
-
Mid-level Full TimeBeijing, China17h ago
-
Mid-level Full TimeChina Shanghai18h ago
-
Asynchronous programming | Dashboards | Data Observability | Data Validation | DatabasesMid-level Full TimeChina, Shanghai18h ago
-
Senior-level Full TimeWuxi, Jiangsu, China1d ago
-
Entry-level Internship上海2d ago
-
Entry-level Full Time上海2d ago
-
数据算法工程师（实习生） CNY 25K-37KC++ | Data Generation | Data Modeling | Data Transformation | Data cleaningInternshipEntry-level Internship上海2d ago
-
nlp算法工程师-2027届 CNY 25K-37KDeep learning | DeepSpeed | Fine Tuning | Information Retrieval | Language ProcessingEntry-level Internship武汉3d ago
-
AI Agent Engineer(Embededd Software Tooling)_ETAS CNY 240K-480KAgent architecture | C++ | Deep learning | Edge AI | Embedded SoftwareSenior-level Full TimeShanghai, Shanghai, China3d ago
-
AI Application Development Engineer CNY 180K-300KAgent systems | Artificial Intelligence | Computer Vision | Deep learning | Image ProcessingEntry-level Full TimeShenzhen, Guangdong Province, China3d ago
-
Access Control | Alerting | BigQuery | DBT | Data GovernanceAutonomy | Growth opportunities | High-impact work | TrustMid-level Full TimeChina3d ago
-
Senior-level Full TimeChina Shanghai3d ago
-
APIs | AWS | Agentic Workflows | Azure | Cloud platformSenior-level Full TimeChina, Shanghai3d ago
-
Senior Platform AI Engineer - Silicon Co-Design Group CNY 360K-540KAuthentication | Authorization | C# | C++ | CachingComprehensive benefits package | Family benefitsSenior-level Full TimeChina, Shanghai3d ago
-
Entry-level Full TimeCN-Beijing-Office, China3d ago
-
Mid-level Full TimeShanghai, CN, 2012033d ago
-
具身智能数据开发实习生 CNY 25K-37KAPI Development | Algorithms | Automation | Data Ingestion | Data StructuresEntry-level Internship上海3d ago
-
None Full Time上海4d ago
-
A/B | A/B Testing | Agent systems | Anomaly Detection | B testingEntry-level Internship上海4d ago
-
Senior-level Full Time北京4d ago
-
BPS & AI engineer_PS CNY 25K-37KArtificial Intelligence | BPS | Business Process | Business process improvement | Continuous ImprovementEntry-level Full TimeWuxi, Jiangsu, China4d ago
-
Lead Embedded Software Engineer CNY 349K-437KARM | BLE | C# | C++ | Embedded LinuxHybrid work model | Remote-friendly | Work from homeSenior-level Full TimeSuzhou, China R4d ago
-
数据开发工程师 CNY 240K-480KAirbyte | BigQuery | Cube.js | DBT | Data GovernanceAI tool subscriptions | API credits | Cloud credits | Flat organizationSenior-level Full Time深圳5d ago
-
数据平台开发工程师 CNY 180K-360KData Lake | Data Warehouse | Data Warehouse Modeling | Data pipeline | Delta LakeMid-level Full Time广州5d ago
-
Entry-level InternshipShenzhen5d ago