大数据平台工程师
Tasks
- Build lakehouse stream batch unified platform
- Construct vector storage and management architecture
- Deploy and monitor lakehouse and vector search platforms
- Design data metrics system and data quality monitoring
- Develop end to end data engineering from logs to vector engine
- Develop real time data pipelines with exactly once semantics
- Implement metadata lineage and SLA governance
- Integrate Kafka or Pulsar for real time ingestion
- Optimize lakehouse write and vector synchronization workflows
- Standardize lakehouse schema evolution and partition strategy
- Tune resources for performance and stability
Perks/Benefits
- N/A
Skills/Tech-stack
ACID | ANN | Apache Flink | Apache Hadoop | Apache Iceberg | Apache Kafka | Apache Paimon | Apache Pulsar | Apache Spark | Arrow | Checkpoint | Doris | Exactly once | Faiss | HDFS | HNSW | Hive | IVF) | Java | Lance | Milvus | Parquet | Python | Qdrant | Scala | Schema evolution | Trino | Watermark | Weaviate | YARN
Education
Bachelor of Arts | Bachelor of Engineering | Bachelor of Science
Related jobs
-
Mid-level Full Time上海3h ago
-
Mid-level Full Time深圳、上海3h ago
-
Mid-level Full Time北京3h ago
-
机器人大模型软件工程师 CNY 180K-300KAgent systems | Asynchronous programming | C++ | Context Scheduling | Conversation ManagementMid-level Full Time深圳3h ago
-
Mid-level Full Time广州3h ago
-
Entry-level Full Time深圳、上海、北京4h ago
-
Mid-level Full Time深圳、上海、北京、中国香港4h ago
-
Data Visualization | Data alignment | Data labeling | Data pipeline | Dataset integrationMid-level Full Time深圳、上海4h ago
-
Entry-level Full Time InternshipBeijing4h ago
-
多模态大模型算法工程师（Vlm / 自动驾驶方向） CNY 180K-264KAgent modeling | Autonomous Driving | Autoregressive modeling | BEV | Behavior ModelingEntry-level Full Time北京、苏州4h ago
-
Java服务端研发工程师 - 商业化 CNY 180K-420KData Analysis | Data Attribution | Distributed Computing | Doris | FlinkMid-level Full Time北京6h ago
-
Mid-level Full Time北京6h ago
-
大模型算法专家 CNY 240K-480KAgentic RL | Language Models | Large Language Models | Linux | Multimodal GenerationSenior-level Full Time北京6h ago
-
Java开发工程师（大数据方向） CNY 180K-420KApache Flink | Apache Spark | Data platform | Data platform architecture | IOMid-level Full Time武汉7h ago
-
Entry-level Full Time北京8h ago
-
Senior-level Full Time北京1d ago
-
Senior-level Full Time上海1d ago
-
【算法】多模态/大模型算法专家（上海） CNY 240K-480KAgent Framework | C plus plus | Computer Vision | Language Processing | LinuxSenior-level Full Time上海1d ago
-
Java技术专家（搜推模型样本数据平台方向） CNY 300K-480KApache Flink | Apache HBase | Apache Kafka | Apache Spark | C plus plusSenior-level Full Time北京、上海1d ago
-
【算法】计算机视觉算法专家（上海） CNY 240K-480KAnomaly Detection | C++ | Computer Vision | Deep learning | Fine-grained classificationSenior-level Full Time上海1d ago
-
大模型应用算法工程师/专家 CNY 240K-480KC++ | Computer Vision | Deep learning | Direct Preference Optimization | Human Computer DialogueSenior-level Full Time上海、北京1d ago
-
【算法】计算机视觉算法专家（杭州） CNY 240K-480K3D Vision | Anomaly Detection | C++ | Computer Vision | Image RetrievalSenior-level Full Time杭州1d ago
-
机器学习特征/样本数据工程研发 CNY 180K-300KC++ | Caching | Data Engineering | Data Governance | Feature EngineeringEntry-level Full Time北京、上海1d ago
-
Senior Perception Algorithm Engineer CNY 400K-540K3D Geometry | BEV | C# | C++ | Camera CalibrationSenior-level Full Time5-8F TOWER C, 788 JINZHONG ROAD, …2d ago
-
Principal Perception Algorithm Engineer CNY 400K-540K3D Geometric Transformations | BEV Depth | BEV Det | C# | C++Senior-level Full Time5-8F TOWER C, 788 JINZHONG ROAD, …2d ago