数据湖开发工程师
Tasks
- Build lakehouse architecture with Iceberg
- Design data lake platform
- Develop and maintain data lake services
- Implement data encryption and tokenization
- Implement data masking and access control
- Integrate data lake with AI and analytics
- Optimize storage and compute performance
- Set up data governance capabilities
Perks/Benefits
- N/A
Skills/Tech-stack
Amundsen | Apache Flink | Apache Kafka | Apache Ranger | Apache Sentry | Apache Spark | Atlas | DAMA | DCMM | Data Governance | Data Lineage | Data Quality | Datahub | DiskANN | Distributed Storage | Docker | Elasticsearch KNN | Embedding | Field level encryption | HNSW | Hudi | Hybrid search | IVF-PQ | Iceberg | Java | KMS | Kubernetes | Metadata Management | Milvus | Object storage | PGVector | Python | Qdrant | ScaNN | Scala | TDE | Tokenization | Vespa | Weaviate
Education
Roles
Big Data Engineer | Data Engineer | Data Lake Engineer | Engineer
Related jobs
-
Mid-level Full Time北京 R9h ago
-
Mid-level Full Time北京 R10h ago
-
Mid-level Full Time北京 R10h ago
-
MiMo-大模型训练框架开发工程师 CNY 240K-480KC++ | CI/CD | DeepSpeed | Distributed Training | GPU Memory OptimizationEntry-level Full Time北京 R1d ago
-
Entry-level Full Time北京 R1d ago
-
Mid-level Full Time北京 R1d ago
-
具身智能算法工程师-模型 CNY 500K-500KDeep learning | Distributed Training | IQL | Inference Optimization | Isaac LabMid-level Full Time北京 R1d ago
-
Senior-level Full Time5-8F TOWER C, 788 JINZHONG ROAD, … R2d ago
-
AI Engineer USD 100K-200KAPIs | Agentic Frameworks | Artificial Intelligence | Backtesting | Data integrationCollaborative work environment | Equity options | Innovation-driven cultureMid-level Full TimeShanghai, Shanghai, China R2d ago
-
Senior Software Engineer (RAG Backend Developer) CNY 120K-180KA/B | A/B Testing | ABAC | Audit Logging | B testingSenior-level Full TimeGuangzhou, Guangdong, China R3d ago
-
Mid-level Full Time广州 R19d ago
-
Ai 院--多模态团队--多模态理解算法研究员-强化学习方向 CNY 240K-480KDPO | Data Preprocessing | Data cleaning | DeepSpeed | Distributed TrainingSenior-level Full Time北京 R23d ago
-
Lead Technical Support Engineer - AI / ML CNY 144K-240KAPI Integration | Agent Frameworks | C plus plus | Cause analysis | Cloud ComputingHybrid work model | Travel for customer workshops | Work from homeSenior-level Full TimeBeijing, China R24d ago
-
AI基础设施研发工程师（Sandbox / 容器化)-MiMo CNY 180K-420KAppArmor | Argo Workflows | CI/CD | CPU resource scheduling | CgroupMid-level Full Time北京 R25d ago
-
Entry-level Full Time北京、上海 R27d ago
-
Behavior Cloning | C++ | Cloud processing | Computer Vision | ControlEntry-level Internship北京、上海 R29d ago
-
AI platforms | API Development | Artificial Intelligence | Cloud AI | Cloud AI PlatformsMid-level Full TimeRemote, China R30d ago
-
AI ML Engineer CNY 280K-360KAWS | Azure | C++ | Cloud Computing | Computer VisionPerformance bonuses | Professional development opportunities | Remote workMid-level Full TimeShenzhen, Guangdong Province, China R1mo ago
-
AI工程师-Agent Memory & RAG 方向（成都） CNY 240K-480KBERT | Chroma | Cross-Encoder | Embedding Models | FaissSenior-level Full Time成都 R1mo ago
-
AI工程师-Agent Memory & RAG 方向（武汉） CNY 240K-480KAlgorithms | BERT | Chroma | Cross-Encoder | Data StructuresSenior-level Full Time武汉 R1mo ago
-
AI工程师-Agent Memory & RAG 方向（北京） CNY 240K-480KBERT | Chroma | Cross-Encoder | Embedding Models | FaissSenior-level Full Time北京 R1mo ago
-
AWS | Agent Orchestration | Agent systems | Azure | DockerMid-level Full TimeShenzhen, Guangdong, China R1mo ago
-
AVP, AI Solution Lead CNY 360K-600KCloud Computing | DataOps | DevOps | Flutter | Generative AIContinuous professional development | Flexible workingSenior-level Full TimeGuangzhou, Guangdong, China R1mo ago
-
AWQ | AWS | Accelerate | Azure | BatchingMid-level Full TimeShenzhen, Guangdong, China R1mo ago
-
Generative AI - ML System Engineering CNY 360K-600KC++ | CUDA | Compilation | Data pipeline | Diffusion ModelsFully remote option | On-site work flexibilitySenior-level Full TimeShanghai R1mo ago