【集团】大数据运维
Tasks
- Build adhoc batch and hybrid analysis engines
- Build real time and batch data integration
- Design data platform architecture
- Develop and maintain distributed microservices
- Develop core platform modules
- Develop data lake based data processing
- Develop data lineage and metadata collection
- Develop real time and batch data development
- Implement data quality evaluation system
- Implement job orchestration
- Implement job scheduling engine
- Set up data quality metrics monitoring and alerting
Perks/Benefits
- N/A
Skills/Tech-stack
Airflow | Batch Processing | ClickHouse | Data Lake | Data integration | DolphinScheduler | Doris | Flink | HBase | HDFS | Hadoop | Hive | Java | Kafka | MapReduce | MySQL | PostgreSQL | Presto | Python | Real Time | Real-time Processing | Redis | SQL | Spark | Spring Boot | Spring Cloud | Stream processing | Time processing | YARN
Education
Related jobs
-
Mid-level Full Time深圳5h ago
-
Entry-level Full Time合肥6h ago
-
None Full Time上海6h ago
-
具身数据策略与质控工程师 CNY 25K-37KBenchmarking | Computer Vision | Data Quality | Data Quality Management | Data cleaningEntry-level Full Time北京6h ago
-
Mid-level Full Time北京6h ago
-
数据Infra工程师 (流式) CNY 180K-300KApache Spark | Automated Data Labeling | Data labeling | Data pipeline | Deep learningMid-level Full Time北京6h ago
-
数据Infra工程师 (具身) CNY 180K-300KApache Spark | Deep learning | Distributed Systems | Error Handling | FastAPIMid-level Full Time北京6h ago
-
Senior-level Full Time北京6h ago
-
Mid-level Full Time北京6h ago
-
Avp, Mgoc Tno Analytics, Gra CNY 300K-420KApplication development | Dash | Data Analysis | Data Visualization | DocumentationContinuous professional development | Flexible working | Inclusive work environmentExecutive-level Full TimeGuangzhou, Guangdong, China R14h ago
-
Principal Data Engineer Data Technology CNY 336K-510KAPIs | AWS | Apache Kafka | Apache Spark | CI/CDSenior-level Full TimeGuangzhou, Guangdong, China15h ago
-
Application Engineer - Associate CNY 25K-37KAgent workflows | Application development | Autogen | Git | LLM Application DevelopmentEntry-level Part TimeShanghai, China21h ago
-
Principal Data Engineer Data Technology CNY 336K-510KAPI Design | AWS | Apache Kafka | Apache Spark | Automated testingSenior-level Full TimeGuangzhou, GD, CN, 5106201d ago
-
Lead Data Scientist CNY 435K-510KAI Agents | AWS | Azure | CI/CD | Cloud ComputingHealthcare | Paid volunteering days | Retirement planning | Wellbeing initiativesSenior-level Full TimeCHN-Beijing-Zhong Guan Cun No.1, China1d ago
-
C# | C++ | CUDA | Computer Vision | GazeboCareer development | Hands-on experienceEntry-level Full TimeChina, Shanghai1d ago
-
Entry-level Internship北京1d ago
-
大模型预训练数据算法工程师 CNY 216K-360KAblation Study | Data Deduplication | Data Engineering | Data Filtering | Data ParsingMid-level Full Time北京、上海1d ago
-
Apache Flink | Apache Spark | Big Data | CI/CD | CassandraMid-level Full TimeShanghai1d ago
-
Analysis and Design | Big Data | Cassandra | Distillation | Distributed SystemsSenior-level Full TimeShanghai1d ago
-
Apache Flink | Apache Kafka | Apache Spark | Artificial Intelligence | CI/CDMid-level Full TimeShanghai1d ago
-
Senior-level Full Time北京1d ago
-
大模型算法工程师（多模态）-校招【27届】 CNY 240K-480KAB Testing | CLIP | Computer Vision | Contrastive Learning | Cross-modal retrievalSenior-level Full Time北京、杭州1d ago
-
Senior Software Engineer, Perception (Robotics) CNY 360K-510K3D Detection | Auto-labeling | BEV | Bayesian Filtering | C++Senior-level Full TimeShanghai, China1d ago
-
Enablement Lead - Data & AI Solution Enablement CNY 144K-240KAPI Integration | Agent automation | Artificial Intelligence | Data analytics | Low CodeSenior-level Full TimeShanghai, Shanghai, CN, 2012031d ago
-
Principal Perception Engineer CNY 360K-600KC++ | Computer Vision | Edge inference | Knowledge Distillation | Model ConversionSenior-level Full Time5-8F TOWER C, 788 JINZHONG ROAD, …2d ago