大数据开发工程师
Tasks
- Build data asset catalog
- Build offline data pipelines
- Create data development coding standards
- Define warehouse layer architecture
- Design data warehouse dimensional models
- Develop ETL jobs with Hive and Spark SQL
- Develop data quality validation jobs
- Develop real time stream processing jobs
- Establish data warehouse architecture principles
- Generate data service interfaces
- Improve query performance
- Integrate data sources with Flink and Spark
- Monitor data production timeliness
- Optimize SQL performance
- Optimize jobs
- Optimize scheduling pipelines
- Write data to big data platforms
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Flink | Apache Hive | Apache Kettle | Apache Spark | Apache Spark SQL | Data Processing | Data Warehouse | DataWorks | Dimensional Modeling | Distributed Computing | Distributed Storage | DolphinScheduler | ETL | Java | Job Scheduling | Python | Real Time | Real-time Data | Real-time Data Processing | SQL | SQL Optimization | Shell | Spark SQL | Sqoop
Education
Bachelor of Arts | Bachelor of Engineering | Bachelor of Science
Roles
Big Data Developer | Data Developer | Data Engineer | Developer | Engineer
Related jobs
-
Mid-level Full TimeHangzhou17h ago
-
Mid-level Full Time深圳17h ago
-
【27届实习】云原生Ai平台研发工程师-杭州 CNY 37K-37KArgo Workflow | Computer networks | Containers | Data Structures | GoEntry-level Internship杭州18h ago
-
【27届实习】数据挖掘工程师 CNY 25K-37KData Structures | Deep learning | Distributed machine learning | Go | JavaFull-time conversion opportunityEntry-level Internship Temporary上海18h ago
-
【27届实习】Dba(数据库工程师) CNY 37K-37KAutomation | Database Architecture | Distributed Systems | MongoDB | MySQLFull-time conversion opportunityEntry-level Internship上海18h ago
-
Ai算法工程师 CNY 180K-300KConvolutional Neural Networks | Data Mining | Data Warehouse | Data labeling | Deep learningMid-level Full Time东莞19h ago
-
AI Native Database | AI-native | APIs | Accelerated computing | Agentic AISenior-level Full TimeChina, Beijing1d ago
-
Sr. Associate Director, Data and Analytics CNY 240K-360KAgile | Alerting | Analytics engineering | Automated testing | CI/CDMid-level Full TimeGuangzhou, Guangdong, China2d ago
-
Senior-level Full TimeChina2d ago
-
Senior-level Full TimeChina2d ago
-
Senior Machine Learning Engineer CNY 360K-600KAWS | Agent architecture | Blender | ComfyUI | Context engineeringSenior-level Full TimeChina2d ago
-
Bash | Data Ingestion | Data Processing | Docker | GCPAsynchronous work culture | Friendly laid-back atmosphereMid-level Full TimeShanghai, China2d ago
-
Computer Graphics | Computer Vision | CoreML | Deep learning | Diffusion ModelsSenior-level Full TimeBeijing, Beijing, China2d ago
-
CUDA | DeepSpeed | Distributed Training | FSDP | Gradient CheckpointingEntry-level Full TimeBeijing, Beijing, China2d ago
-
Associate Data Analyst CNY 240K-300KData Cleansing | Data Modeling | Data Transformation | Data Visualization | Performance TuningMid-level Full TimeChengdu, Manulife Information and Technology Center, …2d ago
-
Senior-level Full TimeBeijing, China2d ago
-
Entry-level Internship深圳2d ago
-
Entry-level InternshipBeijing,Beijing,China3d ago
-
(Sr) Cloud & Data Engineer CNY 192K-240KAWS | Automation | CI/CD | Container Security | Data ModelingMid-level Full TimeBeijing, Beijing, CN3d ago
-
[Growth Engineering] Staff Back-end Engineer I CNY 144K-240KAnomaly Detection | CI/CD | Containerization | FastAPI | Graph DatabaseSenior-level Full TimeShanghai, China3d ago
-
[Growth Engineering] Staff Back-end Engineer I CNY 144K-240KCI/CD | Containerization | FastAPI | Graph Database | Inference ServerSenior-level Full TimeShanghai, China3d ago
-
C# | C++ | Computer Vision | Debugging | Deep learningSenior-level Full TimeChina, Shanghai3d ago
-
Entry-level Internship Part TimeShanghai - Daning Main Blg, China3d ago
-
Senior Data & AI Platform Engineer CNY 144K-240KAWS | Azure | BI Gateway | DAX | Data ManagementMon Fri schedule | Office Work Schedule | Team collaboration with technical and business stakeholdersSenior-level Full TimeChina - Shanghai - Xin Jin …3d ago
-
大数据开发(数据挖掘、数据测试、java) CNY 25K-37KApache Kafka | Apache Spark | Data Mining | Data Modeling | Data WarehousingMid-level Full Time保定3d ago