Big Data Engineer
Tasks
- Build data pipelines
- Design data pipeline architecture
- Design scalable cloud data infrastructure
- Ensure high availability of data infrastructure
- Implement data governance practices
- Implement data validation and cleansing
- Integrate large volume data from multiple sources
- Monitor pipeline health and performance
- Optimize data processing performance
- Perform data enrichment
- Provide mentorship to junior data engineers
- Troubleshoot data pipeline issues
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Apache Airflow | Apache Spark | Azure | Cloud platform | Data Transformation | Data integration | Delta Lake | Distributed Computing | ETL | Google Cloud | Google Cloud Platform | Hadoop | Java | Parquet | PySpark | Python | SQL | Scala
Education
Roles
Related jobs
-
Mid-level Full TimeTel Aviv-Jaffa, Tel Aviv District, IL21h ago
-
Algorithm Optimization | Deep learning | Embedded Systems | Image Processing | PythonFlexible work environment | Work-life balanceSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL22h ago
-
Mid-level Full TimeTel Aviv-Yafo, Tel Aviv District, Israel1d ago
-
Algorithms | Code Analysis | Compilers | Language programming | LintersMid-level Full TimeTel Aviv/ Netanya, Israel1d ago
-
Senior-level Full TimeRamat Gan, Israel1d ago
-
Senior-level Full TimeTel Aviv District, Israel1d ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Diverse and inclusive workplace | Flexible, hands-off management | Remote distributed settingMid-level Full TimeTel Aviv, Israel1d ago
-
Mid-level Full TimeBnei Brak, Tel Aviv District, IL1d ago
-
AI orchestration | AWS | Apache Spark | Data Modeling | Data TransformationSenior-level Full TimeKfar Saba, Israel2d ago
-
Senior-level Full TimeJerusalem, Israel2d ago
-
Airbyte | Apache Airflow | DBT | Data Modeling | Data WarehousingFully remote | Home office setup budget | Language learning sessions | Paid sick days | Paid time offSenior-level Full TimeIsrael R2d ago
-
Data Analysis | Data Classification | MATLAB | Performance Metrics | PythonMid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL2d ago
-
Mid-level Full TimeTel-Aviv, Israel, IL2d ago
-
Senior-level Full TimeBnei Brak, Tel Aviv District, IL2d ago
-
AI API | API Integration | Anthropic API | Automation Pipelines | Google AIOnsite N/A | Professional development opportunities | Remote-friendlySenior-level Full TimeRamat Gan, Israel, IL2d ago
-
Code Versioning | Computer Vision | Continuous integration | Data Analysis | Data PipelinesSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL2d ago
-
Access Control | Automation | CI/CD | Cloud platform | CloudflareCollaborative work culture | Flexible work environment | Mental health support | Wellness supportMid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL2d ago
-
AWS | CI/CD | CUDA | Containerization | Data StreamingOpen source contributionsMid-level Full TimeRamat Gan, Israel3d ago
-
Artificial Intelligence | Computer Vision | Flask | Generative AI | JupyterAcademic hours | Evening courseEntry-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL3d ago
-
Mid-level Full TimeRa'anana, Center District, IL3d ago
-
Apache Spark | Data Modeling | Data Warehousing | Event Streaming | LLMSenior-level Full TimeHerzliya3d ago
-
Big Data | Data Analysis | Machine Learning | Python | Scientific ComputingMid-level Full TimeJerusalem, Israel3d ago
-
Anomaly Detection | Dashboards | Data Modeling | Data Quality | Data ReconciliationContinuous learning opportunities | Education support | Employee discounts | Family-friendly benefits | Flexible time offMid-level Full TimeIsrael3d ago
-
Mid-level Full TimeRamat-Gan, Tel Aviv District, IL3d ago
-
C++ | CUDA | CUDA profiling | Collective communication | Communication Compute OverlapSenior-level Full TimeIsrael, Tel Aviv R3d ago