Data Engineer (Spark)
Tasks
- Collaborate with teams on data requirements
- Design data pipelines for streaming and batch
- Develop and maintain data processing platform
- Leverage AWS for infrastructure scaling
- Manage data lake using Iceberg
- Monitor and troubleshoot platform performance and accuracy
- Optimize data workflows for ingestion processing storage
- Write and maintain Python code for data processing
Perks/Benefits
- Career development
- Conference support
- Flexible work arrangements
- Integration budget
- Knowledge sharing
- Language classes
- Medical coverage
- Paid time off
- Sponsored training
- Sports packages
- Team building events
- Wellness support
- Work equipment provided
- Work remote options
Skills/Tech-stack
AWS | Apache Airflow | Apache Iceberg | Apache Kafka | Apache NiFi | Apache Spark | Cloudera | DBT | Data Governance | Data Management | Data Modeling | Databricks | Dimensional Modeling | Docker | Google BigQuery | Java | Kubeflow | Looker | MLOps | MLflow | Python | Scala
Education
Roles
Related jobs
-
Python Robot QA INR 1500K-2541KAPI Testing | Agile | Azure DevOps | Backup and Recovery | Case managementFlexible work timing | Hybrid work modelSenior-level Full TimeIN-AP-Hyderabad R4h ago
-
Software Engineer (Leadership) - Machine Learning GBP 100K-130KAgent Orchestration | Apache Hive | Apache Spark | Bias Mitigation | Data AnalysisSenior-level Full TimeZurich, Switzerland | Remote, UK | … R5h ago
-
Apache Spark | CI/CD | Cloud | Dataiku | HadoopCareer growth opportunities | Employee representative council | Health insurance | Meal vouchers | Profit sharingSenior-level Full TimeParis, IDF, France R6h ago
-
Test Data Analysis Technology Development Engineer JPY 3120K-5268KAI | Adaptive testing | Anomaly Detection | Data Analysis | Defect predictionHybrid work model | Inclusive workplace | Remote work 2 days per weekEntry-level Full TimeKodaira, Japan R7h ago
-
Senior Data Analytics Engineer (NGD) F/H/X EUR 45K-54KApache Airflow | CI/CD | DBT | GitLab | PythonCSE support | Flexible work options | Hackathons | Health insurance partial coverage | Public transport reimbursementSenior-level Full TimeIssy-les-Moulineaux, IDF, France R7h ago
-
CI/CD | Dataiku | GitLab CI | HDFS | HiveCareer development opportunities | Employee benefits | Health insurance | Meal vouchers | Profit sharing bonusesSenior-level Full TimeVilleneuve-d'Ascq, Hauts-de-France, France R7h ago
-
Data Engineer - Alternance - F/H EUR 24K-24KCloud platform | Data Engineering | Data Ingestion | Data Pipeline Monitoring | Data QualityE-learning access | Employee discount card | Meal voucher program | Telework 1 day per week | Training opportunitiesEntry-level Apprenticeship Full TimeGrenoble, Auvergne-Rhône-Alpes, France R8h ago
-
Data Engineer Cloud Confirmé / Senior (F/H) EUR 50K-65KAgile | Amazon S3 | Apache Spark | Azure | Azure DataEmployee stock ownership plan | Health insurance coverage | Maternity leave partial return schedule | Training budget | Vacation bonusSenior-level Full TimeLille, Hauts-de-France, France R8h ago
-
Data Engineer - H/F EUR 50K-58KAWS Glue | Ansible | Apache Spark | Azure Data | Azure Data FactoryHealth insurance | Maternity leave return 45 percent salary for 6 months | Public transport discount | Restaurant vouchers | Stock ownership planMid-level Full TimeRennes, Brittany, France R9h ago
-
Consultant Azure Data Engineer - H/F (CDI) EUR 50K-65KAgile | Azure Blob | Azure Blob Storage | Azure Data | Azure Data FactoryEmployee stock ownership | Equipment allowance | Flexible remote work | Health insurance | Modern officeSenior-level Full TimeParis, IDF, France R9h ago
-
Data pipeline | Debugging | Distributed Systems | Inference Optimization | KubernetesFlexible leave options | Inclusive parental leave | Remote within Australia | Wellbeing allowanceSenior-level Full TimeMelbourne, VIC, Australia R12h ago
-
AKS | Autogen | Azure DevOps | Azure Machine Learning | CI/CDCareer Development Programs | Hybrid work model | Recognition and rewards programs | Volunteering opportunities | Well-being programsSenior-level Full TimeMalaga, ES, 29590 R17h ago
-
Embedded AI Engineer, Lund SEK 660K-804KASIC design | C++ | Computer Vision | Deep learning | Embedded LinuxCompany bonus | Health insurance | Wellbeing initiatives | Wellness allowance | Work-life balanceSenior-level Full TimeSweden - Lund R17h ago
-
Data Engineer EUR 51K-78KCloud Composer | DBT | Dataflow | GCP | GitHub ActionsDiscounts on Company Services | Extra vacation days | Health insurance | Internal training | Leadership programsSenior-level Full TimeVilnius, Lithuania R17h ago
-
AWS | Apache Airflow | Apache Kafka | Cloud platform | Data ModelingCSE benefits | Occasional travel | Performance bonus | Profit-sharing bonus | Public transit reimbursementMid-level Full TimeParis R17h ago
-
Senior Data Science Engineer CAD 120K-150KDistributed Computing | Machine Learning | Python | R | SQLSenior-level Full TimeRemote - Canada R17h ago
-
Senior Machine Learning Engineer USD 134K-168KA/B | A/B Testing | B testing | CI/CD | DatabricksGaming license requiredSenior-level Full TimeRemote - US, United States R17h ago
-
Senior Software Data Engineer (Roads Team) USD 155K-222KAWS Athena | AWS EC2 | AWS EMR | AWS Glue | AWS IAMSenior-level Full TimeMapbox Minsk R1d ago
-
Software Development Engineer I (Data Engineer), HD Maps USD 157K-222KAWS | Apache Spark | D3.js | Data Visualization | GISFlexible work hours | Health care | Parental leave | Remote workSenior-level Full TimeMapbox Minsk R1d ago
-
Apache Spark | C plus plus | FastAPI | Next.js | PandasHybrid work schedule | In office work Monday to Wednesday | Remote work Thursday and FridayMid-level Full TimeHybrid - San Francisco, CA R1d ago
-
API Gateway | AWS | AWS CodeBuild | AWS CodePipeline | AWS FargateEmployee stock option plan | Flexible working options | Health insurance | Home-office allowance | Parental leaveSenior-level Full TimeRemote Israel R1d ago
-
Senior Cloud Architect, ML/AI EUR 84K-103KAPI Gateway | AWS CodeBuild | AWS CodePipeline | AWS Fargate | AWS GlueEmployee stock option plan | Flexible working options | Health insurance | Home-office allowance | Parental leaveSenior-level Full TimeRemote Netherlands R1d ago
-
Senior Cloud Architect, ML/AI GBP 100K-120KAPI Gateway | AWS | AWS Fargate | AWS Glue | AWS LambdaEmployee stock option plan | Flexible working options | Health insurance | Home-office allowance | Parental leaveSenior-level Full TimeRemote UK R1d ago
-
Senior Cloud Architect, ML/AI CAD 143K-202KAPI Gateway | AWS | AWS CodeBuild | AWS CodePipeline | AWS FargateEmployee stock option plan | Flexible working options | Health insurance | Home-office allowance | Parental leaveSenior-level Full TimeRemote Canada R1d ago
-
Senior Software Engineer - Agentic AI USD 145K-195KAWS | Azure | Cloud Computing | Distributed Systems | Docker401k matching | Health insurance | Hybrid work arrangement | Paid time off | Relocation assistanceSenior-level Full TimeBoston, Massachusetts, United States - Remote R1d ago