Data Engineer (Spark)
Tasks
- Collaborate with teams on data requirements
- Design data pipelines for streaming and batch
- Develop and maintain data processing platform
- Leverage AWS for infrastructure scaling
- Manage data lake using Iceberg
- Monitor and troubleshoot platform performance and accuracy
- Optimize data workflows for ingestion processing storage
- Write and maintain Python code for data processing
Perks/Benefits
- Career development
- Conference support
- Flexible work arrangements
- Integration budget
- Knowledge sharing
- Language classes
- Medical coverage
- Paid time off
- Sponsored training
- Sports packages
- Team building events
- Wellness support
- Work equipment provided
- Work remote options
Skills/Tech-stack
AWS | Apache Airflow | Apache Iceberg | Apache Kafka | Apache NiFi | Apache Spark | Cloudera | DBT | Data Governance | Data Management | Data Modeling | Databricks | Dimensional Modeling | Docker | Google BigQuery | Java | Kubeflow | Looker | MLOps | MLflow | Python | Scala
Education
Roles
Related jobs
-
Apache Spark | CI/CD | Cloud | Dataiku | HadoopCareer growth opportunities | Employee representative council | Health insurance | Meal vouchers | Profit sharingSenior-level Full TimeParis, IDF, France R5h ago
-
Test Data Analysis Technology Development Engineer JPY 3120K-5268KAI | Adaptive testing | Anomaly Detection | Data Analysis | Defect predictionHybrid work model | Inclusive workplace | Remote work 2 days per weekEntry-level Full TimeKodaira, Japan R5h ago
-
Senior Data Analytics Engineer (NGD) F/H/X EUR 45K-54KApache Airflow | CI/CD | DBT | GitLab | PythonCSE support | Flexible work options | Hackathons | Health insurance partial coverage | Public transport reimbursementSenior-level Full TimeIssy-les-Moulineaux, IDF, France R5h ago
-
CI/CD | Dataiku | GitLab CI | HDFS | HiveCareer development opportunities | Employee benefits | Health insurance | Meal vouchers | Profit sharing bonusesSenior-level Full TimeVilleneuve-d'Ascq, Hauts-de-France, France R5h ago
-
Data Engineer - Alternance - F/H EUR 24K-24KCloud platform | Data Engineering | Data Ingestion | Data Pipeline Monitoring | Data QualityE-learning access | Employee discount card | Meal voucher program | Telework 1 day per week | Training opportunitiesEntry-level Apprenticeship Full TimeGrenoble, Auvergne-Rhône-Alpes, France R6h ago
-
Data Engineer Cloud Confirmé / Senior (F/H) EUR 50K-65KAgile | Amazon S3 | Apache Spark | Azure | Azure DataEmployee stock ownership plan | Health insurance coverage | Maternity leave partial return schedule | Training budget | Vacation bonusSenior-level Full TimeLille, Hauts-de-France, France R7h ago
-
Data Engineer - H/F EUR 50K-58KAWS Glue | Ansible | Apache Spark | Azure Data | Azure Data FactoryHealth insurance | Maternity leave return 45 percent salary for 6 months | Public transport discount | Restaurant vouchers | Stock ownership planMid-level Full TimeRennes, Brittany, France R7h ago
-
Consultant Azure Data Engineer - H/F (CDI) EUR 50K-65KAgile | Azure Blob | Azure Blob Storage | Azure Data | Azure Data FactoryEmployee stock ownership | Equipment allowance | Flexible remote work | Health insurance | Modern officeSenior-level Full TimeParis, IDF, France R7h ago
-
Data pipeline | Debugging | Distributed Systems | Inference Optimization | KubernetesFlexible leave options | Inclusive parental leave | Remote within Australia | Wellbeing allowanceSenior-level Full TimeMelbourne, VIC, Australia R11h ago
-
AKS | Autogen | Azure DevOps | Azure Machine Learning | CI/CDCareer Development Programs | Hybrid work model | Recognition and rewards programs | Volunteering opportunities | Well-being programsSenior-level Full TimeMalaga, ES, 29590 R15h ago
-
Embedded AI Engineer, Lund SEK 660K-804KASIC design | C++ | Computer Vision | Deep learning | Embedded LinuxCompany bonus | Health insurance | Wellbeing initiatives | Wellness allowance | Work-life balanceSenior-level Full TimeSweden - Lund R15h ago
-
Data Engineer EUR 51K-78KCloud Composer | DBT | Dataflow | GCP | GitHub ActionsDiscounts on Company Services | Extra vacation days | Health insurance | Internal training | Leadership programsSenior-level Full TimeVilnius, Lithuania R15h ago
-
Senior Software Data Engineer (Roads Team) USD 155K-222KAWS Athena | AWS EC2 | AWS EMR | AWS Glue | AWS IAMSenior-level Full TimeMapbox Minsk R1d ago
-
Software Development Engineer I (Data Engineer), HD Maps USD 157K-222KAWS | Apache Spark | D3.js | Data Visualization | GISFlexible work hours | Health care | Parental leave | Remote workSenior-level Full TimeMapbox Minsk R1d ago
-
API Gateway | AWS | AWS CodeBuild | AWS CodePipeline | AWS FargateEmployee stock option plan | Flexible working options | Health insurance | Home-office allowance | Parental leaveSenior-level Full TimeRemote Israel R1d ago
-
Senior Cloud Architect, ML/AI EUR 84K-103KAPI Gateway | AWS CodeBuild | AWS CodePipeline | AWS Fargate | AWS GlueEmployee stock option plan | Flexible working options | Health insurance | Home-office allowance | Parental leaveSenior-level Full TimeRemote Netherlands R1d ago
-
Senior Cloud Architect, ML/AI GBP 100K-120KAPI Gateway | AWS | AWS Fargate | AWS Glue | AWS LambdaEmployee stock option plan | Flexible working options | Health insurance | Home-office allowance | Parental leaveSenior-level Full TimeRemote UK R1d ago
-
Senior Cloud Architect, ML/AI CAD 143K-202KAPI Gateway | AWS | AWS CodeBuild | AWS CodePipeline | AWS FargateEmployee stock option plan | Flexible working options | Health insurance | Home-office allowance | Parental leaveSenior-level Full TimeRemote Canada R1d ago
-
Data Engineer (DBT, Snowflake, BigQuery) EUR 35K-41KApache Airflow | Automation | Azure Data | Azure Data Factory | BigQueryFlexible benefits | Indefinite contract | Telework support | Training and certifications accessMid-level Full TimeRemote, Spain R1d ago
-
Data Engineer (DBT, Snowflake, BigQuery) EUR 35K-41KApache Airflow | Automation | Azure Data | Azure Data Factory | BigQueryAccess to training and certifications | Flexible compensation benefits | Indefinite contract | Intensive work schedule in July and August | Telework supportMid-level Full TimeBarcelona, Spain R1d ago
-
Data Engineer (DBT, Snowflake, BigQuery) EUR 35K-41KAmazon Redshift | Apache Airflow | Automation | Azure Data | Azure Data FactoryFlexible benefits | Indefinite contract | July and August reduced hours | Telework support | Training accessMid-level Full TimeSantiago de Compostela, Spain R1d ago
-
Experienced Data Engineer - Streaming Platform EUR 60K-80KAWS Kinesis | Apache Beam | Apache Flink | Apache Pulsar | Apache SparkChild day care | Gym membership | Lunch vouchers | Premium healthcare | Relocation supportMid-level Full TimeParis R1d ago
-
ML Engineering Intern USD 70K-140KAPI Integration | CI/CD | Command Line | Command-line Interface | Data PipelinesFlexible schedule | Fully remote | Independent contractor | Optional office visitsEntry-level InternshipUnited States - Remote R1d ago
-
Software Engineer, Perception and Prediction Evaluation USD 155K-213KAWS | Amazon Batch | Amazon ECS | Amazon S3 | Apache AirflowCatered meals | Daily Drinks | Equity awards | Flexible hours | Health and wellness benefitsMid-level Full TimeRemote US & Canada R1d ago
-
Data Engineer USD 133K-162KAWS | Apache Airflow | Azure | CI/CD | DBTEnglish classes | Holiday week off | Internal library access | Remote-first work culture | Vacation 3 weeksMid-level Full TimeBuenos Aires Province, Argentina R1d ago