Lead Data Engineer (F/H)
Tasks
- Analyze pipeline performance
- Conduct code reviews and validate technical designs
- Contribute to data platform evolution
- Create unit and integration tests
- Define data pipeline architecture standards
- Ensure data pipeline robustness in production
- Implement data observability using logs monitoring metrics
- Implement idempotent and resumable data processing
- Implement software engineering practices for data
- Improve processing resilience
- Optimize compute and storage costs
- Perform performance and cost optimization
- Reduce technical complexity and technical debt
- Review critical pipeline architectures
- Set up CI CD for pipelines and notebooks
- Simplify existing architecture
- Support performance and scalability issue resolution
Perks/Benefits
Skills/Tech-stack
Apache Spark | Backfill | CI/CD | Compute Optimization | Cost Optimization | Data Modeling | Data Quality | Data pipeline | Idempotence | Integration Testing | Logging | Metrics | Monitoring | Observability | Python | Resilience | SQL | Stored Procedures | Unit Testing
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Senior Data Engineer - Data Factory (f/m/d) EUR 46K-55KAWS Glue | Airflow | Amazon EMR | Amazon S3 | Apache KafkaFlexible office location provided equipment | Inclusive work culture | International career opportunities | Technical certifications | Telecommuting 2 days per weekSenior-level Full TimeParis, Paris, France R18h ago
-
Analytics Engineer - Pricing data (f/m/d) EUR 45K-48KAWS | Airflow | BigQuery | CI/CD | DBTEquipment provided | International career opportunities | Technical certifications support | Training and development | Two days remote work per weekMid-level Full TimeLille, Upline, Nord, France; Paris, Paris, … R1d ago
-
Analytics Platforms | Apache Airflow | BigQuery | Cloud platform | DBTAnnual off-site events | Co-working allowance | Equity (ESOP) | Flexible asynchronous schedule | Fully remoteSenior-level Full TimeFrance R1d ago
-
Data Scientist confirmé / AI Engineer EUR 50K-55KAzure | CI/CD | Docker | GCP | Gemini APIEmployee Committee | Health insurance | Meal vouchers | Remote work up to 2 days per weekMid-level Full TimeCourbevoie, IDF, France R1d ago
-
Apache Spark | Azure | CI/CD | Data Quality | DatabricksCareer training and certifications | Employee representative council | Health insurance | Meal vouchers | Profit sharingSenior-level Full TimeNantes, Pays de la Loire, France R1d ago
-
Apache Hadoop | Apache Hive | Apache Kafka | Apache Spark | Azure DataBonus vacation | Career development opportunities | Employee representative council | Health insurance | Meal vouchersSenior-level Full TimeNantes, Pays de la Loire, France R1d ago
-
Machine Learning Engineer (W/M/D) EUR 55K-70KAWS SageMaker | Artificial Intelligence | Azure Machine Learning | DVC | Data PreprocessingAdditional paid time off | Employee feedback survey | Family care policy | Flexible work environment | Health insuranceMid-level Full TimeParis, IDF, France R1d ago
-
Data Engineer Confirmé (H/F) EUR 45K-48KAPI | AWS | Azure | ESB | ETLCSE benefits | Coding dojo events | Flexible work hours | Health insurance | Meal ticketsMid-level Full TimeLYON, France; LYON R2d ago
-
AI Coding Assistants | AI coding | Alerting | BigQuery | CI/CDContinuous learning | Flexible working arrangements | Fully remote | High autonomy | International engineering teamSenior-level Full TimeFrance R2d ago
-
AWS | Alerting | Amazon SageMaker | CI/CD | Cost OptimizationFlexible engagement | Fully remote | International collaboration | Professional growthSenior-level Contract Full TimeFrance R2d ago
-
Experienced Data Engineer - Streaming Platform EUR 55K-95KAmazon Kinesis | Apache Flink | Apache Kafka | Apache Pulsar | AvroChild day care facilities | Gym membership | Lunch voucher | Premium healthcare coverage | Relocation packageMid-level Full TimeParis R3d ago
-
Airbyte | BigQuery | DBT | DBT Cloud | Data AutomationFlexible remote work | Paris officeMid-level Full TimeParis R3d ago
-
Azure DevOps | Data Modeling | DevOps Tools | Git | No CodeCompany events | Holiday events | Ski weekend | Summer university | Telework 2 days per weekMid-level Full TimeParis R3d ago
-
Stage Machine Learning EUR 15K-15KApache Spark | Big Data | Deep learning | Hadoop | KerasDurable Mobility Allowance | Flexible remote work | Knowledge sharing sessions | Linux computer option | Mac computer optionEntry-level Full TimeParis, Île-de-France, France R4d ago
-
AWS | Azure | Cloud platform | DevOps | ETLEmployee representative council | Health insurance | Meal vouchers | Mentorship | Profit-sharing bonusSenior-level Full TimeColomiers, Occitanie, France R4d ago
-
Automation | BigQuery | DBT | Dashboard Development | Data GovernanceCareer progression | Compensatory time RTT | Flexible working hours | Health insurance | Learning resourcesSenior-level Full TimeFrance R5d ago
-
BI | DBT | Data Governance | Data Lineage | Data ModelingDental insurance | Flexible working arrangements | Fully remote | Health insurance | Professional growthMid-level Full TimeFrance R5d ago
-
API | AWS | Anthropic | CRM | Data ModelingEmployee stock ownership | Hybrid work | Maternity leave top-up | Training opportunitiesEntry-level Full TimeBordeaux, Nouvelle-Aquitaine, France R5d ago
-
AWS | Airflow | Apache Spark | Azure | CI/CDCooptation bonus | Employee representative council | Great place to work environment | Health insurance | Meal vouchersSenior-level Full TimeLimonest, Auvergne-Rhône-Alpes, France R5d ago
-
AWS | Airflow | Azure | CI/CD | DBTCareer development | Employee committee benefits | Health insurance | Meal vouchers | Mentorship programSenior-level Full TimeAix-en-Provence, Provence-Alpes-Côte d'Azur, France R5d ago
-
AWS | Apache Airflow | Apache Spark | Azure | CI/CDCSE | Cooptation bonus | Employee Incentive Plan | Health insurance | Holiday bonusSenior-level Full TimeCourbevoie, IDF, France R5d ago
-
AWS Glue | Amazon Web Services | Apache Airflow | Apache Spark | AzureCareer development opportunities | Cooptation bonus | Employee representative council | Great Place to Work certified | Health insuranceSenior-level Full TimeLimonest, Auvergne-Rhône-Alpes, France R5d ago
-
AWS | AWS Glue | Airflow | Azure | Azure DataCareer development | Cooption program | Diversity and inclusion initiatives | Employee representatives council | Health insuranceSenior-level Full TimeAix-en-Provence, Provence-Alpes-Côte d'Azur, France R5d ago
-
AWS | AWS Glue | Apache Airflow | Apache Spark | AutomationBonuses and holiday allowance | Career development | Coaching and support for diversity | Employee representative council benefits | Health insuranceSenior-level Full TimeCourbevoie, IDF, France R5d ago
-
Airflow | Ansible | BigQuery | Bigtable | CI/CDEmployee benefits plan | Health insurance | Paid vacation bonus | Referral bonus | Remote workSenior-level Full TimeColomiers, Occitanie, France R5d ago