Data Engineer

EUR 45K-55K (estimate) Mid-level Full Time Found 12d ago

Tasks

• Adapt to new technologies and priorities
• Apply Agile concepts (Scrum, Sprint Planning)
• Automate deployments with CI/CD (GitLab CI)
• Collaborate within team
• Communicate effectively
• Contribute to data microservices architectures
• Design data pipelines
• Design scalable and resilient data architectures
• Develop and maintain data transformations with DBT
• Develop batch and streaming data architectures
• Develop services and APIs in Java (Spring Boot) and Scala
• Ensure data quality and consistency
• Evaluate and propose improvements
• Implement cost and performance strategies
• Implement real-time data flows with Kafka/PubSub
• Implement streaming with Kafka/Kafka Streams
• Maintain datasets in BigQuery
• Manage versioning with Git
• Model analytical data schemas
• Model and develop ELT/ETL pipelines
• Optimize queries and processing performance
• Orchestrate workflows with Airflow/Cloud Composer
• Write optimized SQL queries

Perks/Benefits

• N/A

Skills/Tech-stack

Airflow | BigQuery | CloudGCP | CostOptimization | DBT | DataModeling | Datapipeline | Datawarehouse | GitLab CI | Java | Kafka | Microservices | PerformanceOptimization | Resilience | SQL | Scala | Streaming | WorkflowOrchestration