AWS Data Engineer

Our partner company is a dynamic and innovative IT consulting firm based in Hungary, specializing in custom software development, business intelligence, and cloud solutions. With over two decades of experience, they support their clients through complex digital transformation projects across various industries. Their team consists of dedicated experts who value collaboration, continuous learning, and delivering high-quality solutions. They are committed to fostering a supportive work environment where creativity and professional growth are encouraged.

Tasks:

• Developing data pipelines
• Participating in the planning process
• Consulting with technical experts
• Ensuring code quality, performing testing, bug fixing, and documentation
• Actively contributing as a member of an agile team

Requirements

Requirements / Key competencies:

• Professional working proficiency in English, both spoken and written
• Routine use of Atlassian tools (JIRA, Confluence)

Programming / Scripting:

• Python
• SQL
• PySpark

ETL Concepts:

• Data transformation using Python or Spark
• Data merge strategies (incremental loading, SCD, snapshot, etc.)
• File formats / schemas: Parquet, JSONSchema, Schema Registry, schema validation

IaC / CI/CD:

• Terraform (Terragrunt)
• Git, GitLab
• Containerization:
• Docker / Kubernetes
• API Management:
• Knowledge of REST APIs and experience with CRUD operations

AWS Services:

• Previous experience with data pipeline-related services
• Storage: S3
• Processing: Glue, Lambda, EMR, Batch
• Data storage: RDS (PostgreSQL), Redshift, DynamoDB, etc.
• Messaging: SNS / SQS
• API: API Gateway (resources, integrations)
• Data streams / pipelines

Orchestration:

• Experience with orchestration tools such as Airflow, Dagster, etc.
• Monitoring:
• Log analysis and debugging of pipeline steps

Testing:

• Unit testing and data quality checks during pipeline execution
• Streaming (optional, currently not in focus):
• Experience with streaming solutions (e.g., Kafka or Kinesis)
• Stream data processing: Flink, Kinesis Data Analytics

Benefits

• Weekly community events: team buildings, meetups, workshops, business breakfasts, and more
• Flexible work options
• Continuous development: exciting projects, mentoring, technical and soft skill training
• Supportive leadership and clear career growth opportunities
• Company holiday resort for workshops, team retreats, or family holidays