Senior Data Engineer
Tasks
- Design data pipelines
- Develop data processing with PySpark
- Execute Hadoop commands
- Implement Apache Airflow DAGs
- Implement branching strategies in Git
- Implement secrets management
- Import/export data using Sqoop
- Manage HIVE external tables and partitions
- Manage data workflows and task execution
- Manage surrogate keys
- Migrate workflows with SSIS
- Model data in Snowflake
- Model dimensions and facts
- Move data between HDFS and S3
- Optimize queries with Hive on Tez
- Perform unit testing for workflows
- Raise and review merge requests in GitLab
- Run Spark jobs with Livy
- Use AWS command line for S3 operations
- Use views and materialized views in Snowflake
Perks/Benefits
- Global exposure
- Non technical training
- Nursery reimbursement
- Remote work
- Technical training
- Virtual and in person conferences
- Wellness program
Skills/Tech-stack
AWS CLI | AWS Secrets | AWS Secrets Manager | Amazon Web Services | Ansible Vault | Apache Airflow | Dimension modeling | Git | GitLab | HDFS | Hadoop | Hadoop commands | Hive | ISO 27001 | Livy | Microsoft SQL | Microsoft SQL Server | PySpark | Python | S3 | SOC 2 | SQL Server | SSIS | Secrets Manager | Snowflake | Sqoop | Tez | Web Services | XCOM
Education
Bachelor of Engineering | Bachelor of Science | Bachelor of Science in Computer Science | Bachelor of Science in Information Technology
Roles
Related jobs
-
Mid-level Full TimeBucurești, Municipiul București, RO R2h ago
-
AWS | Azure | CI/CD | Docker | KubernetesHealthcare | Long-term contract | Multinational environment | Remote workMid-level Full TimeRemote R3h ago
-
Apache Airflow | CI/CD | Cloud | Data Governance | DockerComprehensive healthcare | Long-term B2B contract | Remote work modelMid-level Full TimeRemote R3h ago
-
API Security | FastAPI | Flask | Langchain | LanggraphHealthcare | Long-term contract | Multinational environment | Remote workMid-level Full TimeRemote R3h ago
-
Applied AI Engineer EUR 42K-70KAPI Development | API Gateway | Agent Orchestration | Apache Camel | Audit LoggingHybrid work opportunities | Inclusive workplaceMid-level Full Time's-Hertogenbosch, North Brabant, Netherlands R6h ago
-
Applied AI Engineer EUR 42K-70KAPI Gateway | Apache Camel | Apache Kafka | CI/CD | DockerHybrid workMid-level Full Time's-Hertogenbosch, North Brabant, Netherlands R6h ago
-
Applied AI Engineer EUR 42K-70KAPI | API Gateway | Agent Orchestration | Apache Camel | AtomiiHybrid workMid-level Full Time's-Hertogenbosch, North Brabant, Netherlands R6h ago
-
Data Engineer PLN 276K-276KAST Parsing | CI/CD | Data Cleansing | Data Transformation | Data pipelineCompany-paid certifications | Internal events meetups conferences workshops | Language courses | MyBenefit Cafeteria Platform | Premium medical insuranceMid-level Full TimePoland R7h ago
-
API Integration | APIs | Agent systems | Apache Spark | AutomationInternal talks | Learning platform | Remote days | Sports classes | Wellness benefitsEntry-level Full TimeBrooklyn, United States R7h ago
-
GenAI Engineer EUR 61K-90KAWS | Azure | CI/CD | Cloud Platforms | Cloud platformCoaching | Company car option | Eco vouchers | Employee discounts | Hospitalization insuranceSenior-level Full TimeBelgium R9h ago
-
Senior Data Engineer EUR 64K-76KAgile | Airbyte | Airflow | Cloud infrastructure | CubeEmployee well-being programs | Flexible time off | Hybrid work | Two days per week in officeSenior-level Full TimeBarcelona, Catalonia, Spain R9h ago
-
Senior MLOps Engineer - Edge GBP 67K-111KCI/CD | CUDA | Calibration | Canary Releases | ContainerizationEmployee assistance program | Flexible vacation | Medical benefits | Meeting-free days | Remote work optionsSenior-level Full TimeBarcelona, Spain; London, United Kingdom; Spain … R10h ago
-
Mid-level Full TimeNoida R10h ago
-
Software Engineer 2 INR 1500K-2500KAPI Design | AWS | Asynchronous Communication | Data Structures | Distributed SystemsMid-level Full TimeHybrid - Bangalore, India R11h ago
-
Senior-level Full TimeBangalore, India R12h ago
-
Software Engineer - AI Developer Tooling Workflows AUD 100K-100KAI Agents | APIs | CI/CD | Continuous Deployment | Continuous integrationEquity packages | Flexible leave options | Hybrid work flexibility | Inclusive parental leave | Wellbeing allowanceMid-level Full TimeSydney, Australia R12h ago
-
Senior Data Engineer INR 3000K-4000KAWS | AWS Managed Airflow | Airflow | Apache Spark | AzureMedical insurance | Remote-first workplace | Term Life and Personal Accident | Wellness allowanceSenior-level Full TimeIndia - Bangalore R12h ago
-
BI reporting | BigQuery | Business Intelligence | DBT | Dashboard DevelopmentAutonomy | Cross-functional collaboration | Distributed international environment | Fully remote | Mission-driven environmentSenior-level Full TimeBrazil R13h ago
-
Mid-level Full TimeBengaluru, KA, IN R17h ago
-
Platform Engineer INR 2500K-5000KAPI Design | Agentic Frameworks | Backend Development | Distributed Systems | Infrastructure as CodeFull autonomy | Remote workSenior-level Full TimeIndia - Remote R17h ago
-
Senior AI Computer Vision Engineer EUR 54K-75KArtificial Intelligence | C plus plus | Computer Vision | Deep learning | GeometryAI workshops | Advanced training | Deep tech development | Flexible hours | Hybrid work optionsSenior-level Full TimeLeuven, Flanders, Belgium R17h ago
-
Data Science Application Developer BGN 70K-80KAWS | Big Data | Data Mining | Data analytics | JavaScriptCareer path opportunities | Employee stock purchase plan | Learning and development programsMid-level Full TimeBGR - Sofia, Bulgaria R17h ago
-
AI Engineer INR 3264K-5876KARM Templates | AWS SNS | AWS SQS | Access Control | Amazon EKSCareer growth | Hybrid work location | Mentorship and coachingSenior-level Full TimeHyderabad (SEZ), India R17h ago
-
Embedded Systems Engineer - FPGA Platform GBP 55K-65KADC | Automated testing | Board Bring-up | Bootloader | Bring-upCycle to work scheme | Life assurance | Pension contribution | Private healthcare insurance | Tax efficient technology schemesMid-level Full TimeKidlington, Oxfordshire, United Kingdom R17h ago
-
Senior Fullstack Developer (Python+AI+React) (IR-545) UAH 1377K-1549KAWS | Application Security | Authentication | Authorization | AzureAbsence days | Clear career path | Expert knowledge sharing | Flexible hours | Flexible work setupSenior-level Full TimeUkraine - Remote R17h ago