Senior Data Engineer
Amman, Jordan
Aspire, Jordan
About the Job
As a Senior Data Engineer at Aspire, will play a key role in designing, developing, and maintaining data workflows and pipelines, ensuring the efficient and reliable movement of data across various systems.
What you’ll do
What you’ll need
Why Aspire
In addition to a competitive long-term total compensation with salary and performance-based bonus, we have a reward philosophy that expands beyond this.
As a Senior Data Engineer at Aspire, will play a key role in designing, developing, and maintaining data workflows and pipelines, ensuring the efficient and reliable movement of data across various systems.
What you’ll do
- Design, implement, and manage complex DAGs and SubDAGs in Apache Airflow.
- Utilize Operators, macros, connections, variables, and xcom for efficient task execution.
- Conduct unit testing to ensure the robustness of Airflow workflows.
- Perform file operations and data movement between HDFS and S3 (and vice versa).
- Manage Hive external tables, update table DDL, and handle partition operations.
- Execute basic Hadoop commands and navigate/traverse HDFS.
- Work with Hive on Tez for optimized query execution.
- Utilize AWS command line tools for tasks such as listing and copying files from S3.
- Implement and manage secrets using tools like Ansible Vault and AWS Secrets Manager.
- Develop and optimize data processing tasks using PySpark.
- Understand and leverage Livy for efficient Spark job execution.
- Import and export data between Microsoft SQL Server and HDFS using Sqoop.
- Apply dimension modeling principles, define facts and dimensions, and manage surrogate keys.
- Implement effective branching strategies in Git.
- Participate in the process of raising and reviewing Merge Requests in GitLab.
- Utilize SSIS for migrating data workflows, ensuring a smooth transition.
- Model data in Snowflake, differentiating between external and internal tables.
- Understand the use of views and materialized views in Snowflake.
What you’ll need
- Bachelor’s degree in computer science, information technology, or a related field.
- Proficient in Python (both Python 2.7 and 3.6+).
- Experience in unit testing and test-driven development.
- Familiarity with Secrets Management tools and techniques.
- Knowledge of data modeling principles and practices.
- Strong understanding of Git version control and GitLab workflows.
- Experience with SSIS and Snowflake is a plus.
- Excellent problem-solving and communication skills.
Why Aspire
In addition to a competitive long-term total compensation with salary and performance-based bonus, we have a reward philosophy that expands beyond this.
- Be part of a (Remote is
here-to stay) organization.
- Work
and learn from great minds.
- Explore
new opportunities to learn and grow everyday by attending technical and
nontechnical training.
- Get market exposure by
working with international tech leaders.
- Nursery reimbursement
benefit.
- Attend virtual and onsite
international tech conference.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Job stats:
0
0
0
Category:
Engineering Jobs
Tags: Airflow Ansible AWS Computer Science DDL Git GitLab Hadoop HDFS Pipelines PySpark Python Snowflake Spark SQL SSIS TDD Testing
Perks/benefits: Competitive pay
Region:
Middle East
Country:
Jordan
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.
Data Engineer II jobsBI Developer jobsData Scientist II jobsStaff Data Scientist jobsPrincipal Data Engineer jobsData Manager jobsJunior Data Analyst jobsData Science Manager jobsResearch Scientist jobsLead Data Analyst jobsBusiness Data Analyst jobsSenior AI Engineer jobsData Engineer III jobsSr. Data Scientist jobsData Science Intern jobsData Specialist jobsJunior Data Engineer jobsSoftware Engineer, Machine Learning jobsSenior Data Scientist, Performance Marketing jobsSoftware Engineer II jobsData Analyst Intern jobsBI Analyst jobsData Analyst II jobsSr Data Engineer jobsSenior Artificial Intelligence/Machine Learning Engineer - Remote, Latin America jobs
Snowflake jobsEconomics jobsLinux jobsHadoop jobsOpen Source jobsJavaScript jobsPhysics jobsMLOps jobsComputer Vision jobsAirflow jobsKafka jobsRDBMS jobsBanking jobsNoSQL jobsGoogle Cloud jobsData Warehousing jobsScala jobsR&D jobsKPIs jobsData warehouse jobsGitHub jobsOracle jobsClassification jobsPostgreSQL jobsSAS jobs
CX jobsStreaming jobsLooker jobsScikit-learn jobsScrum jobsTerraform jobsPandas jobsPySpark jobsDistributed Systems jobsBigQuery jobsData Mining jobsRobotics jobsJira jobsIndustrial jobsJenkins jobsRedshift jobsdbt jobsReact jobsUnstructured data jobsMicroservices jobsData strategy jobsE-commerce jobsMySQL jobsMatlab jobsNumPy jobs