Senior Data Engineer

Important Information 

Location: Colombia, Costa Rica

Work Mode: Remote 

Job Summary 

As a Senior Data Engineer (13895), you will be responsible for designing, developing, and maintaining high-quality software solutions. You will collaborate with cross-functional teams to understand business requirements and translate them into scalable and efficient software applications. Your role will involve leading technical projects, mentoring junior engineers, and continuously improving software development practices to ensure the delivery of robust and reliable software systems.  

Responsibilities and Duties 

• Deploy and configure Apache Spark clusters on AWS (particularly EKS), optimizing for performance, resource allocation, and scalability.

• Develop, automate, and support robust and reliable Spark data pipelines, focusing on high performance and low latency.

• Design and implement highly optimized queries to improve data processing efficiency, streamline analysis, minimize latency, and enhance overall system performance.

• Collaborate to make the Data Platform an effective, scalable, and resilient system.

• Gather requirements from stakeholders, prioritize work, and document technical solutions clearly and effectively.

• Engage with data engineers, data scientists, product managers, and internal stakeholders to align project goals and implementations.

• Contribute to a team culture that values quality, robustness, and scalability while fostering initiatives and innovation.

.

Qualifications and Skills 

• Bachelor’s degree in computer science, software engineering, or a related field.  
• Extensive experience in software development with a focus on designing and building scalable applications.  
• Professional/ Advanced English skills. 

• 5+ years of data processing experience in large cloud-based infrastructure (AWS is must).

• Hands-on software development experience in Python, with strong proficiency in Py Spark for data engineering tasks and data pipeline development.

• Expert understanding of SQL, dimensional modeling, and analytical data warehouses, such as Snowflake.

• Understanding of Data Engineering best practices for medium to large scale production workloads.

• Expertise with data pipeline orchestration tools, such as Airflow.

• Familiar with processing semi-structured file formats such as json or parquet.

• Team player with good communication skills.

• Problem solver with excellent written and interpersonal skills.

• Strong problem-solving skills.

• Ability to learn new technical skills.

Additional Requirements 

• Experience with Jinja, Shell scripting, DBT.

• Developing in Cloud platform using serverless technologies such as glue, lambda, EMR is a plus.

• Experience with remote development using AWS SDK is a plus.

• Experience with ELT pipelines - DBT.

• REST API design and implementation.

• Familiarity with containers and infrastructure-as-code principles.

• Experience with automation frameworks - Git, Jenkins, and Terraform.

About Encora 
 
Encora is a global company that offers Software and Digital Engineering solutions. Our practices include Cloud Services, Product Engineering & Application Modernization, Data & Analytics, Digital Experience & Design Services, DevSecOps, Cybersecurity, Quality Engineering, AI & LLM Engineering, among others.  

At Encora, we hire professionals based solely on their skills and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.