Data Engineer AWS SR

Important Information 

Location: Brazil
Job Mode: Full-time 
Work Mode: Work from home

Job Summary

We're looking for a talented and motivated Sr Data Engineer to join an Analytics team to develop robust and scalable data pipelines using and improving existing platforms. The successful candidate will drive performance enhancements, build pipelines, and collaborate with other Data Engineering teams. You will work with cutting-edge technologies in big data space.

Responsibilities and Duties

• Deploy and configure Apache Spark clusters on AWS (particularly EKS), optimizing for performance, resource allocation, and scalability;
  
• Develop, automate, and support robust and reliable Spark data pipelines, focusing on high performance and low latency;
  
• Design and implement highly optimized queries to improve data processing efficiency, streamline analysis, minimize latency, and enhance overall system performance;
  
• Collaborate to make the Data Platform an effective, scalable, and resilient system;
  
• Gather requirements from stakeholders, prioritize work, and document technical solutions clearly and effectively;
  
• Engage with data engineers, data scientists, product managers, and internal stakeholders to align project goals and implementations;
  
• Contribute to a team culture that values quality, robustness, and scalability while fostering initiatives and innovation.

Essential Skills

• Data processing experience in large cloud-based infrastructure (AWS is a must);
  
• Hands-on software development experience in Python, with strong proficiency in Py Spark for data engineering tasks and data pipeline development Expert understanding of SQL, dimensional modeling, and analytical data warehouses, such as Snowflake Understanding of Data Engineering best practices for medium to large-scale production workloads;
  
• Expertise with data pipeline orchestration tools, such as Airflow;
  
• Familiar with processing semi-structured file formats such as JSON or parquet;
  
• Ability to learn new technical skills.
  

Highly Desirable Skills

• Experience with Jinja, Shell scripting, DBT;
  
• Developing a Cloud platform using serverless technologies such as glue, lambda, EMR is a plus;
  
• Experience with remote development using AWS SDK is a plus;
  
• Experience with ELT pipelines - DBT;
  
• REST API design and implementation;
  
• Familiarity with containers and infrastructure-as-code principles;
  
• Experience with automation frameworks - Git, Jenkins, and Terraform.

About Encora

Encora is the preferred digital engineering and modernization partner of some of the world’s leading enterprises and digital native companies. With over 9,000 experts in 47+ offices and innovation labs worldwide, Encora’s technology practices include Product Engineering & Development, Cloud Services, Quality Engineering, DevSecOps, Data & Analytics, Digital Experience, Cybersecurity, and AI & LLM Engineering.

At Encora, we hire professionals based solely on their skills and qualifications, and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.