Senior Data Engineer with Spark (m/f/d)

MentorMate creates durable technical solutions that deliver digital transformation at scale by blending strategic insights and thoughtful design with brilliant engineering. With mature and established practices in enterprise web and mobile development, quality engineering, technical architecture, human-centered design, cloud, DevOps, data, and analytics, the company provides challenging careers in a friendly and global people-oriented environment. We value every team member and share our success through flexible working models, competitive pay and benefits, and advancement opportunities. MentorMate Paraguay was established in 2023 after the acquisition of Software Natura and is the Latin American headquarters of our global digital transformation consultancy efforts.As part of global technology company Tietoevry Create, MentorMate enables people to work on impactful, global projects for recognizable brands and grow their careers in a team of over 10,000 experts. 
About the role
We are seeking a skilled Senior Data Engineer with a strong background in Spark and AWS to join our team. The ideal candidate will have experience in building and managing data pipelines with Spark and Databricks. This role involves working with both internal and third-party data sources. As a Senior Data Engineer, you will play a crucial role in ingesting, processing, and storing data from various sources to support our data-driven decision-making processes. 

Responsibilities

• Develop and maintain scalable data pipelines using Apache Spark
• Work within Databricks notebooks to run data processing tasks
• Ingest data from various sources including enterprise software such Salesforce and NetSuite, as well as internal software systems
• Utilize AWS tools, including S3, Redshift, and Aurora PostgreSQL, to manage and store data
• Implement a medallion architecture for data organization and processing
• Collaborate with cross-functional teams to understand data requirements and deliver high-quality data solutions
• Maintain and enhance existing data pipelines with a focus on performance and scalability
• Ensure data quality, integrity, and security across all managed systems

Requirements

• 7+ years of experience in Data Engineering
• Proven experience as a Senior Data Engineer with a strong focus on Apache Spark
• Experience with AWS services such as S3, Redshift, and Aurora PostgreSQL
• Strong knowledge of ingesting and processing data from diverse sources
• Good understanding and practical experience with medallion architecture
• Experience with modern SDLC, CI/CD and SCM
• Excellent problem-solving skills and ability to work independently or as part of a team
• Strong communication skills to collaborate effectively with technical and non-technical stakeholders

A significant advantage would be

• Experience working with Databricks for data processing

MentorMate is a place of constant energy and personality. On the clock, we’re technology experts and leaders that bring passion and knowledge to every project. Off the clock, we’re an eclectic bunch of foodies, music buffs, artists, sports fanatics, karaoke pros, and outdoor enthusiasts.Global leaders in digital consulting, design, and engineering services, MentorMate and Tietoevry Create are equal-opportunity employers. All applicants will be considered for employment without attention to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran, or disability status.