Senior Data Engineer with Spark (m/f/d)
Paraguay Remote
Applications have closed
MentorMate creates durable technical solutions that deliver digital transformation at scale by blending strategic insights and thoughtful design with brilliant engineering. With mature and established practices in enterprise web and mobile development, quality engineering, technical architecture, human-centered design, cloud, DevOps, data, and analytics, the company provides challenging careers in a friendly and global people-oriented environment. We value every team member and share our success through flexible working models, competitive pay and benefits, and advancement opportunities. MentorMate Paraguay was established in 2023 after the acquisition of Software Natura and is the Latin American headquarters of our global digital transformation consultancy efforts.As part of global technology company Tietoevry Create, MentorMate enables people to work on impactful, global projects for recognizable brands and grow their careers in a team of over 10,000 experts.
About the role
We are seeking a skilled Senior Data Engineer with a strong background in Spark and AWS to join our team. The ideal candidate will have experience in building and managing data pipelines with Spark and Databricks. This role involves working with both internal and third-party data sources. As a Senior Data Engineer, you will play a crucial role in ingesting, processing, and storing data from various sources to support our data-driven decision-making processes.
About the role
We are seeking a skilled Senior Data Engineer with a strong background in Spark and AWS to join our team. The ideal candidate will have experience in building and managing data pipelines with Spark and Databricks. This role involves working with both internal and third-party data sources. As a Senior Data Engineer, you will play a crucial role in ingesting, processing, and storing data from various sources to support our data-driven decision-making processes.
Responsibilities
- Develop and maintain scalable data pipelines using Apache Spark
- Work within Databricks notebooks to run data processing tasks
- Ingest data from various sources including enterprise software such Salesforce and NetSuite, as well as internal software systems
- Utilize AWS tools, including S3, Redshift, and Aurora PostgreSQL, to manage and store data
- Implement a medallion architecture for data organization and processing
- Collaborate with cross-functional teams to understand data requirements and deliver high-quality data solutions
- Maintain and enhance existing data pipelines with a focus on performance and scalability
- Ensure data quality, integrity, and security across all managed systems
Requirements
- 7+ years of experience in Data Engineering
- Proven experience as a Senior Data Engineer with a strong focus on Apache Spark
- Experience with AWS services such as S3, Redshift, and Aurora PostgreSQL
- Strong knowledge of ingesting and processing data from diverse sources
- Good understanding and practical experience with medallion architecture
- Experience with modern SDLC, CI/CD and SCM
- Excellent problem-solving skills and ability to work independently or as part of a team
- Strong communication skills to collaborate effectively with technical and non-technical stakeholders
A significant advantage would be
- Experience working with Databricks for data processing
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Job stats:
1
0
0
Category:
Engineering Jobs
Tags: Architecture AWS CI/CD Consulting Databricks Data pipelines Data quality DevOps Engineering Pipelines PostgreSQL Redshift Salesforce SDLC Security Spark
Perks/benefits: Competitive pay Flex hours
Regions:
Remote/Anywhere
South America
Country:
Paraguay
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.
Sr. Data Engineer jobsBusiness Intelligence Developer jobsPrincipal Data Engineer jobsBI Developer jobsStaff Data Scientist jobsStaff Machine Learning Engineer jobsPrincipal Software Engineer jobsJunior Data Analyst jobsData Science Intern jobsDevOps Engineer jobsData Science Manager jobsData Manager jobsSoftware Engineer II jobsAccount Executive jobsData Analyst Intern jobsLead Data Analyst jobsStaff Software Engineer jobsBusiness Data Analyst jobsSr. Data Scientist jobsAI/ML Engineer jobsData Specialist jobsSenior Backend Engineer jobsData Governance Analyst jobsData Engineer III jobsBusiness Intelligence Analyst jobs
Consulting jobsMLOps jobsAirflow jobsOpen Source jobsEconomics jobsLinux jobsKPIs jobsKafka jobsTerraform jobsGitHub jobsJavaScript jobsPostgreSQL jobsRDBMS jobsData Warehousing jobsNoSQL jobsClassification jobsStreaming jobsBanking jobsScikit-learn jobsPrompt engineering jobsComputer Vision jobsRAG jobsPhysics jobsGoogle Cloud jobsPandas jobs
Oracle jobsdbt jobsScala jobsHadoop jobsBigQuery jobsGPT jobsData warehouse jobsReact jobsR&D jobsLangChain jobsScrum jobsDistributed Systems jobsPySpark jobsLooker jobsMicroservices jobsIndustrial jobsELT jobsCX jobsJira jobsSAS jobsOpenAI jobsRedshift jobsModel training jobsTypeScript jobsJenkins jobs