Senior Data Engineer with Spark (m/f/d)
Paraguay Remote
MentorMate
MentorMate creates durable technical solutions that deliver digital transformation at scale by blending strategic insights and thoughtful design with brilliant engineering. With mature and established practices in enterprise web and mobile development, quality engineering, technical architecture, human-centered design, cloud, DevOps, data, and analytics, the company provides challenging careers in a friendly and global people-oriented environment. We value every team member and share our success through flexible working models, competitive pay and benefits, and advancement opportunities. MentorMate Paraguay was established in 2023 after the acquisition of Software Natura and is the Latin American headquarters of our global digital transformation consultancy efforts.As part of global technology company Tietoevry Create, MentorMate enables people to work on impactful, global projects for recognizable brands and grow their careers in a team of over 10,000 experts.
About the role
We are seeking a skilled Senior Data Engineer with a strong background in Spark and AWS to join our team. The ideal candidate will have experience in building and managing data pipelines with Spark and Databricks. This role involves working with both internal and third-party data sources. As a Senior Data Engineer, you will play a crucial role in ingesting, processing, and storing data from various sources to support our data-driven decision-making processes.
About the role
We are seeking a skilled Senior Data Engineer with a strong background in Spark and AWS to join our team. The ideal candidate will have experience in building and managing data pipelines with Spark and Databricks. This role involves working with both internal and third-party data sources. As a Senior Data Engineer, you will play a crucial role in ingesting, processing, and storing data from various sources to support our data-driven decision-making processes.
Responsibilities
- Develop and maintain scalable data pipelines using Apache Spark
- Work within Databricks notebooks to run data processing tasks
- Ingest data from various sources including enterprise software such Salesforce and NetSuite, as well as internal software systems
- Utilize AWS tools, including S3, Redshift, and Aurora PostgreSQL, to manage and store data
- Implement a medallion architecture for data organization and processing
- Collaborate with cross-functional teams to understand data requirements and deliver high-quality data solutions
- Maintain and enhance existing data pipelines with a focus on performance and scalability
- Ensure data quality, integrity, and security across all managed systems
Requirements
- 7+ years of experience in Data Engineering
- Proven experience as a Senior Data Engineer with a strong focus on Apache Spark
- Experience with AWS services such as S3, Redshift, and Aurora PostgreSQL
- Strong knowledge of ingesting and processing data from diverse sources
- Good understanding and practical experience with medallion architecture
- Experience with modern SDLC, CI/CD and SCM
- Excellent problem-solving skills and ability to work independently or as part of a team
- Strong communication skills to collaborate effectively with technical and non-technical stakeholders
A significant advantage would be
- Experience working with Databricks for data processing
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Job stats:
1
0
0
Category:
Engineering Jobs
Tags: Architecture AWS CI/CD Consulting Databricks Data pipelines Data quality DevOps Engineering Pipelines PostgreSQL Redshift Salesforce SDLC Security Spark
Perks/benefits: Competitive pay Flex hours
Regions:
Remote/Anywhere
South America
Country:
Paraguay
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.
Staff Machine Learning Engineer jobsPrincipal Data Engineer jobsBI Developer jobsData Scientist II jobsStaff Data Scientist jobsData Manager jobsJunior Data Analyst jobsResearch Scientist jobsData Science Manager jobsBusiness Data Analyst jobsLead Data Analyst jobsData Engineer III jobsSenior AI Engineer jobsSr. Data Scientist jobsData Science Intern jobsData Analyst Intern jobsData Specialist jobsSoftware Engineer, Machine Learning jobsSoftware Engineer II jobsData Analyst II jobsPrincipal Software Engineer jobsJunior Data Engineer jobsBI Analyst jobsSenior Data Scientist, Performance Marketing jobsSr Data Engineer jobs
Snowflake jobsLinux jobsEconomics jobsOpen Source jobsJavaScript jobsHadoop jobsComputer Vision jobsBanking jobsPhysics jobsMLOps jobsRDBMS jobsGoogle Cloud jobsAirflow jobsNoSQL jobsKafka jobsData Warehousing jobsR&D jobsScala jobsKPIs jobsGitHub jobsData warehouse jobsClassification jobsOracle jobsScikit-learn jobsPostgreSQL jobs
SAS jobsCX jobsStreaming jobsScrum jobsPandas jobsPySpark jobsData Mining jobsTerraform jobsDistributed Systems jobsLooker jobsBigQuery jobsRobotics jobsIndustrial jobsJira jobsJenkins jobsUnstructured data jobsdbt jobsRedshift jobsE-commerce jobsReact jobsMicroservices jobsPharma jobsData strategy jobsNumPy jobsMySQL jobs