PySpark Developer
Tasks
- Build and manage data workflows
- Collaborate in agile teams
- Develop batch data pipelines
- Develop streaming data pipelines
- Implement testing with PyTest
- Integrate data sources and sinks
- Optimize spark performance
Perks/Benefits
Skills/Tech-stack
AWS | Amazon EMR | Apache Airflow | Apache Kafka | Apache Spark | Avro | Azure | CI/CD | Cloud platform | Databricks | Delta Lake | Git | Google Cloud | Google Cloud Platform | Parquet | Prefect | PySpark | Pytest | Python | SQL
Education
N/A
Roles
Data Engineer | Developer | Engineer | PySpark Developer | Senior PySpark Developer
Related jobs
-
Automation | Azure | Azure Data | Azure Data Explorer | CSVCollaborative global working environment | Cross-functional collaboration | Exposure to large scale security telemetry | Hands on development of automation and data quality solutions | Professional growthMid-level Full TimePortugal R8h ago
-
Senior-level Full TimeLisbon, PT14h ago
-
Senior Data Engineer Consultant EUR 63K-79KAI | Azure Data | Azure Data Factory | Data Factory | DatabricksAnnual leave | Employee assistance programme | Flexible work environment | Health insurance | Hybrid work modelSenior-level Full TimePorto, Porto District, Portugal R1d ago
-
Data Engineer EUR 39K-46KApache Spark | Data Modeling | Data Processing | Data Transformation | Data WarehousingSenior-level Full TimeLisbon, Portugal1d ago
-
Data Engineer EUR 35K-51KAWS | Azure | Business glossary | Cloud Architecture | Cloud platformDiscounts and flex options | Health insurance | Hybrid work | Meal allowance | Paid vacationSenior-level Full TimeLisbon, Portugal R1d ago
-
Ansible | CI/CD | Docker | ELK Stack | GitLabHealth insurance | Meal card | Remote workSenior-level Full TimePortugal - Remote R1d ago
-
AI Engineer EUR 36K-65KAPI Development | Agent systems | CI/CD | Docker | FastAPIHealth insurance | Hybrid work | Life insurance | Referral program | Training programsMid-level Full TimeLisboa, PT R1d ago
-
Advanced Analytics Engineer USD 95KAPI Integration | AWS | Azure | CI/CD | Cloud platformCareer opportunities | Inclusive culture | Professional development | Supportive leadership | Wellbeing benefitsMid-level Full TimeLisbon - Antonio, Portugal R1d ago
-
AI/ML Engineer EUR 82K-124KAWS | Agent Orchestration | Airflow | Artificial Intelligence | AzureCompany equity | Companywide hack weeks | ESPP | Flexible work arrangements | Hibernation dutySenior-level Full TimeLisbon R1d ago
-
Senior AI/ML Engineer EUR 86K-129KAWS | Agent Orchestration | Artificial Intelligence | Azure | Cloud platformCompany equity | Employee stock purchase program | Flexible work arrangements | Hack weeks | Mental wellness programsSenior-level Full TimeLisbon R1d ago
-
Software Engineer – Embedded AI (C++) (f/m/div.) EUR 55K-57KAPI Design | C++ | CI/CD | Clean Code | Computer VisionCanteen | Career progression | Discounts | Flexible work conditions | Free parking lotSenior-level Full TimeBraga, Sequeira, Portugal R1d ago
-
AI Engineer (.NET / Azure AI) EUR 25K-65K.NET | Angular | Ansible | Azure Cognitive | Azure Cognitive ServicesComfort budget | Dental insurance | Flexible working hours | Health insurance | MentoringMid-level Full TimeCoimbra, Portugal R1d ago
-
Automated Workflows | CI/CD | Cloud Native | Containerization | Data LineageCollaborative environment | Inclusive workplace | Opportunity to work on advanced AI challenges | Professional growthSenior-level Full TimePortugal2d ago
-
Access Control | Azure | Azure Data | Azure Data Factory | Azure DevOpsCollaborative cross-functional environment | Hands on work with Databricks and Azure | Professional growthMid-level Full TimePortugal2d ago
-
Senior Data Engineer Consultant EUR 46K-77KAzure Data | Azure Data Factory | Data Factory | Databricks | ETLAnnual leave | Employee assistance programme | Flexible work environment | Health insurance | Hybrid work modelSenior-level Full TimePorto, Porto District, Portugal R2d ago
-
Senior Analytics Engineer EUR 64K-85KAWS | AWS Glue | AWS Lambda | Amazon Kinesis | Amazon RedshiftCompany well being days | Donation matching | Equal parental leave | Extra vacation days | Flexible benefitsSenior-level Full TimeLisbon, Portugal R2d ago
-
AI Data Engineer EUR 39K-65KBigQuery | Cloud platform | Data Architecture | Data Compliance | Data GovernanceClient-facing experience | Communication with stakeholders | English C1 proficiencySenior-level Full TimeLisbon, Exeo Lumnia, Portugal2d ago
-
Senior-level Full TimePorto - Portugal R2d ago
-
Senior Data Engineer EUR 68K-79KAmazon Redshift | CDC | Data Contracts | Data Governance | Data LineageSenior-level Full TimePorto - Portugal R2d ago
-
Staff Engineer, Compute EUR 100KAWS | Automation | Capacity Planning | Cloud platform | Data AnalysisCareer development | Cross department buddy program | Employee onboarding program | Employee stock purchase plan | Hybrid work modelSenior-level Full TimeLisbon, Portugal R2d ago
-
Lead Agentic AI Engineer ID80141 EUR 48K-62KAWS | AWS S3 | Azure | CI/CD | Cloud SecurityCollaborative culture | Flexible hours | Internal TechTalks | Learning budget | MentorshipSenior-level Full TimeAveiro, Portugal2d ago
-
Lead Agentic AI Engineer ID80141 EUR 48K-62KAWS | Agentic AI | Azure | CI/CD | Cloud SecurityAnnual learning budget | Challenging projects | Flexible hours | Internal tech talks | MentorshipSenior-level Full TimeLisboa, Portugal2d ago
-
Lead Agentic AI Engineer ID80141 EUR 48K-62KAWS | Agentic AI | Azure | CI/CD | Cloud infrastructureFlexible hours | Internal tech talks | Learning budget | Mentorship | Performance reviewsSenior-level Full TimeBraga, Portugal2d ago
-
Lead Agentic AI Engineer ID80141 EUR 48K-62KAWS | Anthropic | Azure | CI/CD | Cloud SecurityAnnual learning budget | Collaborative culture | Flexible hours | Internal tech talks | MentorshipSenior-level Full TimePorto, Portugal2d ago
-
Lead Agentic AI Engineer ID80141 EUR 48K-62KCI/CD | Cloud Architecture | Cloud Security | Drift Detection | GenAIAnnual learning budget | Flexible hours | Internal TechTalks | Mentorship | Performance reviewsSenior-level Full TimeCoimbra, Portugal2d ago