Senior Data Engineer
Tasks
- Architect lakehouse data solutions
- Build CI/CD pipelines
- Build and maintain data pipelines
- Collaborate with data scientists and analysts
- Create technical documentation
- Deploy containerized solutions
- Design scalable ETL and ELT workflows
- Develop data models and schemas
- Implement data governance and security controls
- Implement real-time streaming with Kafka
- Mentor junior data engineers
- Monitor infrastructure performance and optimize cost
- Optimize distributed data processing with Spark
- Process large volumes of structured and unstructured data
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Amazon S3 | Apache Airflow | Apache Spark | Azure | Azure Blob | Azure Blob Storage | Blob Storage | CI/CD | Data Governance | Data Modeling | Databricks | Dimensional Modeling | Docker | Flink | Git | Google BigQuery | Google Cloud | Informatica | Java | Kafka | Kubernetes | Python | SQL | Scala | Spark Streaming | Talend
Education
N/A
Roles
Related jobs
-
AI readiness | AI workflows | AWS | Access Control | AutomationGrowth opportunities | Remote work | US time zone collaborationSenior-level Full TimeArgentina - Buenos Aires14h ago
-
AWS | Access Management | Azure | Cloud platform | Data ClassificationRemote workSenior-level Full TimeHonduras - Tegucigalpa14h ago
-
Junior - Intermediate Data Engineer ZAR 240K-360KData Transformation | Database Views | Grafana | Microsoft Excel | Microsoft SQLRolling contractEntry-level ContractJohannesburg, South Africa23h ago
-
Analytics Engineer (Data Platform) ZAR 400K-420KAgile | Algorithms | BigQuery | CI/CD | Cube designBirthday leave | Confidential counseling | Financial guidance | Flexible equipment setup | Free parkingMid-level Full TimeCape Town1d ago
-
Analytics Engineer (Data Platform) ZAR 400K-420KAgile | Algorithms | BigQuery | CI/CD | Data ModelingBirthday leave | Confidential counselling | Financial guidance | Free parking | Hybrid work modelMid-level Full TimeCape Town1d ago
-
AI Enablement Analyst ZAR 900K-1000KAI Foundry | API Integration | Acceptance Testing | Agentic AI | AgileMid-level Full TimeJohannesburg, ZA1d ago
-
Senior-level Full TimePretoria, South Africa2d ago
-
Principal Data Engineer ZAR 400K-464KData Engineering | Data Modeling | Database Design | Databricks | Declarative PipelinesMid-level Full TimeCape Town, Western Cape, South Africa2d ago
-
Senior-level Full TimeJohannesburg, South Africa3d ago
-
Senior-level Full TimeJohannesburg, South Africa3d ago
-
Data Engineer (D) ZAR 464K-720KAnomaly Detection | Azure | Batch Processing | By Design | Data DriftMid-level Full TimeJohannesburg, South Africa3d ago
-
Artificial Intelligence | Azure | Azure Data | Azure Data Factory | Batch ProcessingMid-level Full TimeCape Town, South Africa3d ago
-
CRM Integration | Claude API | Docker | Event Driven | Event-driven architectureSenior-level Full TimeCape Town, South Africa4d ago
-
Access Control | Compliance | Data Governance | Data Lineage | Data ModelingAgile team collaborationMid-level Full TimeCape Town, South Africa4d ago
-
Senior Data Engineer (Azure) ZAR 875K-1170KAzure Data | Azure Data Factory | Azure Synapse | Azure Synapse Analytics | Data FactoryIndependent work environment | Remote work | Stakeholder engagementSenior-level ContractJohannesburg, South Africa4d ago
-
Data Modeling | Distributed Systems | Embeddings | Kubernetes | LLM InferenceAccess to technical leadership | Collaborative culture | Flexible schedule | High autonomy | High ownershipEntry-level Full TimeSouth Africa R5d ago
-
Bash | Cloud platform | Data Pipelines | Data Processing | DockerAsynchronous culture | Friendly atmosphere | Hands-off management | Remote distributed settingMid-level Full TimeCape Town, South Africa5d ago
-
BI Data Analyst II ZAR 660K-720KAPI Development | Acceptance criteria | Cloud Computing | Cost Optimization | Data Acceptance CriteriaSenior-level Full TimeJohannesburg, ZA5d ago
-
Senior-level Full TimeStellenbosch, ZA5d ago
-
Process Automation Engineer (AI/ML) ZAR 450K-580KAPI Integration | Anomaly Detection | Classification | Lean Six Sigma | Machine LearningMid-level Full TimeJohannesburg, South Africa8d ago
-
Senior Data Scientist (OVA5666) ZAR 780K-1200KApache Spark | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure KubernetesSenior-level Full TimeJohannesburg, South Africa9d ago
-
Data Engineer ZAR 660K-780KAPI Development | Ab Initio | Agile delivery | Azure Data | Azure Data FactorySenior-level Full TimeJohannesburg, ZA9d ago
-
Bash | Data Ingestion | Data Processing | Docker | GCPMid-level Full TimeJohannesburg, South Africa9d ago
-
Machine Learning Engineer ZAR 780K-860KAI Agents | AI Services | Azure AI | Azure AI Services | Azure OpenAISenior-level Full TimeSandton - 1 Discovery Place, GP, …10d ago
-
Embedded Software Development Engineer ZAR 420K-504KARM | C# | Electronic schematics | Embedded System | Embedded System ArchitectureDisability programs | Hybrid work | Life insurance | Private medical care | Tailored financial programsMid-level Full TimeZA-Century City-Cape Town-The Matrix 2G and …10d ago