Data Engineer
Tasks
- Automate monitor and improve data pipeline performance
- Build and manage data infrastructure
- Build data models and database schemas
- Create and maintain data pipelines for ingestion provisioning and streaming
- Develop APIs for data consumers
- Extract data from golden sources and trusted sources
- Implement data quality checks and governance
- Load data warehouse and data marts
- Maintain data in operational and analytics databases
- Transform data to common data model
Perks/Benefits
- N/A
Skills/Tech-stack
API Development | Ab Initio | Agile delivery | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Cloud Computing | Data Factory | Data Lake | Data Lake Storage Gen2 | Data Warehouse | Denodo | ETL | HBase | Hadoop | Hive | IBM DB2 | Java | Kafka | Microsoft SQL | Microsoft SQL Server | MongoDB | NoSQL | PostgreSQL | Python | SAS | SQL | SQL Server | Spark
Roles
Related jobs
-
Process Automation Engineer (AI/ML) ZAR 450K-580KAPI Integration | Anomaly Detection | Classification | Lean Six Sigma | Machine LearningMid-level Full TimeJohannesburg, South Africa1d ago
-
Senior-level Full TimeEgypt R1d ago
-
Senior Data Scientist (OVA5666) ZAR 780K-1200KApache Spark | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure KubernetesSenior-level Full TimeJohannesburg, South Africa1d ago
-
Senior Machine Learning Engineer ZAR 660K-840KAWS EC2 | AWS IAM | AWS KMS | AWS S3 | Amazon CloudWatchCell phone allowance | Data benefits | Fibre connection discount | Health insurance | Retirement benefitsSenior-level Full TimeMidrand, ZA2d ago
-
Bash | Data Ingestion | Data Processing | Docker | GCPMid-level Full TimeJohannesburg, South Africa2d ago
-
Machine Learning Engineer ZAR 780K-860KAI Agents | AI Services | Azure AI | Azure AI Services | Azure OpenAISenior-level Full TimeSandton - 1 Discovery Place, GP, …3d ago
-
Embedded Software Development Engineer ZAR 420K-504KARM | C# | Electronic schematics | Embedded System | Embedded System ArchitectureDisability programs | Hybrid work | Life insurance | Private medical care | Tailored financial programsMid-level Full TimeZA-Century City-Cape Town-The Matrix 2G and …3d ago
-
Senior Data Engineer - Agents Systems GBP 97K-136KApache Flink | Data Contracts | Data Processing | Data Quality | Feature StoreSenior-level Full TimeUnited Kingdom3d ago
-
Senior Data Engineer - AI Infrastructure GBP 101K-136KApache Flink | Data Contracts | Data Pipelines | Data Quality | Feature StoreSenior-level Full TimeUnited Kingdom3d ago
-
Senior Data Platform Consultant ZAR 600K-730KAWS | Azure | CI/CD | Classification | Data EngineeringHybrid workSenior-level Full TimeCape Town, South Africa R3d ago
-
AWS | Amazon Redshift | Apache Airflow | Azure | BigQueryCareer growth opportunities | Collaborative culture | Continuous learning | Flexible working arrangements | Fully remoteSenior-level Full TimeSouth Africa - Remote R4d ago
-
Senior-level Full TimeCape Town, South Africa4d ago
-
Agile | Artificial Intelligence | Data Architecture | Data Engineering | Data GovernanceHybrid work environment | Mentorship | Training and development | Work from home daysEntry-level Full TimeMidrand, GP, South Africa R4d ago
-
Bash | Cloud platform | Cost Optimization | Data Pipelines | Distributed SystemsAccess to cutting-edge technology | Autonomy | Collaborative culture | Distributed first | Fully remoteSenior-level Full TimeSouth Africa R4d ago
-
AI Development Specialist (Azure) OVA5342 ZAR 840K-1080K.NET | AI Search | AI Studio | AKS | API ManagementSenior-level Full TimeJohannesburg, South Africa4d ago
-
Mid-level Full TimeJohannesburg, ZA5d ago
-
AI Data | AI data pipelines | AWS | Access Management | AzureCompetitive pay | Growth opportunities | Remote work flexibilitySenior-level Full TimeHonduras - Tegucigalpa7d ago
-
AI readiness | AI workflows | AWS | Access Control | Access ManagementRemote work | US time zone collaboration | Video interview requirementSenior-level Full TimeArgentina - Buenos Aires7d ago
-
Senior Applied AI Engineer – Prompting & Evaluation ZAR 450K-650KAgentic Workflows | Azure OpenAI | Context design | Embeddings | GitHub CopilotCoaching and mentoring culture | Community of Practice | Hybrid work policy | Travel between UK and South Africa officesSenior-level Full TimeJohannesburg, SA, ZA | Stellenbosch, SA, …7d ago
-
Data Development Manager ZAR 464K-480KAutomation | Data Warehouse | Data pipeline | Database Design | ETLMid-level Full TimeJohannesburg, ZA8d ago
-
Mid-level ContractJohannesburg, South Africa8d ago
-
Machine Learning Engineer ZAR 420K-480KAutomation | Azure Machine Learning | Data analytics | Databricks | Machine LearningMid-level Full TimeSandton - 1 Discovery Place, GP, …9d ago
-
Azure | CI/CD | Data Governance | Data Lineage | Data PipelinesSenior-level ContractSandton, South Africa9d ago
-
Data Engineer ZAR 348K-420KAWS | AWS CDK | Azure | Azure Data | Azure Data FactoryGrowth opportunities | Hybrid work environment | Learning opportunities | Medical aid | Provident fundMid-level Full TimeJohannesburg, ZA10d ago
-
Senior-level Full TimeStellenbosch, ZA12d ago