Data Operations Engineer
Cape Town, WC, South Africa
A ZAR 700K-960K (estimate) Senior-level Full Time
Tasks
- Administer Linux and Windows on AWS using SSH and RDP
- Analyze large datasets for trends and patterns
- Author AWS security logging and user permissions
- Build and maintain data pipelines
- Build and manage CI/CD pipelines
- Collaborate with stakeholders on data requirements
- Design bulk and delta extraction transformation loading patterns
- Design scalable data ingestion and transformation processes
- Develop infrastructure as code templates
- Implement IAM security policies
- Implement data governance practices
- Ingest large complex datasets
- Manage change requests with RFC process
- Process structured and unstructured data
- Provision AWS infrastructure with CloudFormation
- Support operational data services
Perks/Benefits
- N/A
Skills/Tech-stack
API | AWS CodeBuild | AWS CodeCommit | AWS CodePipeline | AWS DynamoDB | AWS Lambda | AWS Transit Gateway | Amazon EC2 | Amazon EMR | Amazon RDS | Amazon S3 | Amazon VPC | Amazon Web Services | Apache Hive | Apache Spark | CI/CD | CloudFormation | DevOps | ELT | ETL | IAM | Infrastructure as Code | Linux | Python | RDP | SQL | SSH | Transit Gateway | Web Services | Windows server | “as-code”
Education
Related jobs
-
NumPy | Pandas | Python | SciPy | SymPyPart-time work | Project based workSenior-level FreelanceSouth Africa - Remote R19h ago
-
API Integration | Generative AI | Language Models | Large Language Models | Machine LearningAutonomy | Rapid iteration culture | Remote workSenior-level Full TimeCape Town, Western Cape, South Africa1d ago
-
Canonical Data Modelling | Data Management | Data Modelling | Data Normalization | Data QualityMid-level Full TimeStellenbosch, WC, South Africa1d ago
-
Ab Initio | Azure | Data Governance | Data Modeling | Data SecuritySenior-level Full TimeSandton, South Africa1d ago
-
Advanced Excel | Azure Synapse | Azure Synapse Analytics | Data Analysis | Data GovernanceSenior-level Full TimeJohannesburg, GP, South Africa1d ago
-
API Design | Application Support | Design Patterns | DevOps | ElasticsearchSenior-level Full TimeJohannesburg, GP, South Africa1d ago
-
AWS Glue | AWS Lambda | Amazon Athena | Amazon EC2 | Amazon S3Senior-level Full TimeCape Town, Western Cape, South Africa1d ago
-
APIs | Data Engineering | Data Pipelines | Data Streams | JSONSenior-level Full TimeCape Town, WC, South Africa1d ago
-
A/B | A/B Testing | AWS | Amazon SageMaker | AutomlSenior-level Full TimeJohannesburg, South Africa1d ago
-
Analytics | Apache Kafka | Big Data | Cluster tuning | ContainerizationRemote workSenior-level Full TimeJohannesburg, GP, South Africa R1d ago
-
Senior-level Full TimeDurban North, KZN, South Africa1d ago
-
API Development | AWS | Amazon SageMaker | Apache Spark | AzureSenior-level Full TimeJohannesburg, South Africa1d ago
-
API Development | AWS | Amazon SageMaker | Apache Spark | CI/CDSenior-level Full TimeCape Town, Western Cape, South Africa1d ago
-
Azure Data | Azure Data Factory | Azure DevOps | Azure Synapse | Azure Synapse AnalyticsSenior-level Full TimeJohannesburg, GP, South Africa1d ago
-
Azure | Data Engineering | Data Lake | Data Lineage | Data ModelingSenior-level Full TimeRandburg, GP, South Africa1d ago
-
Apache Airflow | Apache Spark | Cloud infrastructure | Data Architecture | Data GovernanceSenior-level Full TimeCape Town, WC, South Africa1d ago
-
AWS | Apache Spark | Azure Synapse | Azure Synapse Analytics | Cloud ComputingSenior-level Full TimeJohannesburg, GP, South Africa1d ago
-
API Development | AWS SageMaker | Amazon Web Services | Apache Spark | CI/CDSenior-level Full TimePort Elizabeth, South Africa1d ago
-
Senior-level Full TimeJohannesburg, South Africa1d ago
-
Ab Initio | Agile | Azure | DB2 | Data LineageAgile working environment | Minimal supervision | Work independentlySenior-level Full TimeJohannesburg, GP, South Africa1d ago
-
Azure | Data Lake | Data Lineage | Data Modeling | Data PipelinesSenior-level Full TimeJohannesburg, GP, South Africa1d ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Databricks | Azure DevOpsSenior-level Full TimeJohannesburg, GP, South Africa1d ago
-
Senior-level Full TimeJohannesburg, GP, South Africa1d ago
-
Senior-level Full TimeJohannesburg, GP, South Africa1d ago
-
Entry-level Full TimeStellenbosch, South Africa1d ago