Senior Data Engineer – Cloud Data Platform (Network & Observability)
Tasks
- Build and maintain ETL/ELT pipelines
- Build and operate masking pipelines
- Build batch and near real time ingestion frameworks
- Collaborate with engineering network and business teams
- Define data lake zoning best practices
- Design and operate AWS data pipelines
- Develop reusable data models and curated data layers
- Ensure data governance with lineage and auditability
- Implement CI CD for data platform deployments
- Implement data masking and tokenisation for PII
- Implement infrastructure as code for data platforms
- Implement partitioning and performance optimisation
- Provide mentorship and technical leadership
- Transform raw data into analytics-ready datasets
- Translate requirements into scalable data architectures
Perks/Benefits
- Additional paid leave
- Corporate partner discounts
- Hybrid work
- Learning platform access
- Mobile Plan
- NBN plan
Skills/Tech-stack
AWS | AWS Glue | AWS Lambda | Amazon Redshift | Amazon S3 | Apache Airflow | Auditability | CI/CD | Data Governance | Data Lineage | Data masking | ELT | ETL | IAM | Infrastructure as Code | PII | Python | SQL | Tokenization | “as-code”
Education
N/A
Roles
Related jobs
-
Senior Machine Learning / ML Engineer AUD 170K-190KAirflow | Amazon EMR | Apache Spark | Dask | Data orchestrationEngineering standards | Hybrid work model | MentoringSenior-level Full TimeAU: Melbourne: (260 Burwood Rd)13h ago
-
Senior AI/Analytics Engineer - CTT AUD 130K-150KApache Airflow | Apache Kafka | Autogen | Container Services | DockerEmployee discounts | Employee-only mortgage rates | Flexible work arrangements | Learning and development | Leave optionsSenior-level Full TimeSydney, NSW, Australia15h ago
-
Sr. Forward Deployed Engineer AUD 145K-185KAnthropic | Apache Spark | Artificial Intelligence | Backend Development | Cloud ComputingSenior-level Full TimeMelbourne, Australia; Sydney, Australia18h ago
-
Staff Data Engineer AUD 125K-138KAWS | Apache Spark | Cloud Computing | Cloud platform | Data ModelingEmissions reduction focus net zero by FY2050 | Flexible working | Paid leave | Parental leave | Personal Flexi DaysSenior-level Full TimeCremorne, Victoria, Australia1d ago
-
Data labeling | Data pipeline | Debugging | Inference | Inference OptimizationEquity compensation | Flexible leave options | Inclusive parental leave | Remote work within Australia | Wellbeing allowanceSenior-level Full TimeSydney, Australia R1d ago
-
AI Engineer (Senior) AUD 145K-156KAWS | Agentic AI | Agile | Amazon SageMaker | Artificial IntelligenceSenior-level Full TimeMelbourne, VIC, Australia1d ago
-
AWS | Ataccama | Azure | Cloud platform | CollibraExperian Shares Discount | Flexible time off | Flexible work environment | Hybrid work model | Novated leasingSenior-level Full TimeSouthbank, VIC, Australia R1d ago
-
Communication skills | Data Analysis | Data Auditing | Data Catalog | Data CleansingFlexible time off | Flexible work environment | Hybrid model | Paid parental leave | Study leaveSenior-level Full TimeNorth Sydney, Australia R1d ago
-
Data Engineer AUD 90K-110KAWS Cloud | AWS Cloud Development Kit | AWS Glue | AWS cloud development | Amazon AuroraAdditional leave | Board games | Hybrid work | In-house workshops | Learning and development budgetEntry-level Full TimeSydney, New South Wales, Australia1d ago
-
Data Platform Engineer AUD 100K-120KAWS | Alerting | CI/CD | DBT | DataOps5 weeks annual leave | Family care parental leave | Global mobility opportunities | Hybrid working | Personal Carer LeaveMid-level Full TimeNorth Sydney, AU1d ago
-
NumPy | Pandas | Python | SciPy | SymPyFlexible part-time schedule | Freelance work | Project based workSenior-level FreelanceAustralia - Remote R1d ago
-
Senior-level Full TimeMelbourne, Australia2d ago
-
Entry-level Full TimeMelbourne, Australia2d ago
-
AI Engineer - DDAI AUD 126K-150KA/B | A/B Testing | Apache Spark | Autogen | B testingAdditional leave | Community volunteering opportunities | Employee benefits | Flexible work | Learning and developmentMid-level Full TimeSydney, NSW, Australia2d ago
-
Mid-level Full TimeSydney, NSW2d ago
-
Azure Functions | CI/CD | Code review | Databricks | Delta LakeFlexible working options | Paid volunteering days | Parental leave | Retail discounts | Return to work supportSenior-level Full TimeMelbourne, VIC, AU2d ago
-
Combinatorics | Graph theory | Mathematical Modeling | NumPy | Number theoryFlexible schedule | Freelance project-based work | Part-time opportunitiesSenior-level FreelanceAustralia - Remote R2d ago
-
Lead Data Engineer AUD 118K-127KAccess Control | Ansible | Batch Processing | CI/CD | ChefCareer development | Collaborative environment | Flexible work culture | Supportive team cultureSenior-level Full TimeCanberra, Australian Capital Territory 2600, Australia2d ago
-
Data Engineer AUD 80K-150KDBT | Data Ingestion | Data Transformation | ELT | ETLAdditional leave | Corporate partner discounts | Free NBN plan | Free mobile plan | Hybrid workMid-level Full TimeBarangaroo, NSW, AU5d ago
-
AI Engineer - Conversational AUD 200K-250KAI voice | APIs | AWS | Agile | Amazon Connect5 days per week office based | Employee discounts | On-site parkingExecutive-level Full TimeSydney Olympic Park, New South Wales …5d ago
-
Senior AI Engineer AUD 142K-163KAWS | Autogen | Azure | CI/CD | Clean CodeAutonomy in role | Learning and development programs | Mentorship cultureSenior-level Full TimeMelbourne, Australia R5d ago
-
Data Engineer - Coles 360 - Melbourne AUD 110K-140KAzure | BigQuery | CI/CD | DBT | Data ModelingDiscounts on supermarket and liquor | Fitness classes | Free parking | Gym facility | Hybrid working optionsMid-level Full TimeHawthorn East Melbourne, VIC, AU, 31235d ago
-
LLMs | Langchain | MLOps | NumPy | PandasProject based workMid-level FreelanceAustralia - Remote R5d ago
-
Freelance Machine Learning Engineer AUD 110KLLMs | Langchain | MLOps | Machine Learning | NumPyPart-time availability | Project based workMid-level FreelanceAustralia - Remote R5d ago
-
Principal Data Engineer (AWS) AUD 140K-154KAWS | Airflow | Apache Spark | Automated testing | Batch ProcessingCareer support | Flexible work arrangements | Flexible work locations | Job share options | Part-time arrangementsSenior-level Full TimeSydney, NSW - CBP North, 1 …5d ago