Principal Data Platform Engineer
Sydney, New South Wales, Australia
AUD 150K-185K (estimate) Senior-level Full Time
Tasks
- Build batch and real-time pipelines
- Build greenfield data platforms
- Consult clients on data solutions
- Design cloud-native data platforms
- Design data contracts
- Design data mesh
- Design data products
- Develop streaming and event driven pipelines
- Ensure security and compliance
- Implement data governance
- Improve data quality
- Improve observability
- Improve operational excellence
- Improve testing
- Lead end to end architecture
- Make ingestion storage processing access decisions
- Mentor engineers
- Optimize data access patterns
- Optimize data storage formats
- Orchestrate workflows with Airflow
- Orchestrate workflows with Dataflow
- Process and transform large datasets
- Review designs and code
- Set engineering standards and best practices
- Translate business needs into engineering decisions
Perks/Benefits
Skills/Tech-stack
AWS | Airflow | Amazon Kinesis | Apache Spark | ArgoCD | Avro | BigQuery | CI/CD | Cassandra | Cloud platform | DBT | Data Contracts | Data Products | Data mesh | Databricks | Delta Lake | Flink | GitHub Actions | Google Cloud | Google Cloud Platform | Great Expectations | Infrastructure as Code | Kafka | Kinesis | MongoDB | ORC | Parquet | PostgreSQL | PubSub | Pulumi | Python | SQL | Snowflake | Soda | Terraform | “as-code”
Education
N/A
Regions
Countries
States
Cities
Related jobs
-
Benchmarking | CI/CD | Document Intelligence | Embeddings | Explainable AIFully remote work | Healthcare coverage | International opportunities | Long-term contract | Technical ownershipSenior-level Full TimeAustralia R22h ago
-
AWS | Airflow | Azure | Azure Data | Azure Data FactoryCareer Development Programs | Discounted health insurance | Discounts on products and services | Mentorship | Paid parental leaveSenior-level Full TimeMelbourne, VIC, Australia1d ago
-
Data and Analytics Senior AUD 130K-135KApache Spark | Azure Data | Azure Data Lake | Azure Data Lake Storage | Data FactoryCareer development | Part-time telework | Time to rechargeSenior-level Full TimeBrisbane, QLD, AU1d ago
-
Software Engineer - AI Developer Tooling Workflows AUD 100K-100KAI Agents | APIs | CI/CD | Continuous Deployment | Continuous integrationEquity packages | Flexible leave options | Hybrid work flexibility | Inclusive parental leave | Wellbeing allowanceMid-level Full TimeSydney, Australia R1d ago
-
Senior Data Engineer — Integration & Pipelines AUD 118K-120KAPI Integration | Anomaly Detection | Apache Spark | CI/CD | Data GovernanceCareer progression | EAP support | Fitness passport | Salary packaging | Social eventsSenior-level Full TimeBowen Hills, Queensland 4006, Australia1d ago
-
Senior Data Engineer — Platform & Architecture AUD 140K-147KCI/CD | Cost Optimization | Data Clustering | Data Governance | Data Lifecycle ManagementCareer progression | EAP support | Fitness passport | Modern office facilities | Salary packagingSenior-level Full TimeBrisbane, Queensland 4000, Australia1d ago
-
Data Engineer AUD 115K-120KAzure Data | Azure Data Factory | Azure Synapse | Azure Synapse Analytics | BI semantic modelsCommunity service leave | Cultural leave | Discounts | Flexible working | Inclusive cultureMid-level Full TimeVIC - Melbourne - 655 Collins … R1d ago
-
Principal Data Platform Engineer (AWS) AUD 150K-170KAWS | AWS Glue | AWS Lambda | Amazon EMR | Amazon RedshiftFlexible work location | Flexible working hours | Job-sharing | Part-time arrangements | Technology hubs in Melbourne and PerthSenior-level Full TimeSydney, NSW - CBP North, 1 …1d ago
-
Principal Data Scientist AUD 133K-146KAWS | Agentic AI | Apache Spark | Azure | CI/CDFlexible work arrangementsSenior-level Full TimeEveleigh, NSW - 5-7 Central Ave, …1d ago
-
Senior AI Engineer - Data Scientist (Sydney/Melbourne) AUD 143K-180KAPIs | Agent systems | Autogen | CI/CD | Computer VisionCertification support | Global mobility | Inclusive culture | Learning platformsSenior-level Full TimeMelbourne, AU1d ago
-
Cloud / Infrastructure Architect — JAPAC Flex AUD 128K-180KBigQuery | CI/CD | Cloud Build | Cloud Composer | Cloud FunctionsSenior-level Full TimeAustralian Capital Territory, Australia1d ago
-
Staff Engineer Chapter Lead – Generative AI AUD 137K-152KAWS | AWS Cloud | AWS Cloud Development Kit | AWS cloud development | AutogenSenior-level Full TimeSydney, NSW - CBP South, 11 …3d ago
-
Senior Robotics Controls Engineer USD 180K-200KActuator control | BLDC | C Sharp | C# | C++401k | Accident insurance | Commuter benefits | Company-Paid Holidays | Critical illness insuranceSenior-level Full TimeBrisbane, CA4d ago
-
Data and Analytics Support AUD 108K-120KADLS | Azure Data | Azure Data Lake | Azure Data Lake Storage | Data FactoryCareer development | Customer service focused benefits | Paid time to recharge | Part-time telework | Program to enhance cultureMid-level Full TimeBrisbane, QLD, AU4d ago
-
Data Management | Data integration | Data pipeline | Dynamic Systems | Dynamic systems modelingDiscounted private health cover | Employee Assistance Program (EAP) | Fitness Passport membership | Flu vaccination | Novated motor vehicle leasingMid-level Full TimeRichmond, New South Wales, Australia4d ago
-
Software Engineer (Level 2) - Data Solutions AUD 90K-120KAutomation Pipelines | Azure | DAX | Data Management | Data ModelingMid-level Full TimeCamberwell, VIC, AU, 31244d ago
-
Data Engineer AUD 125K-126KAgile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageAdditional annual leave | Education assistance program | Employer funded insurance | Parental leave | Wellbeing leaveMid-level Full TimeSydney - Darling Park, Australia R4d ago
-
Machine Learning Platform Engineer SGD 139K-180KAWS | Azure | CI/CD | Dask | Distributed SystemsHealth and wellbeing resources | Paid volunteer daysMid-level Full TimeSydney - Australia - Sydney, 2000 … R4d ago
-
Senior Data Engineer - ATP AUD 153K-153KAgile | Azure DevOps | CI/CD | Confluence | Data ModellingAdditional leave | Career development opportunities | Discount on UNSW courses | Discounts on retail education and fitness passport | Flexible working optionsSenior-level Full TimeSydney, NSW R4d ago
-
HPC Systems Engineer AUD 90K-120KAnsible | C# | Cause analysis | Configuration Management | DebuggingMid-level Full TimeSydney4d ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Flexible work location | Fully remoteSenior-level Contract Full TimeAustralia R5d ago
-
APIs | AWS | Apache Airflow | CI/CD | Data PipelinesAutonomy | Collaborative team | Flexible location | Fully remote | Operational excellence focusSenior-level Full TimeAustralia R5d ago
-
Data Engineer AUD 120K-126KCode review | Data Modeling | Data Warehousing | Data integration | Data pipelineDiscounted air travel | Discounted lounge membership | Discounts on travel insurance and car hire | Dress for your day | Employee assistance programMid-level Full TimeBrisbane, Melbourne, Sydney R5d ago
-
Databricks Solution Lead AUD 118K-127KADLS | AWS Glue | AWS S3 | Airflow | Amazon RedshiftAI Driven Learning Platform | Career development training | Discounted health insurance | Discounts on products and services | Industry partner trainingSenior-level Full TimeAdelaide, SA, Australia5d ago
-
C# | Data Analysis | Deep learning | GPU | High PerformanceDiscount on university courses | Discounted health insurance | Employee assistance program | Flexible working options | Generous paid leaveMid-level Full TimePerth, Crawley5d ago