Principal Data Platform Engineer
Sydney, New South Wales, Australia
AUD 150K-185K (estimate) Senior-level Full Time
Tasks
- Build batch and real-time pipelines
- Build greenfield data platforms
- Consult clients on data solutions
- Design cloud-native data platforms
- Design data contracts
- Design data mesh
- Design data products
- Develop streaming and event driven pipelines
- Ensure security and compliance
- Implement data governance
- Improve data quality
- Improve observability
- Improve operational excellence
- Improve testing
- Lead end to end architecture
- Make ingestion storage processing access decisions
- Mentor engineers
- Optimize data access patterns
- Optimize data storage formats
- Orchestrate workflows with Airflow
- Orchestrate workflows with Dataflow
- Process and transform large datasets
- Review designs and code
- Set engineering standards and best practices
- Translate business needs into engineering decisions
Perks/Benefits
Skills/Tech-stack
AWS | Airflow | Amazon Kinesis | Apache Spark | ArgoCD | Avro | BigQuery | CI/CD | Cassandra | Cloud platform | DBT | Data Contracts | Data Products | Data mesh | Databricks | Delta Lake | Flink | GitHub Actions | Google Cloud | Google Cloud Platform | Great Expectations | Infrastructure as Code | Kafka | Kinesis | MongoDB | ORC | Parquet | PostgreSQL | PubSub | Pulumi | Python | SQL | Snowflake | Soda | Terraform | “as-code”
Education
N/A
Regions
Countries
States
Cities
Related jobs
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous culture | Competitive salary | Entrepreneurial team | Flexible remote work | Friendly work environmentMid-level Full TimeSydney, Australia20h ago
-
Data Engineer AUD 110K-140KAPIs | Azure | Azure Data | Azure Data Factory | Azure DevOpsCareer progression opportunities | Community partnerships | Flexibility in work location | Volunteering opportunitiesMid-level Full TimeMelbourne, VIC, AU1d ago
-
Data Engineer AUD 100K-120KAsynchronous programming | Azure DevOps | CI/CD | Data Quality | Data ReconciliationCoaching and career development | Flexible work arrangements | Flexible working options | Flu vaccinations | Health and wellbeing programMid-level Full TimeSydney, NSW, AU1d ago
-
Data Engineer AUD 118K-127KAWS | AWS Data | AWS Data Factory | AWS Glue | AgileDiscounted health insurance | Paid parental leave | Paid training | Product and service discounts | Wellbeing initiativesMid-level Full TimeMelbourne, VIC, Australia1d ago
-
Principal Data Engineer AUD 145K-151KAWS | Agile | Change Management | Cloud Native | Data LakeFlexible work location | Flexible working hours | Job-sharing | Part-time options | Support for job application processSenior-level Full TimeSydney, NSW - CBP North, 1 …1d ago
-
Data Engineer AUD 110K-140KAPI Integration | Azure Data | Azure Data Factory | Azure DevOps | Azure SynapseMid-level Full TimeMelbourne, VIC, AU2d ago
-
APIs | Cloud platform | Debugging | Google Cloud | Google Cloud PlatformDirect access to engineering support | High travelMid-level Full TimeSydney NSW, Australia2d ago
-
Data Engineer AUD 100K-126KAzure | Cloud Architecture | Data Architecture | Data Ingestion | Data ModelingAnnual training allowance | Career progression pathways | Corporate discounts | Employee assistance program | Employee recognition programsMid-level Full TimeMelbourne, Victoria 3000, Australia2d ago
-
GIS Data Engineer AUD 90K-112KAWS | Apache Spark | Azure | Azure Data | Azure Data FactoryCareer development | Flex leave | Flexible work arrangements | Paid parental leave | Wellness incentiveSenior-level Full TimePerth, WA, AU, 60002d ago
-
Data Engineer AUD 100K-126KAzure | Cloud Architecture | Data Ingestion | Data Modeling | Data integrationAnnual training allowance | Career progression pathways | Corporate discounts | Employee assistance program | Employee recognition programsMid-level Full TimeAdelaide, South Australia 5000, Australia2d ago
-
Senior Data Engineer - Data Platform AUD 135K-142KAPI Integration | AWS | AWS Glue | Apache Airflow | Apache FlinkExtra days of annual leave | Leadership development programs | LinkedIn Learning | Perkbox discountsSenior-level Full TimeSydney, Australia2d ago
-
Data Engineer AUD 113K-125KCloud Data | Cloud Data Platforms | Data Modeling | Data Pipelines | Data WarehousingFlexible work options | Hybrid workMid-level Full TimeMacquarie Park, NSW, AU, 21132d ago
-
Senior-level Full TimeEveleigh, NSW - 5-7 Central Ave, …2d ago
-
Senior Data Engineer - Data Platform AUD 135K-142KAWS | AWS CDK | AWS Glue | Apache Airflow | Apache FlinkExtra days of annual leave | Leadership development programs | LinkedIn Learning | Performance-Based Incentives | Perkbox discountsSenior-level Full TimePyrmont, New South Wales, Australia2d ago
-
AI Agents | AWS CDK | AWS Cloud | AWS CloudFormation | Access ManagementSenior-level Full TimePerth, Western Australia, AUS2d ago
-
.NET | AWS CDK | AWS CloudFormation | Amazon CloudWatch | Amazon Web ServicesSenior-level Full TimePerth, Western Australia, AUS2d ago
-
Data Engineer AUD 146K-146KAnalytics | Automated testing | CI/CD | Data Governance | Data LineageAdditional paid volunteering | Annual leave | Certification support | Flexible work options | Membership supportSenior-level Full TimeAdelaide, AU2d ago
-
Data Platform Engineer AUD 100K-125KAlerting | Apache Airflow | Automation | Azure Data | Azure Data FactoryFlexible work arrangements | Paid parental leave | Paid volunteering days | Part-time options | Professional development reimbursementSenior-level Full TimeSydney, NSW, AU2d ago
-
Data Scientist, People Analytics AUD 129K-142KAWS | AWS Glue | AWS Lambda | Amazon Bedrock | Amazon S3Career development opportunities | TrainingSenior-level Full TimeSydney, NSW - CBP North, 1 …3d ago
-
AWS | Air handlers | Asset Management | Building monitoring | Building monitoring systemsFlexible work hours | Mentorship and career growth | Work-life balanceMid-level Full TimeMelbourne, Victoria, AUS3d ago
-
AWS | Air Handling | Building monitoring | Building monitoring systems | Capacity PlanningFlexible work hours | Work-life balanceMid-level Full TimeMelbourne, Victoria, AUS3d ago
-
AWS | Asset Management | Building monitoring | Building monitoring systems | CRAHFlexible work hoursMid-level Full TimeMelbourne, Victoria, AUS3d ago
-
Data Engineer - 12 Month Opportunity AUD 110K-140KAWS | Agile | Artificial Intelligence | Bash | Business IntelligenceAll-team events | Hybrid social events | Learning and development | Team discounts | Volunteer leaveMid-level Full TimeMelbourne, VIC, AU R4d ago
-
Senior Applied AI Engineer AUD 119K-161KAgent Orchestration | Agent systems | Bias Testing | Context engineering | Evaluation datasetsCoaching | Equity | Extended end of year break | Home workspace setup budget | Inclusive parental leaveSenior-level Full TimeMelbourne4d ago
-
Staff Machine Learning Engineer - Recommendations AUD 119K-136KData-Driven Decision Making | Data-driven | Decision Making | Machine Learning | Model EvaluationEquity packages | Flexible leave options | Parental leave | Virtual interviews | Wellbeing allowanceSenior-level Full TimeSydney, Australia4d ago