Senior Data Engineer — Platform & Architecture
Brisbane, Queensland 4000, Australia
AUD 139K-146K (estimate) Senior-level Full Time
Tasks
- Define Databricks compute strategies cluster policies and workload optimization
- Define medallion layer structures
- Define table formats partitioning and clustering
- Design and maintain Databricks lakehouse architecture
- Design and maintain catalog and schema structures
- Establish data lifecycle policies
- Establish engineering standards for code quality testing and documentation
- Establish schema evolution practices
- Implement CI/CD workflows
- Implement cost optimization guardrails
- Manage data access controls classification and governance
- Mentor data engineers
- Monitor and optimize platform spend
- Review architecture decisions and guide complex data issues
Perks/Benefits
Skills/Tech-stack
CI/CD | Clustering | Cost Optimization | Data Governance | Data Modeling | Databricks | Delta Lake | Infrastructure as Code | Partitioning | Performance optimization | Python | SQL | Scalability | Schema evolution | Terraform | Unity Catalog | “as-code”
Education
N/A
Roles
Related jobs
-
Data and Analytics Senior AUD 130K-135KApache Spark | Azure Data | Azure Data Lake | Azure Data Lake Storage | Data FactoryCareer development | Part-time telework | Time to rechargeSenior-level Full TimeBrisbane, QLD, AU1h ago
-
Staff Engineer Chapter Lead – Generative AI AUD 137K-152KAWS | AWS Cloud | AWS Cloud Development Kit | AWS cloud development | AutogenSenior-level Full TimeSydney, NSW - CBP South, 11 …2d ago
-
Senior Robotics Controls Engineer USD 180K-200KActuator control | BLDC | C Sharp | C# | C++401k | Accident insurance | Commuter benefits | Company-Paid Holidays | Critical illness insuranceSenior-level Full TimeBrisbane, CA2d ago
-
A/B | A/B Testing | Analytics engineering | Application Performance Monitoring | Application performanceEquity package | Flexible leave options | Inclusive parental leave | Remote work within Australia | Wellbeing allowanceSenior-level Full TimeSydney, Australia R3d ago
-
Data and Analytics Support AUD 108K-120KADLS | Azure Data | Azure Data Lake | Azure Data Lake Storage | Data FactoryCareer development | Customer service focused benefits | Paid time to recharge | Part-time telework | Program to enhance cultureMid-level Full TimeBrisbane, QLD, AU3d ago
-
Data Management | Data integration | Data pipeline | Dynamic Systems | Dynamic systems modelingDiscounted private health cover | Employee Assistance Program (EAP) | Fitness Passport membership | Flu vaccination | Novated motor vehicle leasingMid-level Full TimeRichmond, New South Wales, Australia3d ago
-
Software Engineer (Level 2) - Data Solutions AUD 90K-120KAutomation Pipelines | Azure | DAX | Data Management | Data ModelingMid-level Full TimeCamberwell, VIC, AU, 31243d ago
-
Data Engineer AUD 125K-126KAgile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageAdditional annual leave | Education assistance program | Employer funded insurance | Parental leave | Wellbeing leaveMid-level Full TimeSydney - Darling Park, Australia R3d ago
-
Machine Learning Platform Engineer SGD 139K-180KAWS | Azure | CI/CD | Dask | Distributed SystemsHealth and wellbeing resources | Paid volunteer daysMid-level Full TimeSydney - Australia - Sydney, 2000 … R3d ago
-
Senior Data Engineer - ATP AUD 153K-153KAgile | Azure DevOps | CI/CD | Confluence | Data ModellingAdditional leave | Career development opportunities | Discount on UNSW courses | Discounts on retail education and fitness passport | Flexible working optionsSenior-level Full TimeSydney, NSW R3d ago
-
HPC Systems Engineer AUD 90K-120KAnsible | C# | Cause analysis | Configuration Management | DebuggingMid-level Full TimeSydney3d ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Flexible work location | Fully remoteSenior-level Contract Full TimeAustralia R3d ago
-
APIs | AWS | Apache Airflow | CI/CD | Data PipelinesAutonomy | Collaborative team | Flexible location | Fully remote | Operational excellence focusSenior-level Full TimeAustralia R3d ago
-
Data Engineer AUD 120K-126KCode review | Data Modeling | Data Warehousing | Data integration | Data pipelineDiscounted air travel | Discounted lounge membership | Discounts on travel insurance and car hire | Dress for your day | Employee assistance programMid-level Full TimeBrisbane, Melbourne, Sydney R4d ago
-
Databricks Solution Lead AUD 118K-127KADLS | AWS Glue | AWS S3 | Airflow | Amazon RedshiftAI Driven Learning Platform | Career development training | Discounted health insurance | Discounts on products and services | Industry partner trainingSenior-level Full TimeAdelaide, SA, Australia4d ago
-
C# | Data Analysis | Deep learning | GPU | High PerformanceDiscount on university courses | Discounted health insurance | Employee assistance program | Flexible working options | Generous paid leaveMid-level Full TimePerth, Crawley4d ago
-
Apache Spark | Azure Data | Azure Data Factory | Azure Data Platform | Azure DevOpsAgile sponsored training | Digital learning resources | Employee assistance program | Equipment incentive | Gym and wellbeing programSenior-level Full TimeNorth Sydney, New South Wales 2060, …4d ago
-
Forward Deployed Engineer - Sydney AUD 116K-131KCode review | Full Stack | Full-Stack Development | Generative AI | JavaScriptHybrid work | Relocation assistance | Travel up to 50 percentMid-level Full TimeSydney, Australia R4d ago
-
Senior Data Engineer (Microsoft Fabric, PySpark, SQL) AUD 133K-142KAPIs | Agile | Automated testing | Cloud Storage | Continuous DeliverySenior-level Full TimeEveleigh, NSW - 1 Locomotive Street, …4d ago
-
Staff Data Engineer (AWS, Java & Real-Time Data) AUD 140K-146KAWS | AWS Glue | AWS Lambda | Amazon S3 | Apache FlinkSenior-level Full TimeEveleigh, NSW - 5-7 Central Ave, …4d ago
-
Senior Forward Deployed AI Engineer AUD 137K-180KAWS | Agent systems | Agentic Orchestration | Anthropic API | Artificial IntelligenceSenior-level Full TimeSydney, Australia4d ago
-
Principal Software Engineer - Azure Storage AUD 119K-234KAlgorithms | Automation | Azure | C# | C++On-call rotation | Security screening requirementsSenior-level Full TimeAustralia4d ago
-
Senior Data Engineer AUD 129K-142KAWS | Automation | Cybersecurity | Data Governance | Data ModellingFlexible work options | Hybrid work | Mentoring cultureSenior-level Full TimeMacquarie Park, NSW, AU, 2113 R5d ago
-
Lead Data Analytics Platforms | Perth, Brisbane AUD 135K-140KAWS | Agile | Amazon Bedrock | Amazon SageMaker | AnalyticsExtended leave entitlements | Flexible working options | Paid parental leave | Parent roomsSenior-level Full TimeAustralia5d ago
-
Analytics Engineer AUD 104K-120KAgile | Apache Spark | Automation | Azure Data | Azure Data LakeEqual opportunity workplace | Flexible work options | Wellbeing supportMid-level Full TimeMelbourne Office (33X)5d ago