Senior Data Engineer — Platform & Architecture
Brisbane, Queensland 4000, Australia
AUD 140K-147K (estimate) Senior-level Full Time
Tasks
- Define Databricks compute strategies cluster policies workload optimization
- Define medallion layer structures
- Define table formats partitioning and clustering
- Design and maintain Databricks lakehouse architecture
- Design and maintain Unity Catalog structures
- Establish engineering standards code quality testing CI/CD and documentation
- Establish schema evolution practices
- Implement cost optimization guardrails
- Implement data lifecycle policies
- Manage data access controls classification and governance standards
- Mentor data engineers and provide technical guidance
- Monitor and optimize platform spend
- Review architecture decisions for complex data challenges
Perks/Benefits
- Career progression
- EAP support
- Fitness passport
- Modern office facilities
- Salary packaging
- Social events
Skills/Tech-stack
CI/CD | Cost Optimization | Data Clustering | Data Governance | Data Lifecycle Management | Data Modeling | Data Security | Data lifecycle | Databricks | Delta Lake | Infrastructure as Code | Lifecycle Management | Medallion Architecture | Performance optimization | Python | SQL | Table Partitioning | Terraform | Unity Catalog | “as-code”
Education
N/A
Roles
Related jobs
-
Apache Spark | Azure | CI/CD | Containerization | Data ModelingFlexible working options | Paid parental leave | Paid volunteering days | Wellbeing leaveMid-level Full TimeMelbourne, VIC, AU11h ago
-
Benchmarking | CI/CD | Document Intelligence | Embeddings | Explainable AIFully remote work | Healthcare coverage | International opportunities | Long-term contract | Technical ownershipSenior-level Full TimeAustralia R23h ago
-
AWS | Airflow | Azure | Azure Data | Azure Data FactoryCareer Development Programs | Discounted health insurance | Discounts on products and services | Mentorship | Paid parental leaveSenior-level Full TimeMelbourne, VIC, Australia1d ago
-
Data and Analytics Senior AUD 130K-135KApache Spark | Azure Data | Azure Data Lake | Azure Data Lake Storage | Data FactoryCareer development | Part-time telework | Time to rechargeSenior-level Full TimeBrisbane, QLD, AU1d ago
-
Software Engineer - AI Developer Tooling Workflows AUD 100K-100KAI Agents | APIs | CI/CD | Continuous Deployment | Continuous integrationEquity packages | Flexible leave options | Hybrid work flexibility | Inclusive parental leave | Wellbeing allowanceMid-level Full TimeSydney, Australia R1d ago
-
Senior Data Engineer — Integration & Pipelines AUD 118K-120KAPI Integration | Anomaly Detection | Apache Spark | CI/CD | Data GovernanceCareer progression | EAP support | Fitness passport | Salary packaging | Social eventsSenior-level Full TimeBowen Hills, Queensland 4006, Australia1d ago
-
Data Engineer AUD 115K-120KAzure Data | Azure Data Factory | Azure Synapse | Azure Synapse Analytics | BI semantic modelsCommunity service leave | Cultural leave | Discounts | Flexible working | Inclusive cultureMid-level Full TimeVIC - Melbourne - 655 Collins … R1d ago
-
Principal Data Platform Engineer (AWS) AUD 150K-170KAWS | AWS Glue | AWS Lambda | Amazon EMR | Amazon RedshiftFlexible work location | Flexible working hours | Job-sharing | Part-time arrangements | Technology hubs in Melbourne and PerthSenior-level Full TimeSydney, NSW - CBP North, 1 …1d ago
-
Senior AI Engineer - Data Scientist (Sydney/Melbourne) AUD 143K-180KAPIs | Agent systems | Autogen | CI/CD | Computer VisionCertification support | Global mobility | Inclusive culture | Learning platformsSenior-level Full TimeMelbourne, AU1d ago
-
Cloud / Infrastructure Architect — JAPAC Flex AUD 128K-180KBigQuery | CI/CD | Cloud Build | Cloud Composer | Cloud FunctionsSenior-level Full TimeAustralian Capital Territory, Australia1d ago
-
Staff Engineer Chapter Lead – Generative AI AUD 137K-152KAWS | AWS Cloud | AWS Cloud Development Kit | AWS cloud development | AutogenSenior-level Full TimeSydney, NSW - CBP South, 11 …3d ago
-
Senior Robotics Controls Engineer USD 180K-200KActuator control | BLDC | C Sharp | C# | C++401k | Accident insurance | Commuter benefits | Company-Paid Holidays | Critical illness insuranceSenior-level Full TimeBrisbane, CA4d ago
-
Data and Analytics Support AUD 108K-120KADLS | Azure Data | Azure Data Lake | Azure Data Lake Storage | Data FactoryCareer development | Customer service focused benefits | Paid time to recharge | Part-time telework | Program to enhance cultureMid-level Full TimeBrisbane, QLD, AU4d ago
-
Data Management | Data integration | Data pipeline | Dynamic Systems | Dynamic systems modelingDiscounted private health cover | Employee Assistance Program (EAP) | Fitness Passport membership | Flu vaccination | Novated motor vehicle leasingMid-level Full TimeRichmond, New South Wales, Australia4d ago
-
Software Engineer (Level 2) - Data Solutions AUD 90K-120KAutomation Pipelines | Azure | DAX | Data Management | Data ModelingMid-level Full TimeCamberwell, VIC, AU, 31244d ago
-
Data Engineer AUD 125K-126KAgile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageAdditional annual leave | Education assistance program | Employer funded insurance | Parental leave | Wellbeing leaveMid-level Full TimeSydney - Darling Park, Australia R4d ago
-
Machine Learning Platform Engineer SGD 139K-180KAWS | Azure | CI/CD | Dask | Distributed SystemsHealth and wellbeing resources | Paid volunteer daysMid-level Full TimeSydney - Australia - Sydney, 2000 … R4d ago
-
Senior Data Engineer - ATP AUD 153K-153KAgile | Azure DevOps | CI/CD | Confluence | Data ModellingAdditional leave | Career development opportunities | Discount on UNSW courses | Discounts on retail education and fitness passport | Flexible working optionsSenior-level Full TimeSydney, NSW R4d ago
-
HPC Systems Engineer AUD 90K-120KAnsible | C# | Cause analysis | Configuration Management | DebuggingMid-level Full TimeSydney4d ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Flexible work location | Fully remoteSenior-level Contract Full TimeAustralia R5d ago
-
APIs | AWS | Apache Airflow | CI/CD | Data PipelinesAutonomy | Collaborative team | Flexible location | Fully remote | Operational excellence focusSenior-level Full TimeAustralia R5d ago
-
Data Engineer AUD 120K-126KCode review | Data Modeling | Data Warehousing | Data integration | Data pipelineDiscounted air travel | Discounted lounge membership | Discounts on travel insurance and car hire | Dress for your day | Employee assistance programMid-level Full TimeBrisbane, Melbourne, Sydney R5d ago
-
Databricks Solution Lead AUD 118K-127KADLS | AWS Glue | AWS S3 | Airflow | Amazon RedshiftAI Driven Learning Platform | Career development training | Discounted health insurance | Discounts on products and services | Industry partner trainingSenior-level Full TimeAdelaide, SA, Australia5d ago
-
C# | Data Analysis | Deep learning | GPU | High PerformanceDiscount on university courses | Discounted health insurance | Employee assistance program | Flexible working options | Generous paid leaveMid-level Full TimePerth, Crawley5d ago
-
Apache Spark | Azure Data | Azure Data Factory | Azure Data Platform | Azure DevOpsAgile sponsored training | Digital learning resources | Employee assistance program | Equipment incentive | Gym and wellbeing programSenior-level Full TimeNorth Sydney, New South Wales 2060, …5d ago