Lead Databricks Engineer
Adelaide, SA, Australia
AUD 70K-80K (estimate) Senior-level Full Time
Tasks
- Build ML ready datasets
- Build batch and real-time pipelines
- Design Lakehouse platforms
- Develop and deploy Delta Live Tables
- Develop structured and unstructured ingestion frameworks
- Enable data science workflows
- Ensure security, privacy, compliance
- Implement CI/CD and DevOps best practices
- Implement Databricks data pipelines
- Implement data governance quality checks monitoring observability
- Integrate cloud data services
- Lead and mentor data engineering teams
- Manage infrastructure with Infrastructure as Code
- Manage metadata and lineage with Unity Catalog
- Optimize performance, cost, and reliability
- Own end-to-end project delivery
- Support real time analytics with Structured Streaming
- Translate business needs into data solutions
Perks/Benefits
- Career Development Programs
- Discounted health insurance
- Discounts on products and services
- Paid parental leave
- Training programs
- Well-being initiatives
Skills/Tech-stack
AWS Glue | Airflow | Amazon Redshift | Amazon S3 | Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOps | Azure Synapse | Azure Synapse Analytics | CI/CD | Data Factory | Data Governance | Data Lake Storage | Data Lineage | Data Modelling | Data Quality | Data Warehousing | Databricks | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | GitHub | Infrastructure as Code | Metadata Management | Monitoring | Observability | PySpark | Python | SQL | Snowflake | Structured Streaming | Synapse Analytics | Terraform | Unity Catalog | “as-code”
Education
N/A
Roles
Regions
Countries
States
Related jobs
-
Senior Data Engineer (Melbourne/Adelaide) AUD 122K-143KApache Airflow | Automation | Azure | Azure Data | Azure Data FactoryFlexible hours | Free food and snacks | Opportunity to drive projects | Socials and Community Groups | Training and developmentSenior-level Full TimeMelbourne, Victoria, Australia13h ago
-
Senior Data Engineer AUD 122K-135KAWS | Airflow | Bedrock | CI/CD | DBT24/7 support | Financial wellbeing services | Hybrid working | Loyalty leave | Medical supportSenior-level Full TimeSydney, NSW, Australia14h ago
-
Forward Deployed Engineer III, Gen AI, Retail AUD 135K-180KAPI Integration | Agent systems | Agentic AI | Cloud Computing | CrewAISenior-level Full TimeSydney NSW, Australia; Melbourne VIC, Australia1d ago
-
Acceptance Testing | C# | C++ | Change Control | Configuration ManagementContinuous learning | Modern facilities | Supportive engineering teamMid-level Full TimeAdelaide, South Australia 5000, Australia1d ago
-
Staff Data Engineer AUD 130K-140KAI | Data Engineering | Data Governance | Data Interoperability | Data ModelingEmployee discounts | Flexible work arrangements | Learning and development | Leave options | Volunteering opportunitiesSenior-level Full TimeNSW, Australia1d ago
-
Applied AI Engineer (Australia) AUD 100K-100KAutomation | Compliance | Evaluation | Fintech | GuardrailsMid-level Full TimeSydney, Australia1d ago
-
Applied AI Engineer - AI Neobank App (Australia) AUD 100K-100KAPI Integration | Agentic Systems | Data Privacy | Evaluation | LLM APIsMid-level Full TimeSydney, Australia1d ago
-
Data Scientist, Specialist AUD 121K-140KAWS SageMaker | Artificial Intelligence | Azure Machine Learning | CI/CD | Data EngineeringHybrid work modelSenior-level Full TimeAUS - Wesley Place, Australia1d ago
-
Senior Analyst, Data & Analytics AUD 60K-78KAI workflows | Agentic AI | Agentic AI Workflows | Business Intelligence | DBTEmployee network groups | Hybrid work model | Travel discountSenior-level Full TimeMelbourne, Victoria, Australia R1d ago
-
Senior Data Engineer, Core Banking Migrations AUD 146K-205KAWS Glue | Amazon Athena | Amazon EKS | Amazon MWAA | Amazon S3Senior-level Full TimeSydney2d ago
-
Amazon Redshift | DBT | Data Modeling | Databricks | ETLCareer development | Certification support | Coaching & mentoring | Global career opportunities | Hybrid remote workSenior-level Full TimeNorth Sydney, New South Wales, Australia2d ago
-
Senior Analytics Engineer - User Voice AUD 148K-170KAlerting | Analytics engineering | Census | DBT | Data GovernanceEquity packages | Flexible leave options | Hybrid work | Inclusive parental leave | Wellbeing allowanceSenior-level Full TimeMelbourne, Australia R2d ago
-
Data Engineer - Data Platforms AUD 90K-120KAWS | CI/CD | DBT | Data Modeling | Data QualityHybrid workMid-level Full TimeAU: Melbourne: (260 Burwood Rd)2d ago
-
Technical Architect, AI/ML AUD 156K-200KAPI | AWS SageMaker | Agentic Frameworks | Amazon Web Services | Apache SparkTravel opportunitiesSenior-level Full TimeAU-Sydney2d ago
-
Data Engineer AUD 110K-130KAmazon Redshift | Azure | CDP | Customer Analytics | Data AnalysisDiscounted Accommodation | Discounted Travel Insurance | Discounted car hire | Discounted experiences | Discounted lounge membershipMid-level Full TimeBrisbane, Melbourne, Sydney2d ago
-
Amazon Redshift | DBT | Data Modelling | Data Warehousing | DatabricksBuddy scheme | Career development | Certification support | Coaching & mentoring | Global career opportunitiesSenior-level Full TimeNorth Sydney, New South Wales, Australia2d ago
-
Senior AI Engineer, AI Accelerator AUD 130K-157KAWS | Agent Design | Artificial Intelligence | Azure OpenAI | CI/CDCareer development opportunities | Convenient city location | Free Membership | Hybrid working | Modern end of trip facilitiesSenior-level Full TimeMelbourne, VIC, AU2d ago
-
Senior Data Engineer - Data Platform AUD 150K-170KAWS | Automated testing | CI/CD | DBT | Data ModelingFlexible working hours | Hybrid work model | Knowledge sharing culture | Remote work support | Team boost daysSenior-level Full TimeAU: Melbourne: (260 Burwood Rd)3d ago
-
Staff Data Engineer (Data Architect) - Sydney/Melbourne AUD 150K-170KAPI Integration | Batch streaming | Cloud infrastructure | DBT | Data ArchitectureHybrid work model | Knowledge sharing culture | Mentoring | Remote work optionSenior-level Full TimeAU: Melbourne: (260 Burwood Rd)3d ago
-
Senior Data Engineer - MS Fabric AUD 150K-155KAccess Control | Automation | Azure | Backup/Restore | Data ModelingSenior-level Full TimeMelbourne, Victoria 3000, Australia3d ago
-
Black box testing | Black-box | Box testing | CAPL | Case DevelopmentAdditional annual leave purchase | Career opportunities local and international | Discounts on products | Employee assistance program | Flexible work arrangementsSenior-level Full TimeClayton, VIC, Australia3d ago
-
Senior MLOps Engineer AUD 75K-100KAWS | AWS Lambda | Amazon EC2 | Amazon EKS | Amazon SageMakerContinuous learning | Hackathons | Hybrid work | Paid gender-affirming care leave | Paid parental leaveSenior-level Full TimeMelbourne, VIC, Australia3d ago
-
Senior Robotics Engineer - Universal Field Robots AUD 116K-127KAgile | Artificial Intelligence | Automated testing | C++ | CI/CDEmployee benefits program | Length of service recognition | Salary sacrifice options | Sandvik wellness program | Training opportunitiesSenior-level Full TimeBrisbane UFR, Australia3d ago
-
Forward Deployed Engineer III, Conversational AI AUD 147K-180KAPIs | Agent systems | Cloud Platforms | Cloud platform | Data PipelinesSenior-level Full TimeMelbourne VIC, Australia; Sydney NSW, Australia6d ago
-
Staff Engineer, Datacenter Server Lifecycle AUD 149K-191KAWS | Asset tracking | Azure | Cloud platform | Firmware verificationFlexible working hours | Generous vacation | Optional equity donation matching | Parental leaveSenior-level Full TimeSydney, Australia6d ago