Databricks Solution Lead
Adelaide, SA, Australia
AUD 118K-127K (estimate) Senior-level Full Time
Tasks
- Build ML ready datasets
- Build batch data pipelines
- Build real-time data pipelines
- Collaborate with stakeholders to translate business needs
- Design lakehouse platforms on Databricks
- Develop ingestion frameworks for structured data
- Develop ingestion frameworks for unstructured data
- Enable data science workflows
- Implement CI/CD and DevOps best practices
- Implement Unity Catalog
- Implement data governance and quality checks
- Implement scalable data platforms
- Lead and mentor data engineers
- Maintain metadata and data lineage
- Manage security and privacy compliance
- Optimize performance, cost, and reliability
- Own end-to-end project delivery
- Set up monitoring and observability
- Support real time analytics with streaming
Perks/Benefits
- AI Driven Learning Platform
- Career development training
- Discounted health insurance
- Discounts on products and services
- Industry partner training
- Paid parental leave
- Recognition for high performance
- Wellbeing initiatives
Skills/Tech-stack
ADLS | AWS Glue | AWS S3 | Airflow | Amazon Redshift | Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOps | Azure Synapse | Azure Synapse Analytics | CI/CD | Data Factory | Data Governance | Data Lake Storage | Data Modeling | Data Quality | Data Warehousing | Databricks | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | GitHub | Infrastructure as Code | Lakehouse | Medallion Architecture | Metadata Management | Monitoring | Observability | PySpark | Python | SQL | Security Compliance | Snowflake | Structured Streaming | Synapse Analytics | Terraform | Unity Catalog | “as-code”
Education
N/A
Regions
Countries
States
Related jobs
-
Forward Deployed Engineer - Sydney AUD 116K-131KCode review | Full Stack | Full-Stack Development | Generative AI | JavaScriptHybrid work | Relocation assistance | Travel up to 50 percentMid-level Full TimeSydney, Australia R11h ago
-
Senior Data Engineer AUD 129K-142KAWS | Automation | Cybersecurity | Data Governance | Data ModellingFlexible work options | Hybrid work | Mentoring cultureSenior-level Full TimeMacquarie Park, NSW, AU, 2113 R1d ago
-
Lead Data Analytics Platforms | Perth, Brisbane AUD 135K-140KAWS | Agile | Amazon Bedrock | Amazon SageMaker | AnalyticsExtended leave entitlements | Flexible working options | Paid parental leave | Parent roomsSenior-level Full TimeAustralia1d ago
-
Head of AI Strategy and Enablement - MLC Expand AUD 110K-190KAI Governance | Artificial Intelligence | Automation | Cybersecurity | Data GovernanceExecutive-level Full TimeSYDNEY, NSW, AU, 20001d ago
-
Lead, Digital Data Analytics AUD 135K-140KAmplitude | Confluence | Consent Management | Data Architecture | Data EngineeringCareer development | Convenient city location | Free OnePass membership | Hybrid working | Modern end of trip facilitiesSenior-level Full TimeMelbourne, VIC, AU1d ago
-
Analytics Engineer AUD 104K-120KAgile | Apache Spark | Automation | Azure Data | Azure Data LakeEqual opportunity workplace | Flexible work options | Wellbeing supportMid-level Full TimeMelbourne Office (33X)1d ago
-
Customer Engineer, Database, Google Cloud AUD 150K-200KAlloyDB | Backup and Recovery | Cloud Bigtable | Cloud SQL | Cloud SpannerSenior-level Full TimeSydney NSW, Australia2d ago
-
Data Engineer - Senior AUD 140K-140KAPI Design | Agile | Azure Data | Azure Data Factory | Data FactorySenior-level Full TimeMelbourne, VIC, AU2d ago
-
Senior Machine Learning Engineer (Platform / MLOPs) AUD 132K-180KAPI Development | AWS | Amazon EKS | ArgoCD | Arize AIAnnual flu vaccinations | Gym access | Hybrid flexibility | In-office lunch | Nearmap subscriptionSenior-level Full TimeBarangaroo, NSW, Australia R2d ago
-
Sr. Analytics Engineer AUD 96K-116KAI Assisted Development | CI/CD | ClickHouse | DBT | Data GovernanceSenior-level Full TimeNorth Sydney, Australia2d ago
-
Manager, Modern Data Platform AUD 120K-135KAnalytics Cloud | Business Data Cloud | Data Catalog | Data Classification | Data GovernanceMid-level Full TimeSydney, NSW, Australia2d ago
-
AWS | Apache Spark | DBT | Data Ingestion | Data LakesFlexible working options | Paid volunteering days | Parental leave | Retail discounts | Return to work support packageMid-level Full TimeSydney, NSW, AU2d ago
-
AWS | Apache Spark | CI/CD | Cloud Security | Cloud platformFlexible working options | Market-leading parental leave | Paid volunteering days | Retail discounts | Return to work supportExecutive-level Full TimeSydney, NSW, AU2d ago
-
Chapter Lead, Data Science (Operations) AUD 140K-147KCI/CD | Containerization | Data Architecture | Data Pipelines | Deep learningCareer support | Flexible workplace | Inclusion-focused cultureSenior-level Full TimeEveleigh, NSW - 5-7 Central Ave, …2d ago
-
Robotics Software Engineer AUD 401KBenchmarking | C++ | CI/CD | Collision Avoidance | Machine Learning401k matching | Dental insurance | Health insurance | Paid Holidays | Travel supportMid-level Full TimeCharlestown, MA R2d ago
-
Data Platforms Engineer (Azure DataBricks) AUD 129K-138KApache Spark | Azure Data | Azure Data Factory | Azure DevOps | CI/CDAccess to professional memberships and industry networks | Discounted services | Employee assistance program | Flexible work arrangements | Parental leaveSenior-level Full TimeNorth Ryde, NSW, Australia3d ago
-
Senior Manager - Data Platform Governance AUD 129K-140KData Classification | Data Governance | Data Privacy | Data Risk | Data SecurityEmployee discounts | Flexible work arrangements | Inclusive workplace support | Learning and development | Leave optionsSenior-level Full TimeSydney, NSW, Australia3d ago
-
Senior LLMOps Engineer AUD 147K-180KAlerting | Dashboarding | Data Modeling | Data Pipelines | DatadogEquity | Fertility support | Health and wellness allowance | Home office budget | Learning and development budgetSenior-level Full TimeMelbourne R3d ago
-
Card Analytics Lead, Melbourne or Brisbane AUD 96K-113KAnalytics monitoring | Data analytics | Decisioning Platforms | FICO Falcon | Fraud DetectionDiscounts on selected products and services | Flexible working arrangements | Health and wellbeing servicesSenior-level Full TimeDocklands, AU3d ago
-
Data Engineer AUD 130K-165KAgile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageDiversity and inclusion | Hybrid working | Professional growthSenior-level Full TimeMelbourne, Australia3d ago
-
Staff Data Engineer (AWS Cloud and AI) AUD 135K-140KAPI Gateway | AWS | AWS API | AWS API Gateway | AWS DataZoneFlexible workingSenior-level Full TimeSydney, NSW - CBP South, 11 …3d ago
-
Senior-level Full TimeSydney, NSW - CBP South, 11 …3d ago
-
Senior Engineer, Database AUD 130K-150KAgile | Azure | Azure DevOps | Azure SQL | CI/CDAdditional rest day | Discounted health insurance | Flexible working | Hybrid working | Volunteer daySenior-level Full TimeSydney, Australia3d ago
-
Senior Engineer, Database AUD 130K-150KAgile | Azure DevOps | Azure SQL | CI/CD | DevOpsFlexible working | Health insurance discount | Hybrid working | Rest day | Volunteer daySenior-level Full TimeSydney, Australia3d ago
-
Service Owner - Data Platform Engineering AUD 129K-146KAgile | Azure | Azure Data | Azure Data Lake | Azure Data Lake StorageEmployee discounts | Flexible work arrangements | Leave options | Mortgage discounts | Tailored learning and developmentSenior-level Full TimeSydney, NSW, Australia3d ago