Lead Data Engineer
Canberra, Australian Capital Territory 2600, Australia
AUD 118K-127K (estimate) Senior-level Full Time
Tasks
- Automate infrastructure provisioning and operations
- Build observable data ingestion pipelines
- Communicate data risk and issues
- Configure scalable resilient observable analytics platforms
- Design and support data pipelines end to end
- Develop data storage tables and message schemas
- Improve data ingestion standards
- Install configure and operate search and analytics clusters
- Maintain batch and streaming data pipelines
- Manage data lake access controls
- Monitor data platform health and performance
- Optimize data ingestion pipelines
- Remediate platform issues
Perks/Benefits
Skills/Tech-stack
Access Control | Ansible | Batch Processing | CI/CD | Chef | Data Enrichment | Data Lake | Data Transformation | Hadoop | Java | Kafka | Linux | Message serialization | Nifi | Puppet | Python | Role-Based Access | Role-Based Access Control | Scala | Schema Design | Spark | Streaming Data
Education
N/A
Regions
Countries
Related jobs
-
Senior-level Full TimeMelbourne, Australia13h ago
-
Entry-level Full TimeMelbourne, Australia13h ago
-
AI Engineer - DDAI AUD 126K-150KA/B | A/B Testing | Apache Spark | Autogen | B testingAdditional leave | Community volunteering opportunities | Employee benefits | Flexible work | Learning and developmentMid-level Full TimeSydney, NSW, Australia14h ago
-
Data Engineer AUD 80K-150KDBT | Data Ingestion | Data Transformation | ELT | ETLAdditional leave | Corporate partner discounts | Free NBN plan | Free mobile plan | Hybrid workMid-level Full TimeBarangaroo, NSW, AU3d ago
-
Senior AI Engineer AUD 142K-163KAWS | Autogen | Azure | CI/CD | Clean CodeAutonomy in role | Learning and development programs | Mentorship cultureSenior-level Full TimeMelbourne, Australia R3d ago
-
Data Engineer - Coles 360 - Melbourne AUD 110K-140KAzure | BigQuery | CI/CD | DBT | Data ModelingDiscounts on supermarket and liquor | Fitness classes | Free parking | Gym facility | Hybrid working optionsMid-level Full TimeHawthorn East Melbourne, VIC, AU, 31233d ago
-
LLMs | Langchain | MLOps | NumPy | PandasProject based workMid-level FreelanceAustralia - Remote R3d ago
-
Freelance Machine Learning Engineer AUD 110KLLMs | Langchain | MLOps | Machine Learning | NumPyPart-time availability | Project based workMid-level FreelanceAustralia - Remote R3d ago
-
Principal Data Engineer (AWS) AUD 140K-154KAWS | Airflow | Apache Spark | Automated testing | Batch ProcessingCareer support | Flexible work arrangements | Flexible work locations | Job share options | Part-time arrangementsSenior-level Full TimeSydney, NSW - CBP North, 1 …3d ago
-
Data Engineer AUD 100K-138KAzure Synapse | Azure Synapse Analytics | Cloud Data | Cloud Data Platforms | DBTCareer development | Discounted private health insurance | Employee assistance program | Employee discounts | Free health checksMid-level Full TimeAustralia | SEQ | Brisbane - …3d ago
-
Senior-level Full TimePerth, Australia3d ago
-
Embedded Systems Engineer AUD 116K-145KAutomation | Bluetooth Low Energy | Build Automation | CI/CD | Embedded SoftwareEmployee assistance program | Employee discounts | Fitness passport | Flexible work options | Learning and development opportunitiesSenior-level Full TimeKeysborough, VIC, AU4d ago
-
Customer Data Engineer AUD 120K-130KDBT | Data Pipelines | Data Quality | Data Transformation | Data ValidationEquity package | Growth pathways | Remote-firstMid-level Full TimeKensington, Australia; North Wollongong, Australia; Collingwood, …4d ago
-
Associate AI Engineer AUD 100K-120K.NET | Azure | Azure Container | Azure Container Apps | Container AppsAnnual leave | Collaborative team culture | Flexible working environment | Parental leave | Professional developmentMid-level Full TimeMelbourne, VIC, Australia4d ago
-
Senior Data Engineer AUD 129K-142KBitbucket | DB2 | Data Quality | Data quality assurance | MySQLAnnual flu vaccinations | Discounted health and fitness memberships | Employee assistance program | Flexible working arrangements | Generous leave entitlementsSenior-level Full TimeSydney, NSW, AU, 20004d ago
-
AI Engineer | ML, GenAI, LLM, Python, Azure | Sydney AUD 100K-124KAI Safety | AWS | Anomaly Detection | Azure | ClusteringFlexible work arrangements | Flexible working options | Market-leading parental leave | Paid volunteering days | Retail discountsMid-level Full TimeSydney, NSW, AU4d ago
-
Seagoing Software Engineer / Data Analyst AUD 103K-113KData Management | Data Quality | Data Validation | Data Visualization | LinuxAccessible recruitment process | Career development opportunities | Flexible work arrangements | Leave entitlements | Training and mentoringMid-level Full TimeHobart, TAS, AU4d ago
-
Data Engineer - Asset Management Consultant AUD 70K-80KAgile | Amazon Web Services | Apache Spark | Azure | Cloud platformEntry-level Full TimeBrisbane, Australia4d ago
-
Senior Software Engineer - MLOps AUD 130K-150KAWS | Amazon SageMaker | Aurora DB | Batch inference | CI/CDFlexible work arrangements | Job shareSenior-level Full TimeEveleigh, NSW - 1 Locomotive Street, …4d ago
-
Data Engineer - Asset Management Consultant AUD 70K-80KAmazon Web Services | Apache Spark | Azure | Containerization | Data CleansingEntry-level Full TimeBrisbane, Australia4d ago
-
Deep learning | GenAI | Langchain | MLOps | Machine LearningEnglish CV requirement | Flexible part-time hours | Project based workMid-level FreelanceAustralia - Remote R4d ago
-
Advanced Analytics Specialist AUD 100K-120KC++ | CPLEX | Cloud Computing | Data Visualization | DatabricksCheaper retail and technology services | Discounted flights | Discounts on accommodation | Discounts on car hire | Discounts on experiencesMid-level Full TimeBrisbane5d ago
-
API | Automation | Data Architecture | Data Modeling | IndexingCollaborative engineering culture | End-to-end ownership | Exposure to modern APIs automation and AI | Flexible work location | Fully remoteSenior-level Full TimeAustralia R5d ago
-
Service Owner – Data Platform Engineering AUD 110K-156KADLS | Agile | Azure | Azure Purview | Azure SynapseAdditional leave options | Flexible work arrangements | Learning and development opportunities | Volunteer opportunitiesSenior-level Full TimeNSW, Australia6d ago
-
Bash | Data Ingestion | Data Processing | Docker | GCPAsynchronous culture | Collaborative team | Opportunities for impact | Remote-friendlyMid-level Full TimeMelbourne, Australia6d ago