Data Engineer
Tasks
- Conduct post mortems after incidents
- Configure and monitor data orchestration assets
- Debug pod failures and resource bottlenecks
- Design and implement data ELT ETL pipelines
- Develop and optimize Databricks Delta Lake tables
- Implement data tests for data quality
- Integrate AI models for data processing and automation
- Maintain codebase with Git
- Monitor Kubernetes workloads on AWS EKS
- Participate in on-call rotation
- Troubleshoot pipeline failures
- Write Python scripts for data extraction and utilities
- Write efficient Spark and SQL queries
Perks/Benefits
Skills/Tech-stack
AWS | AWS EKS | Airbyte | Airflow | CI/CD | DBT | Dagster | Databricks | Delta Lake | Git | Great Expectations | Infrastructure as Code | Kubernetes | Prefect | Python | Reverse ETL | SQL | Spark | “as-code”
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
Staff Backend Engineer - Experiments AUD 129K-155KAWS | Back pressure | Circuit Breakers | Concurrency | Data ModelingEquity package | Flexible leave | Flexible work options | Parental leave | Wellbeing allowanceSenior-level Full TimeSydney, Australia8h ago
-
Senior Data Engineer AUD 149K-155KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure FabricAdditional leave purchase option | Discounted health insurance | Fitness passport discount | Hybrid work options | Salary sacrificing optionsSenior-level Full TimeBrisbane, QLD, AU12h ago
-
Machine Learning Engineer AUD 70K-138KAPI | Amazon Web Services | CI/CD | Containerization | GroundingAnnual leave | Family care parental leave | Global mobility | Hybrid work | Personal and carer leaveEntry-level Full TimeNorth Sydney, AU12h ago
-
Senior Software Engineer, Robotics Task Planning AUD 160K-190KAlgorithm Optimization | Behavior planning | C++ | Data Analysis | Data Pipelines401k | Dental insurance | Health insurance | Unlimited vacation | Vision insuranceSenior-level Full TimeCharlestown, MA1d ago
-
SM Data Engineering AUD 201K-234KAnalytics | Business Intelligence | Data Consistency | Data Engineering | Data ModelingFlexible work options | Hybrid workExecutive-level Full TimeSydney, NSW, AU, 20001d ago
-
AWS Glue | Amazon EMR | Amazon S3 | Apache Airflow | CI/CDAutomation focused culture | Collaborative engineering squad | Engineering excellence focus | Modern cloud-native environmentMid-level Full TimeSydney, NSW, AU2d ago
-
Senior-level Full TimeSydney, Melbourne2d ago
-
3D Modeling | 3D Reconstruction | AWS | Automation | CI/CDAnnual flu vaccinations | Cycling showers access | Hybrid flexibility | In-office lunch | Lunchtime gym accessSenior-level Full TimeBarangaroo, NSW, Australia R3d ago
-
Software Engineer, Quantum Sensing AUD 70K-80KC# | C++ | Continuous integration | Data Fusion | Data ProcessingCompetitive benefits | Equity potential | FlexibilityMid-level Full TimeSydney3d ago
-
Data Pipelines | Data Preprocessing | Data Validation | Feature Engineering | Machine LearningEquity packages | Flexible leave options | Parental leave | Remote work within Australia | Wellbeing allowanceSenior-level Full TimeSydney, Australia R3d ago
-
Senior Software Engineer - Computer Vision AUD 102K-130K3D Depth Analysis | Anti-spoofing | Biometric Authentication | Data Analysis | Data PreprocessingAccess to learning and development resources | Additional learning resources | Benefits for you and your family | Discounted health plan rate | Employee assistance programSenior-level Full TimeAustralia - (North Sydney- Arthur St)3d ago
-
Senior Software Engineer - Computer Vision AUD 102K-130KAnti Spoofing Standards | Anti-spoofing | Biometric Authentication | Computer Vision | Data AnalysisAccess to learning and development resources | Benefits for family | Discounted health plan rate | Employee assistance program | Flexible working arrangementsSenior-level Full TimeAustralia - (North Sydney- Arthur St)3d ago
-
MATLAB | NumPy | Pandas | Python | RPart-time project-based workSenior-level Full TimeAustralia - Remote R3d ago
-
Data Analysis | Data Visualization | Data cleaning | Exploratory Data Analysis | Feature EngineeringFreelance contract | Part-time flexible hours | Project based workMid-level FreelanceAustralia - Remote R3d ago
-
Freelance Machine Learning Engineer AUD 110KLLMs | Langchain | MLOps | NumPy | PandasFlexible hours | Freelance opportunities | Part-time project work | Project-based employmentMid-level FreelanceAustralia - Remote R3d ago
-
Deep learning | LLM | Langchain | MLOps | Machine LearningFlexible weekly hours estimate | Part-time project work | Project-based engagementMid-level FreelanceAustralia - Remote R3d ago
-
Data Engineer-Medicare Mental Health Check In AUD 112K-129KAnalysis Services | Azure Data | Azure Data Factory | CI/CD | Data FactoryHybrid work | Salary packaging benefitsSenior-level Full TimeVIC Group Office - Melbourne, Australia3d ago
-
Senior Modeling & Optimization Engineer USD 175K-190KApplied Data Science | Data Pipelines | Data Science | Data Visualization | Design Space Exploration401k plan | Accident insurance | Commuter benefits | Company-Paid Holidays | Critical illness insuranceSenior-level Full TimeBrisbane, CA3d ago
-
AI Engineer AUD 115K-135KAI Agents | AWS | Autonomous Agents | Docker | DocumentationCompany townhalls | Diversity and inclusion initiatives | Flexible working | Monthly social events | Office snacksMid-level Full TimeSydney4d ago
-
Analytics Engineer (MarTech) AUD 142K-185KA/B | A/B Testing | Airflow | Amplitude | Attribution ModelingDental insurance | Disability insurance | Employee stock options program | Health insurance | Life insuranceMid-level Full TimeMelbourne, VIC, Sydney, NSW4d ago
-
Data Engineer AUD 110K-114KApache NiFi | Automation | Data Governance | Data Quality | Data ValidationAdditional paid leave | Family-friendly workplace | Learning SupportSenior-level Full Time2822 Russell ACT Australia - Customer …4d ago
-
Senior Data Engineer AUD 130K-150KApache NiFi | Cause analysis | Data Engineering | Data Governance | Data ModelingExtra paid leave | Family-friendly workplaceSenior-level Full Time2822 Russell ACT Australia - Customer …4d ago
-
Data Engineer AUD 126K-126KAPIs | CI/CD | Data Lakes | Data Pipelines | Data PrivacyCareer development | Flexible working | Health and wellbeing programs | Hybrid working | Learning opportunitiesMid-level Full TimeServices Building (G12), Australia4d ago
-
Senior Software Engineer - MLOps AUD 130K-150KAWS KMS | AWS Secrets | AWS Secrets Manager | Amazon Aurora | Amazon CloudWatchFlexible start and finish times | Flexible work arrangements | Hybrid work with office attendance half time monthly | Job-sharing | Part-time optionsSenior-level Full TimeEveleigh, NSW - 1 Locomotive Street, …4d ago
-
Agentic Workflows | Arize Phoenix | CrewAI | DBT | Embedding-based searchEmployee stock options | Flexible work schedule | Free app access | Hybrid work | Novated leasingMid-level Full TimeSydney5d ago