Data Engineer – AWS Glue | Python | PySpark
Tasks
- Apply shift left quality and DevSecOps
- Build and maintain AWS Glue and EMR pipelines
- Collaborate with product platform and QA teams
- Deploy and support data services in production
- Design data processing pipelines on AWS
- Implement CI/CD automation
- Monitor production health with logging metrics and tracing
- Orchestrate workflows with Airflow
- Perform unit integration and contract testing
- Tune performance and optimize cloud costs
Perks/Benefits
- Automation focused culture
- Collaborative engineering squad
- Engineering excellence focus
- Modern cloud-native environment
Skills/Tech-stack
AWS Glue | Amazon EMR | Amazon S3 | Apache Airflow | CI/CD | CloudWatch | DevSecOps | GitHub Actions | Jenkins | Linux | Logging | Metrics | Oracle | PySpark | Python | SQL | Splunk | Teradata | Tracing
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
Senior-level Full TimeSydney, Melbourne1d ago
-
3D Modeling | 3D Reconstruction | AWS | Automation | CI/CDAnnual flu vaccinations | Cycling showers access | Hybrid flexibility | In-office lunch | Lunchtime gym accessSenior-level Full TimeBarangaroo, NSW, Australia R1d ago
-
Software Engineer, Quantum Sensing AUD 70K-80KC# | C++ | Continuous integration | Data Fusion | Data ProcessingCompetitive benefits | Equity potential | FlexibilityMid-level Full TimeSydney1d ago
-
Data Pipelines | Data Preprocessing | Data Validation | Feature Engineering | Machine LearningEquity packages | Flexible leave options | Parental leave | Remote work within Australia | Wellbeing allowanceSenior-level Full TimeSydney, Australia R1d ago
-
Senior Software Engineer - Computer Vision AUD 102K-130K3D Depth Analysis | Anti-spoofing | Biometric Authentication | Data Analysis | Data PreprocessingAccess to learning and development resources | Additional learning resources | Benefits for you and your family | Discounted health plan rate | Employee assistance programSenior-level Full TimeAustralia - (North Sydney- Arthur St)1d ago
-
Senior Software Engineer - Computer Vision AUD 102K-130KAnti Spoofing Standards | Anti-spoofing | Biometric Authentication | Computer Vision | Data AnalysisAccess to learning and development resources | Benefits for family | Discounted health plan rate | Employee assistance program | Flexible working arrangementsSenior-level Full TimeAustralia - (North Sydney- Arthur St)1d ago
-
MATLAB | NumPy | Pandas | Python | RPart-time project-based workSenior-level Full TimeAustralia - Remote R1d ago
-
Data Analysis | Data Visualization | Data cleaning | Exploratory Data Analysis | Feature EngineeringFreelance contract | Part-time flexible hours | Project based workMid-level FreelanceAustralia - Remote R1d ago
-
Freelance Machine Learning Engineer AUD 110KLLMs | Langchain | MLOps | NumPy | PandasFlexible hours | Freelance opportunities | Part-time project work | Project-based employmentMid-level FreelanceAustralia - Remote R1d ago
-
Deep learning | LLM | Langchain | MLOps | Machine LearningFlexible weekly hours estimate | Part-time project work | Project-based engagementMid-level FreelanceAustralia - Remote R1d ago
-
Data Engineer-Medicare Mental Health Check In AUD 112K-129KAnalysis Services | Azure Data | Azure Data Factory | CI/CD | Data FactoryHybrid work | Salary packaging benefitsSenior-level Full TimeVIC Group Office - Melbourne, Australia1d ago
-
Senior Modeling & Optimization Engineer USD 175K-190KApplied Data Science | Data Pipelines | Data Science | Data Visualization | Design Space Exploration401k plan | Accident insurance | Commuter benefits | Company-Paid Holidays | Critical illness insuranceSenior-level Full TimeBrisbane, CA1d ago
-
AI Engineer AUD 115K-135KAI Agents | AWS | Autonomous Agents | Docker | DocumentationCompany townhalls | Diversity and inclusion initiatives | Flexible working | Monthly social events | Office snacksMid-level Full TimeSydney2d ago
-
Analytics Engineer (MarTech) AUD 142K-185KA/B | A/B Testing | Airflow | Amplitude | Attribution ModelingDental insurance | Disability insurance | Employee stock options program | Health insurance | Life insuranceMid-level Full TimeMelbourne, VIC, Sydney, NSW2d ago
-
Data Engineer AUD 110K-114KApache NiFi | Automation | Data Governance | Data Quality | Data ValidationAdditional paid leave | Family-friendly workplace | Learning SupportSenior-level Full Time2822 Russell ACT Australia - Customer …2d ago
-
Senior Data Engineer AUD 130K-150KApache NiFi | Cause analysis | Data Engineering | Data Governance | Data ModelingExtra paid leave | Family-friendly workplaceSenior-level Full Time2822 Russell ACT Australia - Customer …2d ago
-
Data Engineer AUD 126K-126KAPIs | CI/CD | Data Lakes | Data Pipelines | Data PrivacyCareer development | Flexible working | Health and wellbeing programs | Hybrid working | Learning opportunitiesMid-level Full TimeServices Building (G12), Australia2d ago
-
Senior Software Engineer - MLOps AUD 130K-150KAWS KMS | AWS Secrets | AWS Secrets Manager | Amazon Aurora | Amazon CloudWatchFlexible start and finish times | Flexible work arrangements | Hybrid work with office attendance half time monthly | Job-sharing | Part-time optionsSenior-level Full TimeEveleigh, NSW - 1 Locomotive Street, …2d ago
-
Software Engineer II - Azure Data Lake Storage Gen 2 AUD 108K-150KAzure | Azure Data | Azure Data Lake | Azure Data Lake Storage | C#On-call rotationMid-level Full TimeSydney, NSW, AU; VIC,AU; Brisbane, QLD, …3d ago
-
Agentic Workflows | Arize Phoenix | CrewAI | DBT | Embedding-based searchEmployee stock options | Flexible work schedule | Free app access | Hybrid work | Novated leasingMid-level Full TimeSydney3d ago
-
Senior Firmware / Embedded Systems Engineer AUD 110K-122KARM Cortex | ARM Cortex M7 | Bare Metal | C# | C++Annual flu vaccinations | Cycling Support | Gym access | In-office lunch | Nearmap subscriptionSenior-level Full TimeBarangaroo, NSW, Australia3d ago
-
Senior Machine Learning Engineer AUD 85K-95KAmazon EKS | ArgoCD | Arize AI | Batch inference | CI/CDAnnual flu vaccinations | Cycling showers | Hybrid flexibility | In-office lunch | Quarterly wellbeing day offSenior-level Full TimeBarangaroo, NSW, Australia3d ago
-
Data Governance | Data Quality | Data Warehousing | Python | ReportingDiscounted health insurance | Rest day | Volunteer daySenior-level Contract Full Time TemporarySydney, Australia3d ago
-
Senior Data Engineer AUD 72K-80KAWS | AWS DMS | AWS Glue | AWS Lake Formation | AgileFlexible working | Hybrid working options | Part-time work optionsSenior-level Full TimeSydney Office, Australia3d ago
-
Senior Machine Learning Engineer AUD 85K-95KAWS | Amazon Elastic Kubernetes Service | ArgoCD | Arize AI | Batch inferenceAnnual flu vaccinations | Hybrid flexibility | In-office lunch | Nearmap subscription | On-site showersSenior-level Full TimeBarangaroo, NSW, Australia3d ago