Data Engineer – AWS & Snowflake
Tasks
- Analyze Hadoop datasets and pipelines
- Apply security controls IAM RBAC encryption masking privacy audit logging
- Build CI CD and DataOps deployment processes
- Build migrate batch incremental and near real time pipelines to AWS and Snowflake
- Create technical designs data mappings pipeline documentation and runbooks
- Develop Snowflake databases schemas tables views stages Snowpipe Streams and Tasks
- Develop ingestion transformation and integration solutions
- Implement ETL ELT pipelines and enterprise data models
- Implement data quality lineage metadata and governance
- Implement monitoring alerting logging and operational support
- Optimize Snowflake queries warehouse sizing and workload performance
- Perform data profiling cleansing validation reconciliation and migration testing
- Re engineer Hadoop workloads into cloud native solutions
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Apache Spark | Audit Logging | Automated testing | CI/CD | Data Governance | Data Ingestion | Data Lineage | Data Quality | Data Warehousing | Data integration | Data masking | Data profiling | DataOps | Dimensional Modeling | ELT | ETL | Encryption | Git | HDFS | Hadoop | Hive | IAM | Lakehouse | Logging | Metadata Management | Monitoring | Privacy | PySpark | Python | RBAC | Relational modeling | SQL | Snowflake
Education
N/A
Roles
Related jobs
-
Senior AI Engineer, AI Accelerator AUD 143K-180KAgent Design | Amazon Web Services | Artificial Intelligence | Azure OpenAI | CI/CDCareer development opportunities | Convenient location | Free OnePass membership | Hybrid working | Modern end of trip facilitiesSenior-level Full TimeMelbourne, VIC, AU4h ago
-
Data Engineer – AI AUD 110K-140KA/B | A/B Testing | Agile | Automated testing | B testingEmployee discounts | Onsite Parking ProvidedMid-level Full TimeSydney Olympic Park, New South Wales …13h ago
-
Staff Data Engineer AUD 140K-146KAPI Gateway | AWS DataZone | AWS Glue | AWS Lake | AWS Lake FormationChanging start and finish times | Flexible working arrangements | Hybrid work with office time | Job share | Part-time arrangementsSenior-level Full TimeSydney, NSW - CBP South, 11 …20h ago
-
AI Engineer - South Bank, QLD AUD 70K-138KAzure DevOps | Azure Machine Learning | Bicep | CI/CD | DatabricksEmployee assistance program | Employee discounts | Health and wellness discounts | Internal gym access | Learning and development budgetMid-level Full TimeAustralia, Queensland21h ago
-
Senior Data Engineer AUD 132K-140KAPI | Azure | Azure DevOps | CI/CD | Data LakehouseHybrid work flexibility | Learning and development opportunities | Max term contractSenior-level Full TimeMelbourne, VIC, AU1d ago
-
Automated Workflows | CI/CD | Cloud Native | Containerization | Data LineageCollaborative environment | Inclusive workplace | Professional growthSenior-level Full TimeAustralia1d ago
-
Senior Software Engineer (Data & Distributed Systems) AUD 155K-186KAgile | C# | Cloud | Databricks | Distributed SystemsSenior-level Full TimeSydney1d ago
-
Azure | Azure Data | Azure Data Factory | Azure DevOps | BashHands on Azure experience | Hands on Databricks experience | Professional growthMid-level Full TimeAustralia1d ago
-
Senior AI Engineer AUD 160K-200KAzure | C# | Databricks | Datadog | FastMCPAnnual leave | Flexible working environment | Generous leave | Karbon Days off | Parental leaveSenior-level Full TimeMelbourne, VIC, Australia1d ago
-
Computer Vision Engineer - VisionAI (AU) AUD 143K-161KC++ | CUDA | Control Theory | Edge Computing | Embedded SystemsSenior-level Full TimeSydney, Australia1d ago
-
Lead Data Engineer - Azure AUD 125K-140KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksEducation assistance program | Extra annual leave | Extra insurance | Flexible hybrid work model | Parental leaveSenior-level Full TimeSydney - Darling Park, Australia R1d ago
-
Agentic AI Engineer (AWS) AUD 100K-135KAWS | AWS CDK | Agent systems | Alembic | Amazon BedrockFlexible working options | Market-leading parental leave | Mentoring coaching programs | Paid volunteering days | Professional development reimbursementsMid-level Full TimeSydney, NSW, AU R1d ago
-
Senior-level Full TimeMelbourne, Victoria, Australia1d ago
-
Head of AI & Automation AUD 150K-170KAccess Management | Artificial Intelligence | Automation | Claude Enterprise | ConnectorsExecutive-level Full TimeSydney, New South Wales, Australia1d ago
-
Data Engineer AUD 65K-70KAWS Redshift | Apache Spark | Automation | CI/CD | DBTDiscounts to retailers | Fitness passport | Flexible working | Paid parental leave | Professional developmentMid-level Full TimeHead Office- Melbourne, Australia R1d ago
-
Access Management | Azure | Azure Data | Azure Data Factory | Azure DatabricksClear communication | Fair disclosed margins | Hybrid work | Professional respectful experienceSenior-level Full TimeCanberra, Australian Capital Territory, Australia R2d ago
-
Senior AI Engineer AUD 160K-180KAWS | Agent Orchestration | Agentic AI | Artificial Intelligence | AutogenCareer development | Collaborative team environment | Stable work environmentSenior-level Full TimeMelbourne, Victoria 3000, Australia2d ago
-
Senior AI Engineer AUD 160K-180KAI-generated code | AWS | Agent Orchestration | Agentic AI | Artificial IntelligenceCareer developmentSenior-level Full TimeSydney, New South Wales 2000, Australia2d ago
-
Senior AI Engineer AUD 102K-130KAWS | Amazon S3 | Bedrock | CI/CD | DockerConferences | Corporate discounts | Flexible hybrid workplace | Home office support | In-house trainingSenior-level Full TimeMelbourne, Australia R2d ago
-
Mid-level Full TimeSydney, NSW, Australia2d ago
-
Staff AI & Data Engineer AUD 160K-215KAgent systems | Agentic AI | Azure | Azure Event | Azure Event HubsSenior-level Full TimeAirTrunk HQ - North Sydney2d ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous work culture | Friendly laid-back atmosphere | Hands-off management | Opportunity for high impact productsMid-level Full TimeSydney, Australia2d ago
-
Apache Spark | Azure | CI/CD | Containerization | Data ModelingFlexible working options | Paid parental leave | Paid volunteering days | Wellbeing leaveMid-level Full TimeMelbourne, VIC, AU2d ago
-
Databricks Data Engineer Senior Analyst AUD 70K-120KAgile | Apache Spark | Azure Blob | Azure Blob Storage | Azure DataSenior-level Full TimeBrisbane, 10 Eagle Street, Australia2d ago
-
Principal Data Engineer (Apache Spark, dbt, Airflow) AUD 140K-146KAWS | Amazon Redshift | Apache Airflow | Apache Iceberg | Apache SparkFlexible working | Hybrid working optionsSenior-level Full TimeSydney Office, Australia2d ago