Big Data Lead
Tasks
- Analyze data flow
- Build ETL pipelines
- Collaborate with DevOps
- Document code lineage
- Ensure data quality and integrity
- Ensure data security compliance
- Implement CI/CD pipelines
- Transform data
- Troubleshoot PySpark workflows
- Troubleshoot Snowpark applications
Perks/Benefits
- N/A
Skills/Tech-stack
CI/CD | Data Warehousing | DevOps | ETL | Hadoop | Hive | Kafka | PySpark | Python | SAS | SQL | Snowflake | Snowpark | Spark
Education
N/A
Roles
Big Data Engineer | Data Engineer | Data Engineering | Data Engineering Lead | Engineer | Engineering Lead | Lead
Related jobs
-
Senior AI/ML Solution Architect, ANZ Public Sector NZD 124K-169KAPI Development | AWS SageMaker | AWS Transform | Agentic AI | Amazon BedrockCareer growth | Flexible work hours | Mentorship | Work-life balanceSenior-level Full TimeWellington, Wellington, NZL1d ago
-
Senior Data Engineer NZD 136K-152KAWS | Azure | Data Modelling | DataOps | DatabricksAdditional paid leave purchase option | Employee energy discounts | Employee share scheme access | Enhanced parental leave | Flexible work optionsSenior-level Full TimeFlexible, New Zealand7d ago
-
Data Engineer NZD 120K-140KAI Data Cloud | Coalesce | Cortex | DBT | Data cloudCareer development | Family-friendly policies | Flex leave | Flexible work arrangements | Hybrid workSenior-level Full TimeWellington, NZ, 60117d ago
-
Senior AI Engineer NZD 120K-145KAPIs | AWS | Agents | Asynchronous programming | AzureFlexible Work N/A | Growth opportunities | Mentoring opportunitiesSenior-level Full TimeAuckland, Auckland, New Zealand7d ago
-
Principal Consultant - Data Engineering NZD 138K-170KAzure | Azure Data | Azure Data Catalog | Azure Data Factory | Azure Data LakeCareer pathways | Diversity | Flexible schedule | Inclusion | Professional developmentSenior-level Full TimeNew Zealand7d ago
-
Principal Consultant - Data Engineering NZD 138K-170KAzure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCareer pathways | Flexibility | Inclusion | Ongoing learning | Remote or hybrid workSenior-level Full TimeNew Zealand7d ago
-
Data Engineer - SMARTLY NZD 120K-145KCI/CD | Cloud infrastructure | Data Modeling | Data Pipelines | Data QualityCompany events | Flexible working | Meals provided | Work from homeSenior-level Full TimeAuckland, Auckland, New Zealand8d ago
-
Senior Data Engineer (Database Operations) NZD 119K-140KAWS | Azure SQL | Azure SQL Managed Instance | Bash | CI/CDCounselling | Employee life insurance | Free flu shots | Health insurance discount | Learning and development budgetSenior-level Full TimeAuckland, Auckland, New Zealand8d ago
-
Digital & Data Transformation - Senior Analyst NZD 110K-143KData Engineering | Data Modeling | Data analytics | Language Processing | Machine LearningSenior-level Full TimeAuckland - PwC Tower, New Zealand11d ago
-
Staff Machine Learning Engineer NZD 135K-165KAWS EMR | Airflow | Apache Spark | Dask | Distributed ProcessingSenior-level Full TimeAU: Sydney (45 Clarence St)14d ago
-
Technical Lead - Data, AI and Analytics NZD 120K-145KAgile | Azure Data | Azure Data Factory | Azure Databricks | Azure SynapseChill-out spaces | Flexi hours | Professional development courses | Remote Working | Social eventsSenior-level Full TimeAuckland, Auckland, New Zealand15d ago
-
Data Engineer NZD 97K-115KAnalytics | Automation | Change Data Capture | Data Capture | Data ModelingFree parking | Professional development | Subsidized cafeteria | Supportive work environment | Work-life balanceMid-level Full TimeAuckland, New Zealand20d ago
-
Senior/Tech Lead - AI Development NZD 136K-155KAI Foundry | AI Search | AWS Bedrock | Amazon SageMaker | Artificial IntelligenceChill-out spaces | Flexi hours | Professional development courses | Remote Working | Social eventsSenior-level Full TimeAuckland, Auckland, New Zealand25d ago
-
AI & Data Platform Operations Lead NZD 129K-152KAI Ops | AWS | Agile | Auditability | Change ManagementEmployee discounts | Flex hours | Lifestyle leave | Southern Cross Health insurance subsidized | Work from home flexibilitySenior-level Full TimeChristchurch, NZ25d ago
-
AI Engineer NZD 92K-108KAmazon Web Services | Apache Airflow | Artificial Intelligence | Azure | Azure DataLearning opportunities | Meaningful projects | Supportive team environmentEntry-level Full TimeAuckland, Auckland, New Zealand1mo ago
-
ML & Agentic Solutions Engineer NZD 145K-170KAgent systems | Agentic AI | Agents SDK | Conversational AI | Data PrivacySenior-level Full TimeWellington, New Zealand1mo ago
-
ML & Agentic Solutions Engineer NZD 145K-170KAgent Orchestration | Agent systems | Agents SDK | Context Routing | FastMCPSenior-level Full TimeAuckland, New Zealand1mo ago
-
Data & AI Engineer NZD 105K-130KAKS | Azure | Azure DevOps | Azure Functions | Azure QueuesClient-facing consulting experience | Collaborative team environment | Remote-friendly collaborationMid-level Full TimeWellington, New Zealand1mo ago
-
Mid-level Full TimeAuckland, New Zealand1mo ago
-
Intermediate Data Engineer NZD 105K-120KAPI Integration | Abinitio | Airflow | Apache Spark | CI/CDDiscounts on selected products and services | Flexible working arrangements | Wellbeing services accessMid-level Full TimeAuckland or Wellington, NZ1mo ago
-
Senior Data Engineer, 12-Month Fixed Term Contract NZD 127K-138KAPI | Ab Initio | Airflow | Automation | CI/CDAccess to additional support for recruitment adjustments | Discounts on selected products and services | Flexible working arrangements | Wellbeing services accessSenior-level Contract TemporaryAuckland or Wellington, NZ1mo ago
-
Data Engineer - Senior Analyst / Manager NZD 112K-132KAccess Controls | Apache Spark | Azure Data | Azure Data Factory | Azure Data LakeAnnual summer shutdown | Coaching | Employee assistance program | Flexible working | Health insuranceSenior-level Full TimeWellington - PwC Centre, Level 4, …1mo ago