Senior Data Engineer
Tasks
- Build ETL/ELT workflows
- Deploy containerized applications
- Develop data ingestion scripts
- Develop data lake solutions
- Ensure data security compliance
- Implement data quality checks
- Implement real-time streaming pipelines
- Integrate data pipelines with DataHub and Ranger
- Migrate data pipelines
- Orchestrate workflows with Apache Airflow
- Orchestrate workloads on Kubernetes
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Airflow | Apache Iceberg | Apache Kafka | Azure Data | Azure Data Lake | Azure Data Lake Storage | Big Data | Data Lake Storage | Data Modeling | Data Warehousing | Datahub | Docker | Java | Kubernetes | Linux | Minio | Python | Ranger | Shell | Trino
Education
N/A
Roles
Regions
Countries
States
Related jobs
-
Hugging Face | Hugging Face Transformers | LLM | Langchain | LoRAEntry-level Full TimeBatu Kawan, Penang, Malaysia8h ago
-
API | Azure Data | Azure Data Factory | BOM | DAXMid-level Full TimeIpoh, Perak, Malaysia11h ago
-
Senior Full Stack Engineering Lead-Brokerage Industry MYR 168K-183KC++ | CFD | CRM | Copy Trading | CySECEmployee-centric policies | Entertainment | Fast Paced Career Progression | Food and beverage | Learning opportunitiesSenior-level Full TimeKuala Lumpur, Wilayah Persekutuan Kuala Lumpur, …13h ago
-
Data Quality Developer USD 99K-135KAzure | Data Architecture | Data Cleansing | Data Enrichment | Data GovernanceSenior-level Full TimeKuala Lumpur, MY17h ago
-
AWS | AWS Glue | AWS Lambda | Amazon Athena | Amazon EC2Startup-like cultureSenior-level Full TimeKuala Lumpur, Malaysia1d ago
-
Data Engineer USD 122K-169KAlerting | BigQuery | Cloud platform | DBT | Data GovernanceAutonomy | Growth opportunities | High ownershipMid-level Full TimeMalaysia2d ago
-
Senior-level Full TimePenang 15, Penang, Malaysia3d ago
-
Software Enabling and Optimization Engineer USD 100K-172KAI | C# | C++ | Code optimization | GoOn-site workEntry-level Full TimeMYS - Penang, Malaysia3d ago
-
Senior Software Engineer, Customer Data Platform USD 142K-235KAPI Development | AWS | Amazon DynamoDB | Apache Flink | Apache SparkBirthday leave | Employee assistance programme | FlexWork Benefits Package | Flexible work arrangements | Medical insuranceSenior-level Full TimePetaling Jaya, Malaysia4d ago
-
Lead Software Engineer, Customer Data Platform USD 140K-207KAPI Development | AWS | Amazon DynamoDB | Apache Flink | Apache HiveBirthday leave | Employee assistance programme | Flex work | Flexible benefits package | Life insuranceSenior-level Full TimePetaling Jaya, Malaysia4d ago
-
Mid-level Full TimeKuala Lumpur4d ago
-
Activiti | BPMN 2.0 | BigQuery | Cloud platform | CollibraMid-level Full TimeKuala Lumpur, MY4d ago
-
AI Developer MYR 68K-113KAWS | Algorithms | Big Data | Databases | Google CloudAnnual health screening | Career development | Company mobile phone | Continuous training | Employee engagement activitiesMid-level Full TimeKelana Jaya, Selangor, Malaysia R5d ago
-
C# | Element analysis | Finite Element Analysis | Finite element | GrasshopperSenior-level Full TimeKuala Lumpur, Malaysia6d ago
-
Senior Data Engineer PLN 206K-328KAWS | Agile | Airflow | Alerting | Apache SparkCareer development planning | Casual dress | Conferences | Courses | Flexible workingSenior-level Full TimeKuala Lumpur, Federal Territory of Kuala …6d ago
-
Data Engineer CAD 120K-145KAWS | Amazon ECR | Amazon EKS | Azure | CI/CDDevelopment opportunities | Equal opportunity | Reward and recognition | Volunteering | WellnessSenior-level Full TimeCyberjaya, Selangor, Malaysia6d ago
-
Data Engineer CAD 95K-122KAWS | Airflow | Apache Spark | CI/CD | Code reviewCasual dress | Flexible working | Tailored career development planning | Work from homeMid-level Full TimeKuala Lumpur, Federal Territory of Kuala …6d ago
-
Azure Databricks USD 172K-241KAPI Integration | Access Control | Access Control Lists | Apache Deequ | Apache SparkMid-level Full TimeKuala Lumpur, MY6d ago
-
Senior Data Engineer USD 161K-216KAPI Integration | Airflow DAGs | Airflow operators | Airflow sensors | Apache AirflowSenior-level Full TimePetaling Jaya (First Avenue), Malaysia6d ago
-
Hiring: Data Engineer for Leading Client in Malaysia, KL INR 1500K-2000KAzure Data | Azure Data Factory | Azure Databricks | Azure Key Vault | Azure SynapseMid-level Full TimeKuala Lumpur, Federal Territory of Kuala …6d ago
-
Senior-level Full TimeKuala Lumpur, MY6d ago
-
Azure Databricks Engineer USD 180K-248KADLS | Apache Deequ | Apache Hadoop | Apache Spark | Azure DataSenior-level Full TimeKuala Lumpur, MY6d ago
-
AWS | Automation | Capacity Planning | Clustering | ComplianceHybrid workSenior-level Full TimeKuala Lumpur, Malaysia7d ago
-
Senior-level Full TimeBatu Kawan, Penang, Malaysia7d ago
-
T&T Senior Consultant - Artificial Intelligence & Data (Google Cloud Data Engineer) - SG USD 118K-179KAnalytics | BigQuery | Cloud platform | Data Cleansing | Data GovernanceMid-level Full TimeKuala Lumpur, MY7d ago