Azure Databricks
Tasks
- Build and optimize big data platforms
- Design and manage Spark compute environments
- Develop batch and streaming data pipelines
- Ensure data governance and security
- Implement data quality frameworks
- Manage cloud administration for cost monitoring backups automation
- Manage data storage and modeling
Perks/Benefits
- N/A
Skills/Tech-stack
API Integration | Access Control | Access Control Lists | Apache Deequ | Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Databricks | Batch Processing | Command Line | Command-line Interface | DBT | Data Factory | Data Governance | Data Lake | Data Lake Storage | Data Lakehouse | Data Lineage | Data Pipelines | Data Quality | Data Security | Data Warehouse | Distributed Computing | Great Expectations | PII | RDBMS | Schema evolution | Streaming
Education
N/A
Regions
Countries
States
Related jobs
-
Data Engineer CAD 120K-145KAWS | Amazon ECR | Amazon EKS | Azure | CI/CDDevelopment opportunities | Equal opportunity | Reward and recognition | Volunteering | WellnessSenior-level Full TimeCyberjaya, Selangor, Malaysia6h ago
-
Senior-level Full TimeBatu Kawan, Penang, Malaysia1d ago
-
T&T Senior Consultant - Artificial Intelligence & Data (Google Cloud Data Engineer) - SG USD 118K-179KAnalytics | BigQuery | Cloud platform | Data Cleansing | Data GovernanceMid-level Full TimeKuala Lumpur, MY1d ago
-
Senior Machine Learning Engineer USD 157K-200KApache Spark | CI/CD | Causal Inference | Deep learning | ETLBirthday leave | Flexwork arrangements | GrabFlex Benefits Package | Life Assistance Programme | Medical insuranceSenior-level Full TimePetaling Jaya, Malaysia1d ago
-
Automation | Dashboarding | Data Quality | Data Validation | Data VisualizationMid-level Full TimeKuala Lumpur, Malaysia2d ago
-
Data Engineer USD 118K-165KBigQuery | Cloud platform | DBT | Data Governance | Data LakesAutonomy | Growth opportunities | TrustMid-level Full TimeMalaysia2d ago
-
Software Developer/ Software Programmer(ETL) USD 110K-155KApache NiFi | Data Governance | Data Warehouse | ETL | Metadata ManagementSenior-level Full TimeKuala Lumpur4d ago
-
Entry-level Full TimeKuala Lumpur - RedQ, Malaysia4d ago
-
AI Engineer CAD 119K-157KAccess Control | Anonymization | Authentication | Authorization | Cloud ComputingMid-level Full TimeMY-Cyberjaya, Malaysia4d ago
-
Senior Machine Learning Engineer USD 154K-200KApache Spark | Causal Inference | Data pipeline | Deep learning | ETLBirthday leave | Employee assistance programme | FlexWork | Flexible benefits plan | Medical insuranceSenior-level Full TimePetaling Jaya, Malaysia5d ago
-
Intern, Transformation Automation USD 50K-58KAPI Integration | Artificial Intelligence | Machine Learning | OpenAI | Performance TuningEntry-level Full Time InternshipMY-Kuala Lumpur-IT, Malaysia6d ago
-
Intern, Transformation Automation USD 50K-58KAPI Integration | Agent systems | Artificial Intelligence | Automation | Machine LearningCareer growth | Learning opportunitiesEntry-level Full Time InternshipMY-Kuala Lumpur-IT, Malaysia6d ago
-
Intern, Transformation Automation USD 50K-58KAI Model Monitoring | AI model | API Integration | Agent systems | Language ProcessingCareer development | Learning opportunities | MentorshipEntry-level Full Time InternshipMY-Kuala Lumpur-IT, Malaysia6d ago
-
AI Staff Software Engineer – AI Agent Development USD 139K-220KAPI Integration | Context handling | Decision Making | Evaluation | ExperimentationSenior-level Full TimeBayan Lepas, Penang, Malaysia7d ago
-
AI Senior Software Engineer – AI Agent Development USD 144K-212KAPI Integration | Decision Making | Evaluation | Language Models | Language ProcessingSenior-level Full TimeBayan Lepas, Penang, Malaysia7d ago
-
AI Senior Software Engineer – AI Agent Development USD 144K-212KAPI Integration | Context handling | Decision Making | Evaluation | GuardrailsSenior-level Full TimeBayan Lepas, Penang, Malaysia7d ago
-
AI Senior Software Engineer – AI Agent Development USD 144K-212KAPI Integration | Agent Orchestration | Context handling | Debugging | Decision MakingSenior-level Full TimeBayan Lepas, Penang, Malaysia7d ago
-
AI Senior Software Engineer – AI Agent Development USD 144K-203KAPI Integration | Artificial Intelligence | Context handling | Evaluation | GuardrailsSenior-level Full TimeBayan Lepas, Penang, Malaysia7d ago
-
AI Staff Software Engineer – AI Agent Development USD 139K-220KAPI Integration | Context handling | Debugging | Decision Making | EvaluationSenior-level Full TimeBayan Lepas, Penang, Malaysia7d ago
-
AI Senior Software Engineer – AI Agent Development USD 144K-203KAPI Integration | Debugging | Decision Making | Evaluation | Language ModelsSenior-level Full TimeBayan Lepas, Penang, Malaysia7d ago
-
Data Engineer USD 90K-120KApache Airflow | Apache Kafka | Azure Data | Azure Data Factory | Azure Data LakeAnnual leave | Car loan subsidy | Career development opportunities | Company insurance | Flexi-work hoursMid-level Full TimeBatu Kawan, Penang, Malaysia8d ago
-
AI Lead USD 155K-180KAI strategy | Agentic AI | Data Pipelines | Deep learning | Language ProcessingSenior-level Full TimeSoutheast Asia, Malaysia8d ago
-
Data & AI Engineer USD 139K-196KAI Services | Access Control | Apache Spark | Azure AI | Azure AI ServicesSenior-level Full TimeKuala Lumpur, MY, Malaysia8d ago
-
Senior Software Engineer, Data (L3) MYR 168K-185KAmazon Redshift | Apache Airflow | Apache Flink | Apache Kafka | ClickHouseAnnual company offsite | Insurance coverage | Learning allowance | Meal allowance | Parking allowanceSenior-level Full TimeMalaysia11d ago
-
Data Engineer USD 91K-130K.NET | Apache Storm | Azure Data Factory | Bash | CassandraHybrid workingMid-level Full TimeMYS, Petaling Jaya (NDMY), Malaysia11d ago