Data engineer (Azure)
Tasks
- Build ETL data pipelines using PySpark and Python
- Connect to source systems including on premises databases IoT devices and APIs
- Deploy pipeline artifacts across environments with Azure DevOps
- Design and develop Pyspark scripts
- Design and manage data storage and databases
- Develop Power BI reports and dashboards
- Implement Power BI row level security
- Orchestrate data pipelines
- Perform data integration with database joins
- Test and troubleshoot data pipelines
Perks/Benefits
Skills/Tech-stack
APIs | Azure | Azure Data | Azure Data Explorer | Azure Data Factory | Azure DevOps | Azure Synapse | Azure Synapse Analytics | Copy Data | Copy Data Activity | DAX | Data Explorer | Data Factory | Data Ingestion | Data Mart | Data Warehouse | ETL | HANA | Hadoop | Power BI | PySpark | Python | SQL | Synapse Analytics
Education
Roles
Related jobs
-
APIs | Azure | Azure Data | Azure Data Explorer | Azure Data FactoryOnsite workMid-level Full TimeSingapore, Singapore5h ago
-
.NET | AKS | App Service | Azure App | Azure App ServiceMid-level Full TimeSingapore, Singapore5h ago
-
Algorithms | Apollo | AutoWare | C++ | CARLABirthday leave | FlexWork | Insurance assistance programme | Life insurance | Medical insuranceSenior-level Full TimeSingapore, Singapore16h ago
-
Lead Software Engineer – Data Engineering, Python/KDB SGD 150K-186KAI | API Development | AWS | Agile | AnalyticsSenior-level Full TimeSingapore16h ago
-
Entry-level Full TimeSingapore18h ago
-
Software Engineer (Generative AI Engineer) SGD 159K-220KAPI Design | AWS Bedrock | Bedrock Agents | Data Privacy | Data maskingFlexible work practices | Learning opportunities | Self-development timeMid-level Full TimeSingapore, SG19h ago
-
Software Engineer (Full Stack / GenAI Applications) SGD 120K-143KAWS | AWS KMS | Data Visualization | DynamoDB | FastAPIFlexible work practices | Learning opportunities | Paid self-development timeMid-level Full TimeSingapore, SG19h ago
-
Software Engineer, Quality & Platform (GenAI) SGD 138K-143KAWS | AWS Bedrock | Amazon CloudWatch | Amazon ECS | Amazon S3Flexible work practices | Learning opportunities | Self-development timeMid-level Full TimeSingapore, SG19h ago
-
AI Data Engineer Intern SGD 40K-45KAirflow | Amazon Web Services | Business Intelligence | CI/CD | Cloud platformEntry-level Full Time InternshipSingapore19h ago
-
AI Software Engineering (Industry Attachment) SGD 70K-120KArtificial Intelligence | Machine Learning | Python | Software EngineeringGamer-centric culture | Global team collaboration | Personal growth | Professional growthMid-level Full TimeSingapore19h ago
-
CAN bus | Connector selection | Current Sense Amplifiers | DC Power Systems | DC convertersCross-functional collaboration | Fast-paced environment | Open source contributions | Ownership of hardware deployments | Work on robotics platformSenior-level Full TimeSingapore1d ago
-
Amazon S3 | Cloud Native | Concurrency | Data Consistency | Data IngestionFreedom to innovate | Open collaborative environment | Ownership of system architectureSenior-level Full TimeSingapore1d ago
-
Full Stack Engineer (GenAI application) SGD 100K-143KAPI Integration | AWS Bedrock | AWS DynamoDB | AWS IAM | AWS KMSMid-level Full TimeSingapore1d ago
-
AI Governance | AI Risk | AI Search | AI risk management | Agent systemsCoaching | Continuous learning | Flexible working arrangementsEntry-level Full TimeSG, 0485831d ago
-
Senior Embedded Functional Test Engineer SGD 96K-132KAgile | Bug tracking | Case design | Continuous integration | Embedded SystemsDigital course library | Employee assistance program | Hybrid work model | Mindfulness programs | OnTrack wellbeing programsSenior-level Full TimeSingapore Singapore (Corporation Road)1d ago
-
Entry-level Full TimeSINGAPORE, Singapore1d ago
-
Staff Software Engineer SGD 150K-203KApache Hadoop | Apache Spark | Backpressure | CAP Theorem | Circuit BreakerContinued Career Development | Employee resource groups | Flexible WFH policy | Generous PTO | Internet reimbursementSenior-level Full TimeSingapore-Singapore1d ago
-
Computer Vision | Data Mining | Deep learning | Graph Neural Networks | Language ProcessingEntry-level Full TimeSingapore, Singapore2d ago
-
Software Engineer, Machine Learning, Payments SGD 138K-148KC# | C++ | Data Processing | Debugging | GoMid-level Full TimeSingapore2d ago
-
API Integration | Agent systems | Cloud platform | Cost Per Request | CrewAISenior-level Full TimeSingapore2d ago
-
Professional Services Engineer SGD 73K-150KAPI | Amazon Web Services | Authentication | CI/CD | JavaMid-level Full TimeSingapore, SG2d ago
-
Senior-level Full TimeSingapore2d ago
-
Senior-level Full TimeSingapore2d ago
-
Senior-level Full TimeSingapore, Singapore2d ago
-
Data Analytics Engineer/Data Scientist SGD 150K-214KAirflow | Amazon Redshift | BigQuery | DBT | Data BlendingSenior-level Full TimeSingapore - Technology Centre2d ago