Data engineer (Quantexa)
Singapore, Singapore, Singapore
SGD 96K-140K (estimate) Mid-level Full Time
Tasks
- Collaborate with DevOps on CI/CD pipelines
- Collaborate with cross-functional teams on data requirements
- Design and deploy solutions on OpenShift Container Platform
- Design develop and implement Spark Scala data processing pipelines
- Develop fault tolerant Spark applications
- Document data engineering processes workflows and configurations
- Ensure data quality and integrity
- Implement data governance and data lineage
- Implement data transformation aggregation and enrichment
- Implement monitoring and logging
- Integrate elasticsearch with spark
- Monitor analyze job performance metrics and bottlenecks
- Optimize and tune Spark jobs performance and scalability
- Optimize workflows for containerized deployment
- Troubleshoot Spark and Elasticsearch data processing issues
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Hadoop | Apache Spark | CI/CD | Container Platform | Docker | Elasticsearch | OpenShift Container Platform | Scala
Education
N/A
Roles
Related jobs
-
Sr. Solutions Engineer SGD 184K-225KApache Spark | Cloud Computing | Data Lakehouse | Delta Lake | IntegrationSenior-level Full TimeSingapore13h ago
-
AI Engineer (FDE - Full Development Engineer) SGD 74K-74KAI Foundry | AI Governance | AWS Bedrock | Autogen | Azure AISenior-level Full TimeSingapore, Singapore, Singapore18h ago
-
AWS DataZone | AWS Glue | AWS Glue Data Catalog | AWS KMS | AWS Lake FormationFlexible work environment | Inclusive cultureMid-level Full TimeSG, 04858318h ago
-
.NET | AKS | Azure DevOps | Azure Functions | Azure OpenAIMid-level Full TimeSingapore, Singapore1d ago
-
Senior Systems Development Engineer SGD 122K-190KAutomated testing | CI/CD | Cause analysis | Cloud Computing | Code reviewSenior-level Full TimeSingapore1d ago
-
Data Engineer SGD 132K-167KAWS DMS | AWS Glue | AWS Lambda | Agile | Amazon AthenaCertification support | Coaching | Hybrid work environment | Learning opportunities | Wellbeing programsEntry-level Full TimeKSG51588 Singapore (KSG51588)1d ago
-
Principal Forward Deployed Engineer - AI PlatformPrincipal Forward Deployed Engineer - AI Platform/Kubernetes/Pytorch SGD 191K-225KAgentOps | Agentic Operators | C# | C++ | CI/CDSenior-level Full TimeSingapore R1d ago
-
Senior-level Full TimeSingapore, Singapore, Singapore1d ago
-
Big Data Site Reliability Engineer SGD 18K-27KAlgorithms | ClickHouse | Data Structures | Elasticsearch | GoIncident Response | On-call rotationEntry-level Full Time新加坡1d ago
-
Full Stack Engineer SGD 74K-74KAI orchestration | AWS | Agent Frameworks | Authentication | Automated testingSenior-level Full TimeSingapore2d ago
-
Business Support Engineer - Meta Business Agents SGD 100K-190KAI Agents | API troubleshooting | AWS | Agent Orchestration | Azure24 7 Oncall Rotation | Professional documentation supportMid-level Full TimeSingapore2d ago
-
M42 - Full Stack Engineer SGD 74K-74KAI orchestration | AWS | Agent Frameworks | Authentication | Automated testingSenior-level Full TimeSingapore2d ago
-
Software Senior Engineer SGD 162K-225KAPI Design | Agentic AI | Agile methodologies | Backend Development | BenchmarkingSenior-level Full TimeSingapore2d ago
-
Bash | Cloud infrastructure | Cost Throughput Optimization | Data Ingestion | Data Ingestion PipelinesAsynchronous work culture | Career impact on large scale product | Friendly laid-back atmosphereMid-level Full TimeSingapore, Singapore2d ago
-
Mid-level Full TimeRemote - Singapore R2d ago
-
Sr Full-Stack AI Engineer, IPAI, SMAI SGD 162K-225K3D Modeling | AWS | Agentic AI | Angular | Anomaly DetectionSenior-level Full TimeFab 10A, Singapore2d ago
-
Full-Stack AI Engineer, IPAI, SMAI SGD 130K-200KAWS | Agentic patterns | Angular | Azure | BatchingEntry-level Full TimeFab 10A, Singapore2d ago
-
Senior Product Development Test Engineer, Data Analytics SGD 160K-191KAWS | Apache Spark | Binning | Data Governance | Data LineageSenior-level Full TimeSingapore, Central Singapore, Singapore2d ago
-
Software Engineer SGD 60K-120KAgile | Automated testing | CI/CD | Database Design | Design PatternsMid-level Full TimeSingapore - Singapore - Block 33 …2d ago
-
(Senior) Development Partner, Logistics Sector Resilience / Crisis Distribution (Data Platforms & Operations) SGD 118K-162KAPI Integration | Apache Spark | Cloud Migration | Compliance | DashboardingSenior-level Full TimeESG - Bugis Junction Office Tower, …2d ago
-
Senior Data Scientist / Machine Learning Engineer SGD 135K-168KAPIs | Cross-validation | Data Analysis | Data Visualization | Data WranglingCollaborative workplace | Comprehensive benefits | Meaningful projects | Safe space for diverse perspectivesSenior-level Full TimeDigital Hub, SG2d ago
-
Apache Kafka | CI/CD | Docker | GraphQL | JavaScriptSenior-level Full TimeSingapore, Singapore3d ago
-
Lead Data & Integration Engineer SGD 162K-225KBatch Processing | Branching | CI/CD | Cloudera | Code ReviewsSenior-level Full TimeSingapore3d ago
-
AI Engineer SGD 74K-74KAI Platform | Amazon SageMaker | Azure Machine Learning | CI/CD | Cloud AI PlatformSenior-level Full TimeSingapore, Raffles City Tower3d ago
-
Senior-level Full TimeIOI Central Boulevard Towers, Singapore3d ago