Data Engineer (Spark)
Tasks
- Collaborate with teams on data requirements
- Design data pipelines for streaming and batch
- Develop and maintain data processing platform
- Leverage AWS for infrastructure scaling
- Manage data lake using Iceberg
- Monitor and troubleshoot platform performance and accuracy
- Optimize data workflows for ingestion processing storage
- Write and maintain Python code for data processing
Perks/Benefits
- Career development
- Conference support
- Flexible work arrangements
- Integration budget
- Knowledge sharing
- Language classes
- Medical coverage
- Paid time off
- Sponsored training
- Sports packages
- Team building events
- Wellness support
- Work equipment provided
- Work remote options
Skills/Tech-stack
AWS | Apache Airflow | Apache Iceberg | Apache Kafka | Apache NiFi | Apache Spark | Cloudera | DBT | Data Governance | Data Management | Data Modeling | Databricks | Dimensional Modeling | Docker | Google BigQuery | Java | Kubeflow | Looker | MLOps | MLflow | Python | Scala
Education
Roles
Related jobs
-
Featured Feat. AI Engineer (MTS) USD 160K-300KAPI Development | AWS | Amazon Web Services | Deep learning | FastAPIMentoring | Open source contributions | Remote workMid-levelRemote R11d ago
-
Featured Feat. Data Engineer USD 80K-150KData Monitoring | Data Quality | Data Validation | ELT | ETLRemote workEntry-levelRemote R11d ago
-
Mid-level Full Time北京 R5h ago
-
Mid-Level Data Engineer USD 90K-98KAPI Development | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageRemote workMid-level Full TimeWork from home, VA, United States R7h ago
-
Senior Data Engineer USD 165K-180KAPIs | Anomaly Detection | Azure | Azure Data | Azure Data FactorySenior-level Full TimeWork from home, VA, United States R7h ago
-
AWS Glue | AWS Lambda | Airflow | Amazon S3 | AzureRemote workSenior-level Full TimeRemote R9h ago
-
Data Engineer (MS) (Remote) INR 2040K-3380KCI/CD | Data Transformation | Data Validation | Date normalization | ETLMentorship opportunities | Professional growth | Remote workSenior-level Full TimeMaharashtra, Pune, India R9h ago
-
AI / LLM Integration Engineer (Remote) INR 1800K-2800KAPI Integration | Anthropic SDK | Async APIs | Benchmarking | Confidence scoring100 percent remote | Collaborative culture | Inclusive team culture | Professional growthMid-level Full TimeMaharashtra, Pune, India R9h ago
-
ML / LLM Engineer (Remote) INR 2500K-3000KAmazon Web Services | Azure | Classification | Feature Engineering | Language ModelsRemote workMid-level Full TimeMaharashtra, Pune, India R10h ago
-
Evergreen - Mathematics for Machine Learning USD 80K-300KAutodiff | JAX | Linear Algebra | Machine Learning | Matrix OperationsPart-time workMid-level Full TimeRemote, Remote, BR R10h ago
-
C++ | Cloud platform | Data Pipelines | ETL | Google CloudCDI | Career growth opportunities | Flexible work environment | Telework 1 day per weekSenior-level Full TimeCastelnaudary, France R11h ago
-
Maitrise Ouvrage/Support Booking & Risk H/F EUR 21K-21KOffice 365 | Python | SQLAgile team environment | Career coaching | HR support | Modern campus services | Remote work opportunityEntry-level Full TimeEurope, France, Ile-de-France, 92 - Hauts-De-Seine R14h ago
-
Data Engineer Databricks (H/F) EUR 47K-55KAmazon Web Services | Apache Spark | Azure | Azure DevOps | CI/CDCareer development | Flexible remote work | Meal tickets | Paid time off | RTT daysSenior-level Full TimeSAINT OUEN, France R15h ago
-
Analytics Engineer EUR 48K-78KAmazon Redshift | Cloud platform | DBT | Data Modeling | Data TestingAnnual blood tests | Company apartments | Extra paid vacation days | Flexible schedule | Insurance and wellness programsMid-level Full TimeKaunas, Lithuania R15h ago
-
Senior Databricks EUR 46K-55KAWS | Apache Spark | Azure | Azure DevOps | Batch ProcessingCareer coaching | Conference speaking opportunities | Flexible telework | Meal tickets | Paid time offSenior-level Full TimeSAINT OUEN, France R15h ago
-
Anthropic Claude | Async Programming | ChatGPT | Claude Code | CodexDirect user impact | Flexible work schedule | Occasional international travel | Work from home optionMid-level Full TimeSlovenia / Remote R17h ago
-
Data Operations Engineer INR 2040K-3380KApache Airflow | Data Pipelines | Data Refresh | Data Warehousing | Data pipelineEmergency incident response support | On-call rotationSenior-level Full TimeBengaluru, INDIA, India R17h ago
-
APIs | Anomaly Detection | Data Modeling | Data Pipelines | Docker100% remote work | Career growth opportunities | Flexible work environmentMid-level Full TimeEstonia R19h ago
-
APIs | Anomaly Detection | Data Modeling | Data Pipelines | Docker100 percent remote work | Autonomous work environment | Career growth | Flexible work environment | International team cultureMid-level Full TimeHungary R19h ago
-
Anomaly Detection | Data Modeling | Data Pipelines | Docker | ForecastingCareer growth | Flexible work environment | Remote workMid-level Full TimeFinland R19h ago
-
Anomaly Detection | Data Modeling | Data Pipelines | Docker | JavaScript100% remote work | Career growth opportunities | Flexible work environmentMid-level Full TimeCzechia R19h ago
-
APIs | Anomaly Detection | Data Modeling | Data Pipelines | Docker100% remote work | Career growth opportunities | Flexible work environment | International team cultureMid-level Full TimeNorway R19h ago
-
API Integration | Anomaly Detection | Data Modeling | Docker | Machine Learning100 percent remote work | Autonomy | Career growth | Flexible work environment | International team cultureMid-level Full TimeLuxembourg R19h ago
-
API Integration | Anomaly Detection | Data Modeling | Data Pipelines | DockerCareer growth | Flexible schedule | International team culture | Remote workMid-level Full TimeCroatia R19h ago
-
APIs | Anomaly Detection | Data Modeling | Data Pipelines | Docker100% remote work | Autonomy | Career growth | Flexible work environment | International team cultureMid-level Full TimeBulgaria R19h ago