Data Engineer
Tasks
- Apply data modeling and schema evolution
- Automate deployment scripts with Python
- Collaborate on source and downstream data requirements
- Design data ingestion pipelines
- Develop and maintain Kafka Connect connectors
- Implement change data capture with Debezium
- Ingest MySQL and PostgreSQL data into data lake
- Load data into AWS S3 GCS and BigQuery
- Monitor and optimize Kafka Connect clusters
- Optimize ingestion performance and cost
- Partition data in bronze layer
- Stay current with Apache Hudi and Apache Iceberg
Perks/Benefits
- N/A
Skills/Tech-stack
AWS CloudFormation | Amazon S3 | Apache Airflow | Apache Hudi | Apache Iceberg | Apache Kafka | Cloud Storage | Dagster | Data Modeling | Data Partitioning | Data Quality | Debezium | Docker | ELT | ETL | Google BigQuery | Google Cloud | Google Cloud Storage | Kafka Connect | Kubernetes | MySQL | Observability | PostgreSQL | Python | Schema evolution | Terraform
Education
N/A
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R12d ago
-
Senior Data Engineer-JT0224 USD 115K-175K.Net Core | .Net Framework | AWS CloudFormation | Apache Airflow | Azure401k match | Career growth opportunities | Dental insurance | Employee resource groups | Health insuranceSenior-level Full TimeRemote, United States R2h ago
-
Senior Machine Learning Engineer USD 140K-224KAIOps | AWS | Apache Spark | Cloud platform | Data Analysis401k match | Dental insurance | Flexible paid time off | Life insurance | Medical insuranceSenior-level Full TimeRemote, United States R2h ago
-
Senior-level Full TimeRemote R5h ago
-
Data Warehouse, ETL, Cloud / Consultant Specialist INR 3000K-4000KAWS | Agile | Ansible | Cloud platform | Confluent KafkaFlexible working | Inclusive work environment | Professional developmentSenior-level Full TimePune, Maharashtra, India R5h ago
-
Amazon Redshift | Data Migration | Data Modeling | Data Quality | Data WarehousingFlexible schedule | Remote workSenior-level Full TimeRemote R6h ago
-
Asynchronous training | Containerization | Curriculum learning | Distributed Systems | Distributed TrainingSenior-level Full TimeSF Bay Area, CA, Remote, US, … R6h ago
-
Machine Learning Engineer / AI Model Developer (Mid to Senior)(Top Secret Clearance Required) (Hybrid) USD 78K-176KAPIs | Agile methodologies | Backend Development | Cloud Computing | Data Preparation401k | Employee discount program | Employee referral rewards | Flexible spending account | Flexible work scheduleMid-level Full TimeFort Belvoir, VA, US R8h ago
-
GCP, Digital Analytics, Bigquery/Senior Consultant Specialist INR 2500K-3900KAttribution Modeling | Batch Processing | BigQuery | Cloud platform | Customer AnalyticsCareer growth | Flexible working | Inclusive workplace | Opportunities for professional developmentSenior-level Full TimeHyderabad, Telangana, India R9h ago
-
Ingénieur Big Data (F/H) EUR 50K-70KApache Kafka | Apache Spark | Big Data | Data Ingestion | Distributed ComputingFlexible internal work modes | Parental leave support | Telework 3 days per week | Training and ASI AcademyMid-level Full TimeRennes, Brittany, France R9h ago
-
Senior Databricks Engineer USD 150K-170KAsset bundles | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2401k matching | Dental insurance | Flexible spending accounts | Health insurance | Paid HolidaysSenior-level Full TimeVirtual R10h ago
-
Data Engineer GBP 723KAWS | AWS CDK | Amazon Redshift | Amazon Web Services | BitbucketBike loan scheme | Discounted private healthcare | Employee assistance programme | Enhanced family leave | Health cash planMid-level Full TimeManchester / Hybrid, England, United Kingdom R12h ago
-
Senior Staff Engineer (LLM) GBP 80K-109KAgent systems | Anthropic API | Authentication | Chunking | EmbeddingsSenior-level Full TimeRemote, United Kingdom R14h ago
-
Azure Databricks | Azure DevOps | Data Quality | Data Warehousing | DatabricksCollaborative Agile environment | Fully remote | Hands on cloud data platform exposureMid-level Full TimeIndia R15h ago
-
AWS | Autoscaling | Bash | CI/CD | Data PipelinesFully remote work | Healthcare coverage | Paid parental leave | Relocation assistance | Technical ownershipMid-level Full TimeRomania R16h ago
-
A/B | A/B Testing | AWS | Alerting | AutoscalingFully remote within Europe | Healthcare coverage | Paid parental leave | Professional growth | Relocation assistanceMid-level Full TimeItaly R16h ago
-
AWS | Autoscaling | Azure | Bash | CI/CDFully remote within Europe | Healthcare coverage | Paid parental leave | Professional growth | Relocation assistanceMid-level Full TimePortugal R16h ago
-
AWS | Bash | Cloud platform | Databricks | DatadogEquity package | Fully remote Europe | Healthcare coverage | Paid parental leave | Relocation assistanceMid-level Full TimeNetherlands R16h ago
-
A/B | A/B Testing | AWS | Alerting | AzureFully remote Europe | Healthcare coverage | Paid parental leave | Professional growth | Relocation assistanceMid-level Full TimeIreland R16h ago
-
A/B | A/B Testing | AWS | Azure | B testingFully remote work | Healthcare coverage | Paid parental leave | Relocation assistanceMid-level Full TimeSwitzerland R16h ago
-
AWS | Autoscaling | Azure | Bash | CI/CDEquity package | Fully remote work within Europe | Healthcare coverage | Paid parental leave | Relocation assistanceMid-level Full TimeFrance R16h ago
-
A/B | A/B Testing | AWS | Autoscaling | B testingEquity package | Fully remote work Europe | Healthcare coverage | Paid parental leave | Professional growthMid-level Full TimeSpain R16h ago
-
AWS | Autoscaling | Azure | Bash | CI/CDFully remote work Europe | Healthcare coverage | Paid parental leave | Relocation assistanceMid-level Full TimeGermany R17h ago
-
Sr Clojure Backend Software Engineer MXN 911K-1300KClojure | ClojureScript | LLM APIs | PostgreSQL | Prompt engineeringSenior-level Full TimeRemote, Mexico R19h ago
-
Conversational AI Engineer EUR 75K-125KAI orchestration | API Integration | Agentic AI | JSON | JavaScriptMid-level Full TimeRemote, Any Location, ES R20h ago