Data Engineer
Tasks
- Apply data modeling and schema evolution
- Automate deployment scripts with Python
- Collaborate on source and downstream data requirements
- Design data ingestion pipelines
- Develop and maintain Kafka Connect connectors
- Implement change data capture with Debezium
- Ingest MySQL and PostgreSQL data into data lake
- Load data into AWS S3 GCS and BigQuery
- Monitor and optimize Kafka Connect clusters
- Optimize ingestion performance and cost
- Partition data in bronze layer
- Stay current with Apache Hudi and Apache Iceberg
Perks/Benefits
- N/A
Skills/Tech-stack
AWS CloudFormation | Amazon S3 | Apache Airflow | Apache Hudi | Apache Iceberg | Apache Kafka | Cloud Storage | Dagster | Data Modeling | Data Partitioning | Data Quality | Debezium | Docker | ELT | ETL | Google BigQuery | Google Cloud | Google Cloud Storage | Kafka Connect | Kubernetes | MySQL | Observability | PostgreSQL | Python | Schema evolution | Terraform
Education
N/A
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R12d ago
-
Amazon Redshift | Data Migration | Data Modeling | Data Quality | Data WarehousingFlexible schedule | Remote workSenior-level Full TimeRemote R2h ago
-
Machine Learning Engineer / AI Model Developer (Mid to Senior)(Top Secret Clearance Required) (Hybrid) USD 78K-176KAPIs | Agile methodologies | Backend Development | Cloud Computing | Data Preparation401k | Employee discount program | Employee referral rewards | Flexible spending account | Flexible work scheduleMid-level Full TimeFort Belvoir, VA, US R4h ago
-
Ingénieur Big Data (F/H) EUR 50K-70KApache Kafka | Apache Spark | Big Data | Data Ingestion | Distributed ComputingFlexible internal work modes | Parental leave support | Telework 3 days per week | Training and ASI AcademyMid-level Full TimeRennes, Brittany, France R6h ago
-
Senior Databricks Engineer USD 150K-170KAsset bundles | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2401k matching | Dental insurance | Flexible spending accounts | Health insurance | Paid HolidaysSenior-level Full TimeVirtual R6h ago
-
Data Engineer GBP 723KAWS | AWS CDK | Amazon Redshift | Amazon Web Services | BitbucketBike loan scheme | Discounted private healthcare | Employee assistance programme | Enhanced family leave | Health cash planMid-level Full TimeManchester / Hybrid, England, United Kingdom R8h ago
-
Senior Staff Engineer (LLM) GBP 80K-109KAgent systems | Anthropic API | Authentication | Chunking | EmbeddingsSenior-level Full TimeRemote, United Kingdom R10h ago
-
Azure Databricks | Azure DevOps | Data Quality | Data Warehousing | DatabricksCollaborative Agile environment | Fully remote | Hands on cloud data platform exposureMid-level Full TimeIndia R11h ago
-
AWS | Autoscaling | Bash | CI/CD | Data PipelinesFully remote work | Healthcare coverage | Paid parental leave | Relocation assistance | Technical ownershipMid-level Full TimeRomania R13h ago
-
A/B | A/B Testing | AWS | Alerting | AutoscalingFully remote within Europe | Healthcare coverage | Paid parental leave | Professional growth | Relocation assistanceMid-level Full TimeItaly R13h ago
-
AWS | Autoscaling | Azure | Bash | CI/CDFully remote within Europe | Healthcare coverage | Paid parental leave | Professional growth | Relocation assistanceMid-level Full TimePortugal R13h ago
-
AWS | Bash | Cloud platform | Databricks | DatadogEquity package | Fully remote Europe | Healthcare coverage | Paid parental leave | Relocation assistanceMid-level Full TimeNetherlands R13h ago
-
A/B | A/B Testing | AWS | Alerting | AzureFully remote Europe | Healthcare coverage | Paid parental leave | Professional growth | Relocation assistanceMid-level Full TimeIreland R13h ago
-
A/B | A/B Testing | AWS | Azure | B testingFully remote work | Healthcare coverage | Paid parental leave | Relocation assistanceMid-level Full TimeSwitzerland R13h ago
-
AWS | Autoscaling | Azure | Bash | CI/CDEquity package | Fully remote work within Europe | Healthcare coverage | Paid parental leave | Relocation assistanceMid-level Full TimeFrance R13h ago
-
A/B | A/B Testing | AWS | Autoscaling | B testingEquity package | Fully remote work Europe | Healthcare coverage | Paid parental leave | Professional growthMid-level Full TimeSpain R13h ago
-
AWS | Autoscaling | Azure | Bash | CI/CDFully remote work Europe | Healthcare coverage | Paid parental leave | Relocation assistanceMid-level Full TimeGermany R13h ago
-
Sr Clojure Backend Software Engineer MXN 911K-1300KClojure | ClojureScript | LLM APIs | PostgreSQL | Prompt engineeringSenior-level Full TimeRemote, Mexico R15h ago
-
Conversational AI Engineer EUR 75K-125KAI orchestration | API Integration | Agentic AI | JSON | JavaScriptMid-level Full TimeRemote, Any Location, ES R16h ago
-
Senior-level Full TimeRemote, KA, IN R16h ago
-
Data Analyst & Administrator Trainee EUR 28K-28KDashboard Development | Data Analysis | Data Integrity | Data Visualization | Data cleaningCross-functional team | Flexible working hours | Hybrid work schedule | MentorshipEntry-level Full TimeLisboa, Portugal R16h ago
-
CI/CD | CPU | Computer Vision | Data Augmentation | Deep learningHybrid work | Occasional remote work | Professional mentorshipMid-level Full TimeParis, France R16h ago
-
Sr. Embedded Software/Firmware Engineer USD 180K-240KARM | ASPICE | AUTOSAR | C# | C++Competitive compensation | Dental insurance | Flexible hours | Medical insurance | Pre-IPO stock optionsSenior-level Full TimeUnited States - Remote R16h ago
-
A/B | A/B Testing | B testing | Cost Optimization | Data PipelinesEquity packages | Flexible leave | Parental leave | Remote within Australia | Wellbeing allowanceSenior-level Full TimeMelbourne, VIC, Australia R16h ago
-
Staff Data Platform Engineer USD 216K-270KApache Airflow | Apache Kafka | CI/CD | DBT | Data Governance401k | BetterUp coaching | Company holidays | Flexible paid time off | Learning and development stipendSenior-level Full TimeAll Hubs R18h ago