Data Engineer
Tasks
- Apply data modeling and schema evolution
- Automate deployment scripts with Python
- Collaborate on source and downstream data requirements
- Design data ingestion pipelines
- Develop and maintain Kafka Connect connectors
- Implement change data capture with Debezium
- Ingest MySQL and PostgreSQL data into data lake
- Load data into AWS S3 GCS and BigQuery
- Monitor and optimize Kafka Connect clusters
- Optimize ingestion performance and cost
- Partition data in bronze layer
- Stay current with Apache Hudi and Apache Iceberg
Perks/Benefits
- N/A
Skills/Tech-stack
AWS CloudFormation | Amazon S3 | Apache Airflow | Apache Hudi | Apache Iceberg | Apache Kafka | Cloud Storage | Dagster | Data Modeling | Data Partitioning | Data Quality | Debezium | Docker | ELT | ETL | Google BigQuery | Google Cloud | Google Cloud Storage | Kafka Connect | Kubernetes | MySQL | Observability | PostgreSQL | Python | Schema evolution | Terraform
Education
N/A
Roles
Related jobs
-
Featured Feat. Software Engineer USD 40K-150K.NET | APIs | AWS | Angular | AzureContractor hours flexibility | Remote workMid-level ContractRemote R10d ago
-
Featured Feat. Senior Software Engineer USD 80K-170KAlgorithms | C++ | Data Structures | Go | JavaContractor flexibility | Remote workSenior-level ContractRemote R10d ago
-
AWS Bedrock | Agent systems | Anthropic API | Autogen | Azure401k matching program | Adoption Assistance | Development and career growth opportunities | Fertility treatments | Flexible work schedulesSenior-level Contract Full TimeRemote, OR, United States R10h ago
-
Data Engineer USD 74K-133KAgile | Apache Airflow | BigQuery | Cloud Composer | Cloud Data401k retirement plan | Dental insurance | Disability insurance | Flexible time off | Health insuranceMid-level Full TimeLisle, IL, United States R12h ago
-
AWS | Airflow | DBT | Fine Tuning | Language ModelsBonuses | Disability insurance | Life insurance | Paid parental leave | Paid time offSenior-level Full TimeRemote, India R13h ago
-
API Testing | Cypher | Data Quality | DataOps | DevOpsBenefits | Competitive pay | Growth opportunity | Remote work | Travel requiredSenior-level Full TimeReston, VA, United States R14h ago
-
Principal Engineer - Data Platform USD 221K-387KAWS | Airflow | Apache Hive | Apache Iceberg | Apache ImpalaRemote workSenior-level Full TimeSanta Clara, California, United States R14h ago
-
AI Engineer H/F - CDI EUR 50K-65KAI Agents | Agent systems | Cloud Computing | Deep learning | Fine TuningCooptation bonus | Equipment bonus | Flexible remote work | Health insurance | Meal vouchersMid-level Full TimeParis, IDF, France R16h ago
-
AWS | Azure | Data Governance | Data Marts | Data ModelingCSE | Career development opportunities | Cooptation bonus | Diversity initiatives | Employee representative councilSenior-level Full TimeAix-en-Provence, Provence-Alpes-Côte d'Azur, France R16h ago
-
Data Engineer Sr. BRL 235K-396KAmazon Redshift | Data Modeling | Data Pipelines | Data Storytelling | Data VisualizationRemote work | Work-life balance flexibilitySenior-level Full TimeRemote, Brazil R17h ago
-
Airbyte | Cognos | DBT | Data Security | ELTAfterworks | Flexible work schedule | Health insurance | Long-term assignments | Mobility supportMid-level Full TimeAix-en-Provence, France R17h ago
-
Senior DBA - PostgreSQL, RDS & DevOps Tooling USD 125K-180KAWS Aurora | AWS Aurora PostgreSQL | AWS CloudWatch | AWS CloudWatch Insights | AWS RDSFlexible off hours maintenance availability | Remote workSenior-level Full TimeRemote R17h ago
-
Senior Database Engineer - #3190 (Remote) INR 1500K-2700KAWS IAM | AWS KMS | AWS RDS | Amazon Aurora | BashOn-call rotation | Remote workSenior-level Full TimeChennai, India R18h ago
-
A/B | A/B Testing | AI Search | Azure | Azure AIFlexible working hours | Modern office | Multisport card | Private medical care | Unlimited access to Microsoft technologyMid-level ContractRemote R18h ago
-
Software Engineer SRE (Site Reliability Engineer) INR 1600K-3000KAWS | Ansible | Apache Airflow | Apache Kafka | Apache SparkSenior-level Full TimeBangalore, India Office (BANGALORE) R18h ago
-
API Development | AWS | Amazon SageMaker | CI/CD | Cloud platformAnnual leave | Employee referral program | HMO coverage | Night differential pay | Remote workSenior-level Full TimeRemote R18h ago
-
Anthropic API | AutoGluon | CUDA | CatBoost | Cloud platformRemote workMid-level Full TimeRemote R19h ago
-
GCP BigQuery,DevOps,Credit Card Domain/Associate Director, Data and Analytics Specialist INR 650K-900KApache Airflow | Apache Kafka | BigQuery | CI/CD | Cloud ComposerMid-level Full TimeHyderabad, Telangana, India R20h ago
-
Senior Staff Engineer, Data Migration Lead INR 3000K-4000KAccess Control | Audit Trail | Cutover planning | Data Governance | Data ManagementSenior-level Full TimeRemote, India R22h ago
-
AWS | CI/CD | Dataiku | GitLab CI | HDFSEmployee representative council | Health insurance | Meal vouchers | Profit sharing | Referral bonusSenior-level Full TimeVilleneuve-d'Ascq, Hauts-de-France, France R23h ago
-
AWS | Apache Hadoop | Apache Kafka | Apache Spark | AzureCareer Development Programs | Cooptation bonus | Employee representative council | Health insurance | Holiday bonusesSenior-level Full TimeVilleneuve-d'Ascq, Hauts-de-France, France R23h ago
-
Data Scientist confirmé / AI Engineer EUR 50K-55KAzure | CI/CD | Docker | Docker Compose | GCPHealth insurance | Telework | Ticket restaurant | Works CouncilMid-level Full TimeCourbevoie, IDF, France R23h ago
-
Python, GenAI, LLM/Consultant Specialist INR 3700K-5000KAccess Management | Active Directory | Active Directory Domain Services | Agent Frameworks | Amazon Web ServicesFlexible working | Professional developmentMid-level Full TimePune, Maharashtra, India R1d ago
-
AWS Glue | AWS Lambda | AWS S3 | Access Control | Data GovernanceCareer growth opportunities | Collaborative and inclusive work environment | Diverse and inclusive culture | Flexible work arrangements | Permanent remote working modelSenior-level Full TimeCanada R1d ago
-
AI Agents | Artificial Intelligence | Backend Development | CI/CD | Cloud ComputingCoworking access | Flexible work arrangement | Fully remote | Healthcare coverage | Home-office equipmentMid-level Full TimeItaly R1d ago