ETL Modernization Developer - 1645
Tasks
- Assess ETL jobs for migration readiness
- Build CI CD processes for ETL
- Conduct knowledge transfer and training
- Decommission legacy DataStage jobs and artifacts
- Develop test and deploy cloud data pipelines
- Perform data pipeline testing and validation
- Prepare technical documentation
- Reengineer IBM DataStage jobs with Databricks Spark on AWS
- Support migration cutover and production deployment
- Validate schemas and data quality
Perks/Benefits
Skills/Tech-stack
AWS | AWS Glue | AWS Lambda | Amazon Redshift | Apache Spark | Automated ETL | Automated ETL testing | Azure DevOps | CI/CD | Databricks | Databricks Photon | Delta Lake | ELT | ETL | ETL Testing | Functional testing | GitLab | IBM DataStage | JSON parsing | Performance Testing | PySpark | Pytest | Python | Regression testing | SQL | Shell Scripting | UAT | Unit Testing | Unity Catalog | XML Parsing
Education
N/A
Roles
Related jobs
-
Stručnjak/inja za Data Science i AI EUR 30K-65KAWS | Artificial Intelligence | Azure | Cloud Computing | Data AnalysisChildcare subsidy | Health insurance | Hybrid work | Meal allowance | MultiSport PassSport discountMid-level Full TimeZagreb, Hrvatska (Hybrid) R7h ago
-
Software Engineer Level 1-FFPP-8933 USD 78K-250KAWS Glue | AWS Lambda | Amazon DynamoDB | Amazon Web Services | BSON401k match | Bonus opportunities | Dental insurance | Employee referral program | Flexible spending accountsSenior-level Full TimeHanover, MD; Information Technology R8h ago
-
Software Engineer Level 2-FFPP-8934 USD 78K-250KAWS | Accumulo | BSON | HBase | HDFS401k match | Dental insurance | Employee referral programs | Flexible spending accounts | Flexible work arrangementsSenior-level Full TimeHanover, MD; Information Technology R8h ago
-
Senior-level Full TimeRemote (EST/PST/CT) R9h ago
-
Data Engineer Sr MXN 950K-1300KApache Airflow | Apache Hadoop | Apache Hive | Apache Impala | Apache NiFiAgile delivery | Remote workSenior-level Full TimeCiudad de México, CDMX, MX R9h ago
-
Data Engineer Lead MXN 950K-1176KAccess Management | Apache Airflow | Apache Hadoop | Apache Hive | Apache ImpalaRemote workSenior-level Full TimeCiudad de México, CDMX, MX R10h ago
-
Senior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL R10h ago
-
Senior Data Engineer (MS Fabric Experience) EUR 54K-69KAgile | Apache Spark | Azure | Azure Architecture | Azure DataCareer development plan | Certification sponsorship | Hybrid work | Relocation support | Remote up to 50 percentSenior-level Full TimePorto/Lisbon, Portugal R10h ago
-
AI Systems & ML Engineering Industry Expert MXN 960K-1300KA/B | A/B Testing | API Design | AWS | Agentic SystemsConference Speaking Visibility | Flexible schedule | Fully remote | Part time commitmentSenior-level Part TimeMexico City, Mexico City, MX R10h ago
-
AI Systems & ML Engineering Industry Expert USD 140K-209KA/B | A/B Testing | API Design | AWS | Agentic SystemsFlexible schedule | Fully remote | Hourly payment | International teamSenior-level Part TimeNew York City, NY, US R10h ago
-
AI Systems & ML Engineering Industry Expert GBP 92K-120KA/B | A/B Testing | API Design | AWS | Agentic SystemsFlexible scheduling | Fully remote | Low time commitmentSenior-level Part TimeLondon, Greater London, GB R10h ago
-
AI Systems & ML Engineering Industry Expert CAD 140K-185KA/B | A/B Testing | API Design | AWS | Agentic SystemsFlexible schedule | Fully remote | Hourly paySenior-level Part TimeToronto, Ontario, CA R10h ago
-
.NET | Access Control | Asynchronous processing | Authentication | AuthorizationRemote workSenior-level ContractRemote R10h ago
-
AI Security | API Development | Agent systems | Claude API | FastAPIRemote workSenior-level ContractRemote R10h ago
-
Cognos Analytics | DB2 | Data Mart | Data Modeling | Data QualityMid-level Full TimeAlbany, NY, United States R10h ago
-
AWS | Azure | Azure Data | Azure Data Factory | Azure DevOpsHybrid workSenior-level ContractCentral Toronto (Summerhill West / Rathnelly … R11h ago
-
API Design | AWS | Agent Orchestration | AgentCore) | Amazon Bedrock401k matching | Medical | Paid sick leaveSenior-level ContractRemote R11h ago
-
API Design | AWS | Agent Orchestration | AgentCore) | Amazon Bedrock401k | Medical | Paid sick leaveSenior-level ContractRemote R11h ago
-
Junior Data Engineer (m/f/d) - Martech EUR 40K-44KA/B | A/B Testing | AWS | Apache Spark | B testingBike leasing | Employee Discounts Partner Organizations | Employee discounts | Hybrid work model | MentorshipEntry-level Full TimeBerlin, Berlin, Deutschland R11h ago
-
Machine Learning Engineer USD 100K-120KAWS | Apache Spark | Classification | Data Quality | Data Quality ChecksHybrid work model | Onsite 1 to 2 days per week | Opportunities for growthMid-level Full TimeOakbrook Terrace, IL, US R12h ago
-
Senior-level Full TimeRa'anana, Center District, IL R13h ago
-
Data Engineer ZAR 464K-464KAgile | Apache Airflow | Apache Kafka | Apache NiFi | CI/CDCareer growth opportunity | Fringe benefits | Hybrid work opportunity | Learning and development opportunitiesMid-level Full TimeBaku, Azerbaijan R13h ago
-
Medior Data Pipeline Engineer HUF 7500K-9000KAI | Airflow | DB2 | DBT | Data MigrationHybrid work schedule | Learning and development opportunities | MentoringEntry-level Full TimeBudapest, Hungary R13h ago
-
Senior-level Full TimeBengaluru East, Karnataka, India R14h ago
-
Staff Data Engineer INR 3242K-4200KCloud Data | Cloud data platform | Continuous integration | Data Engineering | Data QualityFlexible work hours | Mentorship | Remote work optionsSenior-level Full TimeHyderabad, Telangana , India R14h ago