Enterprise Data Warehouse (EDW) ETL/Data Engineer
Tasks
- Build lakehouse ingestion and transformation workflows
- Create monitoring logging and auditing solutions
- Design and develop scalable data pipelines
- Design dimensional data models
- Develop PySpark notebooks and distributed jobs
- Implement CI/CD for data pipelines
- Implement data vault patterns and slowly changing dimensions
- Implement medallion architecture solutions
- Lead cloud data modernization and migration initiatives
- Optimize distributed SQL workloads and query performance
- Perform workload assessment capacity planning and cost optimization
- Support code reviews and release management
- Troubleshoot and respond to production incidents
Perks/Benefits
- N/A
Skills/Tech-stack
ARM Templates | Active Directory | Azure Active Directory | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure DevOps | Azure Key Vault | Azure Monitor | Azure SQL | Azure Synapse | Azure Synapse Analytics | Bash | Bicep | CD pipelines | CI/CD | CI/CD pipelines | Change Data Capture | Data Capture | Data Factory | Data Lake Storage Gen2 | Data Vault | Databricks Delta | Databricks Delta Lake | Dedicated SQL Pool | Delta Lake | Dimensional Modeling | Entra ID | Execution Plan | Execution Plan Analysis | Fabric Lakehouse | Indexing | Integration runtime | KQL | Key Vault | Kimball Method | Log Analytics | Managed Identity | Microsoft Fabric | Pandas | Parquet | Partitioning | Plan analysis | Powershell | PySpark | Pytest | Query Optimization | REST API | SQL | SQL Pool | SQL Pools | Serverless SQL | Serverless SQL Pools | Service Principal | Slowly Changing Dimensions | Spark Pools | Synapse Analytics | Synapse Dedicated SQL Pool | T-SQL | Teradata | Terraform | Unit Testing
Education
N/A
Related jobs
-
ETL Developer - US Healthcare Experience USD 90K-130KAWS Glue | Apache Spark | Azure Data | Azure Data Factory | Azure SQLMid-level Full TimeChantilly, United States3h ago
-
Apex | Azure DevOps | CI/CD | Claude Code | Data cloudSenior-level Full TimeArlington/Rosslyn, Virginia, United States3h ago
-
AI and Data Science Engineer (TS/SCI Poly) USD 93K-155KAPI | Advanced Analytics | Analytical products | Artificial Intelligence | CI/CDProfessional development | Travel opportunitiesMid-level Full TimeMcLean, Virginia, United States3h ago
-
Acceptance Testing | Data Visualization | Data analytics | Microsoft Office | Relational databasesEntry-level Full TimeArlington/Rosslyn, Virginia, United States3h ago
-
Cloud Engineer - Project Delivery Specialist USD 97K-179KAWS | AWS Fault Injection Simulator | Amazon EKS | Ansible | BashLimited travel | On-call rotationMid-level Full TimeDallas, Texas, United States3h ago
-
Asset & Wealth Management- New York- Associate, Quantitative Internal Product Specialists- 9649762 USD 191K-191KC# | Component analysis | Derivatives | Excel | Hypothesis TestingMentorship | Travel reimbursementEntry-level Full TimeNew York, NY, United States4h ago
-
Cloud Computing | Data Processing | Data Storage | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA5h ago
-
Senior Software Engineer, Managed Spark, Open Source USD 174K-252KApache Hadoop | Apache Hive | Apache Spark | Data Analysis | DebuggingSenior-level Full TimeSunnyvale, CA, USA5h ago
-
Software Engineer III, Knowledge Catalog For AI USD 147K-211KArtificial Intelligence | BigQuery | C++ | Cloud Computing | Data CatalogSenior-level Full TimeSunnyvale, CA, USA5h ago
-
Data and Analytics - AI Engineer II USD 100K-148KAI Foundry | AIOps | Azure | Azure AI | C#Continuous learning | Hybrid work environment | Professional growth | Supportive collaborative environmentMid-level Full TimeOmaha, NE, 68102, USA8h ago
-
Bash | Buildkite | C++ | CI/CD | Catch2401k | Dental insurance | Equity program | Fully stocked kitchen | Green card supportSenior-level Full TimeMilpitas, CA10h ago
-
Senior-level Full TimeAustin, Texas15h ago
-
Principal Data Engineer USD 246K-311KApache Airflow | Apache Kafka | Backwards Compatibility | CDC | Change Data Capture401k match | Flexible work from home or in office | Health benefits | Learning and development budget | Mentorship and coachingSenior-level Full TimeUSA15h ago
-
Principal Process Development Engineer - Data Analytics USD 164K-273KANOVA | Capability analysis | Cause analysis | Control Charts | Design of ExperimentsCareer development | Comprehensive benefits | Tuition reimbursementSenior-level Full TimeSan Diego, United States16h ago
-
Manager, Data Engineer (Remote) USD 100K-174KApache Spark | Azure | CI/CD | Data Modeling | Data Quality401k matching | Backup child and elder care | Dental insurance | Health insurance | Life insuranceMid-level Full TimeRaleigh, NC, United States R16h ago
-
AWS | Apache Airflow | Apache Spark | Azure | CI/CDContinuing education | Flexible time off | Healthcare | Learning resources | Retirement benefitsSenior-level Full Time0T5 STERLING VA, United States16h ago
-
Principal Data/AI Engineering USD 175K-261KAWS | Artificial Intelligence | Azure | Data Tuning | Database Administration401k plan | Accidental death insurance | Adoption reimbursement | Employee assistance program | Employee discountsSenior-level Full TimeUSA:TX:Dallas / Four AT&T Plaza (311 …16h ago
-
Senior Analytics Engineer, Go-To-Market Data USD 138K-225KAgent Frameworks | Airflow | Analytics engineering | DBT | Data ArchitectureHybrid work | MentorshipSenior-level Full TimeMountain View, CA, United States16h ago
-
Forward Deployed AI Engineer (Post-Sales) USD 230K-300KAWS | Azure | Compute Orchestration | Configuration Management | Data Pipelines401k company match | Daily meals and snacks | Health benefits | Learning & development stipend | Relocation assistanceMid-level Full TimeRedwood City17h ago
-
Sr. Data Governance Engineer USD 149K-205KAsset Management | CCPA | Data Asset | Data Asset Management | Data Catalog401k match | Disability insurance | Employee assistance program | Generous paid time off | Health insuranceSenior-level Full TimeUtah | Hybrid R17h ago
-
AI Dev (Features + CHQ)_Tempe,AZ USD 56K-196KAPI Integration | AWS | Agile | Angular | Artificial Intelligence401k retirement plan | Dental benefits | Independent contractor | Medical benefits | Paid HolidaysMid-level Full TimeUnited States18h ago
-
FPGA Embedded Software Engineer USD 122K-147KARM | Bare Metal | Bare metal firmware | Bash | BitLockerMid-level Full TimeDuluth, GA, US18h ago
-
AI Intern, Data Engineering & Agent Workflows USD 62K-90KAirflow | BigQuery | Claude API | DBT | DatabricksMentorship | Professional development | Remote workEntry-level InternshipUnited States (Remote) R19h ago
-
Software Engineer, Data Migration USD 109K-215KAlgorithms | Apache Kafka | Change Data Capture | Data Capture | Data Modeling401k plan | Fertility and adoption assistance | Flexible paid time off | Health benefits | Mental health counselingSenior-level Full TimeCalifornia; Oregon; Washington19h ago
-
Senior AI Engineer, Internal AI Platforms USD 86K-120KAWS | CI/CD | Data Governance | DevSecOps | EmbeddingsEntry-level Full TimeLos Angeles, CA19h ago