Enterprise Data Warehouse (EDW) ETL/Data Engineer
Tasks
- Build lakehouse ingestion and transformation workflows
- Create monitoring logging and auditing solutions
- Design and develop scalable data pipelines
- Design dimensional data models
- Develop PySpark notebooks and distributed jobs
- Implement CI/CD for data pipelines
- Implement data vault patterns and slowly changing dimensions
- Implement medallion architecture solutions
- Lead cloud data modernization and migration initiatives
- Optimize distributed SQL workloads and query performance
- Perform workload assessment capacity planning and cost optimization
- Support code reviews and release management
- Troubleshoot and respond to production incidents
Perks/Benefits
- N/A
Skills/Tech-stack
ARM Templates | Active Directory | Azure Active Directory | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure DevOps | Azure Key Vault | Azure Monitor | Azure SQL | Azure Synapse | Azure Synapse Analytics | Bash | Bicep | CD pipelines | CI/CD | CI/CD pipelines | Change Data Capture | Data Capture | Data Factory | Data Lake Storage Gen2 | Data Vault | Databricks Delta | Databricks Delta Lake | Dedicated SQL Pool | Delta Lake | Dimensional Modeling | Entra ID | Execution Plan | Execution Plan Analysis | Fabric Lakehouse | Indexing | Integration runtime | KQL | Key Vault | Kimball Method | Log Analytics | Managed Identity | Microsoft Fabric | Pandas | Parquet | Partitioning | Plan analysis | Powershell | PySpark | Pytest | Query Optimization | REST API | SQL | SQL Pool | SQL Pools | Serverless SQL | Serverless SQL Pools | Service Principal | Slowly Changing Dimensions | Spark Pools | Synapse Analytics | Synapse Dedicated SQL Pool | T-SQL | Teradata | Terraform | Unit Testing
Education
N/A
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R3d ago
-
Senior-level Full TimeUS - Remote; Canada - Remote R8h ago
-
Senior-level Full TimeMountain View, CA10h ago
-
Senior Software Engineer Remote/Travel- Active Secret -Top Secret clearance(SCI-eligibility) USD 145K-204KAPI Testing | Cypher | Data Processing | DataOps | DevOpsCompetitive benefits | Growth opportunity | Remote work | TravelSenior-level Full TimeReston, VA, United States R20h ago
-
AWS | AWS CloudFormation | Amazon ECS | Automation | CI/CD401-k plan | Company match | Employee stock purchase plan | Healthcare benefits | Life insuranceMid-level Full TimeAtlanta, GA, United States21h ago
-
Software Engineer - X Data USD 125K-400KApache Flink | Apache Kafka | Apache Spark | BigQuery | Cause analysis401k retirement plan | Dental insurance | Discounts | Health insurance | Life insuranceMid-level Full TimePalo Alto, CA1d ago
-
AI Software Engineer USD 80K-210KAWS Kinesis | Apache Airflow | Apache Kafka | C# | C++401k | Equity grant | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
Applied AI Engineer USD 204K-352KBackend Development | Cloud infrastructure | Deep learning | Docker | Fine TuningAnnual membership | Centralized coffee stipend | Flexible PTO | Health and wellness stipend | Health, dental, and vision insuranceSenior-level Full TimeNew York, NY (HQ)1d ago
-
Staff AI Engineer USD 140K-210KAPI Development | AWS | Access Management | Bedrock AgentCore | CI/CD401k | All-hands meetings | Dental insurance | Employee equity program | Health insuranceSenior-level Full TimeNew York City1d ago
-
Senior Quantum Embedded Engineer USD 142K-175K10G Ethernet | AMD Xilinx | AMD Xilinx toolchain | AMD/Xilinx SoC | BashHybrid work options | Remote work optionsSenior-level Full TimeNew Haven, CT1d ago
-
Senior Quantum Applications Engineer - QEC USD 120K-258KCUDA-Q | Decoder algorithms | Docker | End to End | End-to-End TestingSenior-level Full TimeNew Haven, CT1d ago
-
Senior AI Engineer USD 125K-145KAPI Management | AWS | Azure | Azure DevOps | C#401k plan | Career advancement | Dental insurance | Dependent care | Flexible spending accountSenior-level Full TimeCleveland, OH, United States1d ago
-
Senior Data Solutions Lead USD 175K-200KAWS | Agile | Apache Airflow | Apache Kafka | Apache SparkDHS Public Trust suitability determination | Remote workSenior-level Full TimeWork from home, VA, United States R1d ago
-
Senior Analyst, Business Strategy and Analytics USD 95K-135KData Governance | Data Integrity | Data Mining | Data Validation | Microsoft Access401k match | Education assistance | Employee discounts | Family & fertility benefits | Free virtual health careSenior-level Full TimeColumbus, OH, United States1d ago
-
Big Data Engineer / Sr. Level - GCP USD 88K-174KAPI Development | API Versioning | Apigee | Avro | Big DataContract-to-hireSenior-level Full TimeNashville, TN, United States1d ago
-
Senior Python AI/ML Developer USD 140K-170KAsynchronous programming | CI/CD | Django | DynamoDB | FastAPISenior-level Full TimeUS-NJ-Jersey City1d ago
-
Mid-level Full TimeUS-NJ-Jersey City1d ago
-
C plus plus | CI/CD | Cloud Computing | Computer Vision | Deep learningSenior-level Full TimeBurlingame, CA2d ago
-
Senior Staff Software Engineer, Knowledge Catalog For AI USD 262K-365KBig Data | C++ | Computer Vision | Data Governance | Data ProcessingSenior-level Full TimeSunnyvale, CA, USA2d ago
-
Asynchronous programming | BigQuery | C++ | Concurrency | Data ProcessingSenior-level Full TimeKirkland, WA, USA2d ago
-
Principal Data Engineer USD 180K-220KAWS | Apache Airflow | Artificial Intelligence | CI/CD | Confluence401k | Autonomy | Company paid life insurance | Disability coverage | FSASenior-level Full TimeRemote - US, Canada and Poland R2d ago
-
Senior Data Engineer, Public Sector USD 150K-259KBusiness Intelligence | DBT | Data Mart | Data Modeling | Data PipelinesCommuter stipend | Generous PTO | Health, dental and vision coverage | Learning and development stipend | Retirement benefitsSenior-level Full TimeWashington, DC2d ago
-
Data Engineer USD 132K-165KAWS | Azure | Cassandra | Data Governance | Data Lakes401k match | Dental insurance | Employee assistance program | Life insurance | Medical insuranceSenior-level Full TimePlano, TX, US2d ago
-
Data Platform Engineer, Infrastructure USD 70K-300KAWS | Alerting | Audit Logging | CI/CD | Capacity PlanningFlexible hours | Onsite work | Work-life balanceMid-level Full TimeIrvine, CA2d ago
-
AWS | Azure | CI/CD | Cloud Native | Distributed SystemsCareer development support | Flexible working models | Health and wellbeing benefits | Hybrid work | MentoringEntry-level InternshipSan Ramon, CA, US, 945 832d ago