Enterprise Data Warehouse (EDW) ETL/Data Engineer
Tasks
- Build lakehouse ingestion and transformation workflows
- Create monitoring logging and auditing solutions
- Design and develop scalable data pipelines
- Design dimensional data models
- Develop PySpark notebooks and distributed jobs
- Implement CI/CD for data pipelines
- Implement data vault patterns and slowly changing dimensions
- Implement medallion architecture solutions
- Lead cloud data modernization and migration initiatives
- Optimize distributed SQL workloads and query performance
- Perform workload assessment capacity planning and cost optimization
- Support code reviews and release management
- Troubleshoot and respond to production incidents
Perks/Benefits
- N/A
Skills/Tech-stack
ARM Templates | Active Directory | Azure Active Directory | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure DevOps | Azure Key Vault | Azure Monitor | Azure SQL | Azure Synapse | Azure Synapse Analytics | Bash | Bicep | CD pipelines | CI/CD | CI/CD pipelines | Change Data Capture | Data Capture | Data Factory | Data Lake Storage Gen2 | Data Vault | Databricks Delta | Databricks Delta Lake | Dedicated SQL Pool | Delta Lake | Dimensional Modeling | Entra ID | Execution Plan | Execution Plan Analysis | Fabric Lakehouse | Indexing | Integration runtime | KQL | Key Vault | Kimball Method | Log Analytics | Managed Identity | Microsoft Fabric | Pandas | Parquet | Partitioning | Plan analysis | Powershell | PySpark | Pytest | Query Optimization | REST API | SQL | SQL Pool | SQL Pools | Serverless SQL | Serverless SQL Pools | Service Principal | Slowly Changing Dimensions | Spark Pools | Synapse Analytics | Synapse Dedicated SQL Pool | T-SQL | Teradata | Terraform | Unit Testing
Education
N/A
Related jobs
-
Data Engineer-Secret Clearance Required USD 100K-127KAWS | AWS Glue | AWS Redshift | Azure | Azure Data401k match | Bereavement leave | Disability insurance | Employee assistance program | Employee discount programSenior-level Full TimeRemote - Nationwide, United States R9h ago
-
Senior Software Engineer - San Francisco (Onsite) USD 130K-220KAWS | Amazon EMR | Amazon S3 | Apache Flink | Apache SparkFast-paced startup environment | Onsite work environment | Rapid hiring process feedback | Relocation supportSenior-level Full TimeSan Francisco, CA, US11h ago
-
Lead Data Engineer – Snowflake USD 170K-216KAmazon S3 | Apache Airflow | Azure Data | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeUnited States12h ago
-
IT Data Engineer USD 60K-65KANSI X12 | AWS | Azure | Azure Data | Azure Data Factory401k retirement plan | Disability coverage | Employee assistance program | Flexible spending account | Medical/Dental/Vision insuranceMid-level Full TimeUS - Remote R13h ago
-
Data Engineer III USD 134K-161KAccess Control | Analytics Cloud | Data Governance | Data Marts | Data Quality401k | Adoption Assistance | Dental insurance | Education reimbursement | FSASenior-level Full TimeIrvine, CA13h ago
-
GenAI Engineer III USD 110K-218KArtificial Intelligence | Containerization | Data Analysis | Data Pipelines | DockerProfessional developmentSenior-level Full TimeArlington/Rosslyn, Virginia, United States13h ago
-
Generative AI Engineer III USD 110K-218KArtificial Intelligence | Data Analysis | Data Pipelines | Docker | KubernetesDiscretionary annual incentive program | Mentorship | Professional developmentSenior-level Full TimeAustin, Texas, United States; Boston, Massachusetts, …13h ago
-
Lead Generative AI Data Engineer III USD 159K-265KArtificial Intelligence | Data Pipelines | Docker | Generative AI | KubernetesSenior-level Full TimeAustin, Texas, United States; Boston, Massachusetts, …13h ago
-
Lead Generative AI Data Engineer III USD 159K-265KArtificial Intelligence | Data Pipelines | Generative AI | Language Models | Language ProcessingSenior-level Full TimeAtlanta, Georgia, United States; New York, …13h ago
-
Generative AI Data Engineer III USD 131K-218KAlgorithm Development | Artificial Intelligence | Data Analysis | Data Modeling | Data PipelinesMentorship | Travel opportunitiesSenior-level Full TimeAtlanta, Georgia, United States; New York, …13h ago
-
Integration Engineer II - AI and Python Developer USD 86K-170KAI Ops | AWS Bedrock | Azure | CI/CD | Fine TuningAgile team collaboration | Mentorship | Professional developmentMid-level Full TimeAtlanta, Georgia, United States; Baltimore, Maryland, …13h ago
-
Sr. AI Engineer USD 150K-175KAccess Control | Agentic AI | Auditability | CI/CD | Cloud platform401k | Dental insurance | Expense reimbursement for internet costs | Life insurance | Medical insuranceSenior-level Full TimeRemote, USA, United States R13h ago
-
Senior Data Engineer, Strategic Partnerships & IMPACT360 USD 154K-209KAWS Glue | AWS Lake Formation | AWS Lambda | Amazon Athena | Apache AirflowSenior-level Full TimeSeattle, Washington, USA1d ago
-
AI Engineer, Global Operations USD 107K-143KArtificial Intelligence | ETL | Langchain | Machine Learning | PythonContinuous learning | Flexible work options | Health, wellness, and retirement plans | Professional growth opportunitiesMid-level Full TimeCalifornia - Remote, United States R1d ago
-
Senior Cloud Database Engineer USD 107K-185KAWS | Amazon RDS | CI/CD | Capacity Planning | CassandraSenior-level Full TimeTwo Destiny Way, Westlake TX, United …1d ago
-
Sr. Data Engineer USD 130K-203KData Modeling | Data Pipelines | Databases | ETL | HDFS401k retirement savings plan | CVS store discount | Discount programs | Education assistance | Employee stock purchase planSenior-level Full TimeWoonsocket-1 CVS Drive, United States1d ago
-
AWS | Access Control | Apache Spark | Azure | CI/CD401k retirement plan | Comprehensive benefits | Dental insurance | Employee assistance program | Medical insuranceSenior-level Full TimeUSA, TX, Work-at-Home, United States R1d ago
-
Senior-level Full TimeMcLean, VA, United States1d ago
-
Lead Data Engineer USD 179K-225KAmazon Redshift | Amazon Web Services | Cassandra | Cloud platform | Data StreamingHealth benefits | Performance bonusSenior-level Full TimeMcLean, VA, United States1d ago
-
Associate Director, Data Science USD 160K-297KADME | Bioinformatics | Data Analysis | Data Governance | Data Mining401k match | Hybrid work | Paid time offMid-level Full TimeCambridge (USA), United States R1d ago
-
Sales Data & Analytics Product Manager USD 131K-190KAmazon Web Services | Analytics reporting | Apache Airflow | Apache Kafka | Apache Spark401k | Bonus | Dental coverage | Holidays | Medical coverageMid-level Full TimeUS, MA, Wilmington, United States1d ago
-
ML/ML Ops Principal Engineer USD 139K-190KAWS | BigQuery | Classification | Cloud Computing | Cloud platformSenior-level Full TimeGA250 - Atlanta GA, United States1d ago
-
Staff Software Embedded Engineer USD 169K-253KAgile | Bash | C++ | Debugging | Design PatternsCompany-Paid Holidays | Disability insurance | Health insurance | Life insurance | On-site workSenior-level Full TimeILRM03, United States1d ago
-
Senior AI Engineer – Agentic AI & Automation - VP USD 170K-230KBPMN | Camunda | Containers | Context Management | Embeddings401k | Incentive award | Life insurance | Medical, dental, and vision coverage | Paid leaveSenior-level Full TimeBuilding 400-Whippany Campus, Jefferson Park, United …1d ago
-
Data Engineer USD 150K-190KAccess Control | Alerting | Audit Logging | Backfill | BigQueryDental insurance | Long-term disability | Medical insurance | Paid time off | Short-term disabilitySenior-level Full TimeColumbus, OH1d ago