Databricks Migration Engineer (Remote)
Rosslyn, VA, United States
R
USD 115K-171K (estimate) Mid-level Full Time
Tasks
- Align Lakehouse RBAC with Entra ID
- Build ETL and ELT pipelines
- Conduct code reviews and workshops
- Configure liquid clustering and materialized views
- Create technical documentation and design playbooks
- Define governance for cluster sizing and autoscaling
- Deliver knowledge transfer and training
- Design Bronze Silver Gold Lakehouse layers
- Design Delta Lake partitioning and file size management
- Develop dimensional models and star schemas
- Enforce row level and column level security
- Implement Unity Catalog security model
- Implement Z ordering and data skipping
- Implement monitoring dashboards for DBU cost
- Lead legacy data migration to Databricks Lakehouse
- Optimize Databricks SQL Warehouses for Power BI
Perks/Benefits
Skills/Tech-stack
Active Directory | Apache Spark | Azure Active Directory | Azure Data | Azure Data Factory | Data Factory | Data Partitioning | Data Quality | Data Skipping | Data loading | Databricks | Databricks SQL | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | Dimensional Modeling | DirectQuery | Distributed Computing | ELT | ETL | Entra ID | File Size Management | Import | Incremental data loading | Liquid clustering | Materialized views | Metadata Driven | Metadata Driven Ingestion | Microsoft Entra | Microsoft Entra ID | Power BI | PySpark | RBAC | SQL Server | Star Schema | Unity Catalog | Z-ordering
Education
Bachelor of Engineering | Bachelor of Science | Master of Science | PhD
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R23d ago
-
Data Engineer USD 91K-114KANSI SQL | API | Amazon Redshift | Apigee | Azure401k with company match | Bereavement leave | Employee assistance program | Employee discount program | Family planning benefitsMid-level Full TimeRemote - Nationwide, United States R20h ago
-
Senior Data Engineer USD 111K-139KAPI Integration | Amazon Redshift | Apigee | Azure | BI Publisher401k match | Bereavement leave | Dental insurance | Employee assistance program | Employee discountsSenior-level Full TimeRemote - Nationwide, United States R20h ago
-
AI & Machine Learning Vice President USD 120K-220KAWS | Alteryx | Anomaly Detection | Azure | C#401k | Dental insurance | Disability insurance | Health insurance | Hybrid work modelExecutive-level Full TimeUnited States R1d ago
-
Director, Data Analyst/BI Engineer USD 138K-175KAWS | Access Control | Aggregated Event Measurement | Airflow | BigQuery401k match | Healthcare Dental Vision | PTO | Paid Holidays | Parental leaveExecutive-level Full TimeUnited States R1d ago
-
Data Engineer USD 160K-200KAWS | Apache Spark | Azure | CI/CD | DBT401k match | Continuing education support | Employee assistance program | Flexible spending accounts | Generous paid time offSenior-level Full TimeAuburn Hills, MI, US R1d ago
-
Senior Data Engineer II, AI Native USD 148K-216KAmazon Web Services | Apache Airflow | Apache Spark | Azure | BigQuery401k matching | Dental insurance | Disability insurance | Employee assistance program | Equipment reimbursementSenior-level Full TimeRemote, USA R1d ago
-
API Design | AWS | Apache Airflow | Apache Flink | Apache KafkaDental insurance | Employee discounts | Employee equity | Health insurance | Pet insuranceSenior-level Full TimeRemote - United States R1d ago
-
Senior Data Engineer USD 115K-175KAWS Fargate | AWS Glue | AWS IAM | AWS Lambda | AWS MWAACharitable giving program | Company-Paid Holidays | Dental insurance | Flexible PTO | Life insuranceSenior-level Full TimeMassachusetts - Boston R1d ago
-
Senior Analytics Engineer USD 148K-218KAutomation | BI tools | CCPA | CI/CD | DBTDental insurance | Equity | Medical insurance | Remote work | Vision insuranceSenior-level Full TimeRemote, USA ; Remote, Canada R1d ago
-
Amazon Web Services | Apache Airflow | Apache Flink | Apache Hive | Apache HudiContinuing education | Dental insurance | FSA | HSA | Health insuranceSenior-level Full TimeRochester, MN, United States R1d ago
-
Apache Airflow | Apache Hudi | Apache Iceberg | Apache Kafka | Apache Kafka StreamsContinuing education | Dental insurance | Flexible spending account | Health savings account | Medical insuranceSenior-level Full TimeRochester, MN, United States R1d ago
-
Cleared Hybrid Data Engineer (5418) USD 103K-171KAPI strategy | AWS | AWS Glue | AWS Redshift | Access ControlHealth insurance | Paid leave | RetirementSenior-level Full TimeHanover, MD R1d ago
-
AWS | Agile | Apache Hive | Apache Spark | Cloudera Data PlatformRemote workSenior-level ContractLincoln, United States R2d ago
-
Middle Data Engineer (Azure Databricks) USD 160K-223KApache Spark | Azure Data | Azure Data Factory | Azure Databricks | Azure DevOpsCertification programs | Health insurance | Mentorship | Professional development opportunities | Relocation programMid-level Full TimeMadrid, MD, Spain R2d ago
-
Senior Data Engineer, Platform USD 116K-146KAWS | AWS DMS | Airflow | CI/CD | CloudFormationBonus | Equity | Health benefitsSenior-level Full TimeRemote - US, United States R2d ago
-
AI Engineer (Data Intelligence Analyst) - Remote USD 110K-160KAnomaly Detection | Automation | Azure | Azure Fabric | Data GovernanceRemote workMid-level Full TimeLocation Los Angeles, California, United States R2d ago
-
Architect - Data Engineer USD 165K-216KAPI Design | Access Control | Batch Processing | Change Data Capture | CubeSenior-level Full TimeUSA - Remote, United States R2d ago
-
Snowflake Data Engineer USD 80K-115KAzure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage401k retirement plan | Dental insurance | Employee stock purchase plan | Learning opportunities | Long-term disabilityMid-level Full TimeRemote - USA, United States R2d ago
-
Snowflake Data Engineer USD 80K-115KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure SQL401k retirement plan | Employee stock purchase plan | Long-term disability | Medical, dental, vision plans | Paid Holidays & PTOMid-level Full TimeRemote - USA, United States R2d ago
-
Data Governance & Quality Engineer USD 124K-194KBusiness glossary | Cause analysis | Data Catalog | Data Governance | Data LineageRemote work optionMid-level Full TimeUS-DE-Wilmington, United States R2d ago
-
Senior-level Full TimeUnited States - Remote R2d ago
-
ETL Developer USD 115K-175KAWS | AWS Glue | Apache Airflow | Auditability | AutosysCareer growth potential | Remote workSenior-level Full TimeUnited States - Remote R2d ago
-
Senior Analytics Engineer USD 157K-185KAirflow | Anonymization | BigQuery | Compliance | DBT401k match | Cellphone stipend | Commuter benefits | Company-Paid Holidays | Family planning benefitsSenior-level Full TimeRemote U.S. R2d ago
-
Senior Software Engineer, Data Engineering USD 243K-295KAWS | Airflow | Azure | Batch Processing | CI/CDSenior-level Full TimeSan Mateo, CA, United States R2d ago