Senior Databricks Migration Engineer
Tasks
- Architect dimensional models and star schemas for Power BI performance
- Build monitoring dashboards for DBU consumption and cost optimization
- Create technical documentation and optimization playbooks
- Define governance standards for clusters autoscaling and serverless compute
- Design Lakehouse layers Bronze Silver Gold
- Develop ETL ELT frameworks with PySpark Delta Live Tables
- Develop knowledge transfer framework for post migration self sufficiency
- Enforce row level security and column level security
- Implement data security model with Unity Catalog
- Implement optimization techniques Z ordering data skipping liquid clustering materialized views
- Implement partition strategy and Delta Lake file size management
- Integrate Lakehouse security with Azure Active Directory Entra ID RBAC
- Lead data migration from SQL systems to Databricks Lakehouse
- Optimize Databricks SQL Warehouses for Power BI DirectQuery and Import
- Provide technical leadership through code reviews and workshops
Perks/Benefits
- N/A
Skills/Tech-stack
Active Directory | Azure Active Directory | Column-Level Security | Data Quality | Data Skipping | Databricks | Databricks SQL | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | Dimensional Modeling | ELT | ETL | Entra ID | Incremental loads | Liquid clustering | Materialized views | Microsoft Entra | Microsoft Entra ID | Power BI | PySpark | RBAC | Row Level Security | SQL Server | Serverless SQL | Star Schema | Unity Catalog | Z-Order
Education
Bachelor of Engineering | Bachelor of Science | Master of Science | PhD
Related jobs
-
AWS | Automated testing | Big Data | Cloud Native | Code reviewSenior-level Full TimePlano, TX, United States1h ago
-
Data Engineer Intern (E-commerce) - 2027 Summer USD 80K-90KApache Spark | ClickHouse | Data Modeling | ETL | FlinkDevelopment workshops | Industry exposure | Learning programs | Social eventsEntry-level InternshipSan Jose, California, United States2h ago
-
Senior data engineer (Python) con inglés USD 57K-73KAgile | Alerting | Azure | CI/CD | Clean CodeCertifications | Online training platformSenior-level Full TimeMadrid, MD, Spain2h ago
-
Data Engineer, Google Maps USD 106K-150KApache Spark | ArcGIS | Business Intelligence | Cloud platform | Data AnalysisMid-level Full TimeMountain View, CA, USA3h ago
-
Machine Learning Co-Op (Fall 2026) USD 87K-124KDAX | Data Quality | Data Visualization | Deep learning | ForecastingNone Full TimeCanton, OH, United States3h ago
-
Sr. Data Engineer - AI USD 99K-139KApache Airflow | Azure Data | Azure Data Factory | CI/CD | Data FactorySenior-level Full TimeKansas City, KS, United States3h ago
-
Data Platform Engineer-Solutions Delivery USD 119K-160KAPI Development | API Management | Azure API | Azure API Management | Azure Functions401k match | Company holidays | Dental insurance | Employee discount | Medical insuranceSenior-level Full TimeMerriam, KS, United States4h ago
-
Senior Data Engineer USD 120K-175K.Net Core | .Net Framework | AWS CloudFormation | Apache Airflow | Azure401k match | Career growth opportunities | Dental insurance | Employee resource groups | Life insuranceSenior-level Full TimeRemote, United States4h ago
-
Mid-Level Data Engineer USD 90K-115KAPI Development | Azure | Azure Data | Azure Data Factory | Azure Data LakePublic trust clearance support | Remote workMid-level Full TimeWork from home, VA, United States4h ago
-
Data Engineer USD 145K-160KAWS Glue | AWS IAM | Apache Airflow | Apache Kafka | Apache SparkHybrid remote work | Public Trust eligibility requiredMid-level Full TimeFAIRFAX, VA, United States4h ago
-
Data Engineer USD 110K-165KAmazon Web Services | Artificial Intelligence | Computer Vision | Data Pipelines | Data ProcessingHealth benefits | Hybrid work | Onsite at customer location | Paid time off | Retirement benefitsSenior-level Full TimeFAIRFAX, VA, United States4h ago
-
Senior Databricks Platform Engineer USD 140K-165KAWS | AWS IAM | AWS KMS | Active Directory | Amazon S3Senior-level Full TimeArlington, VA, United States4h ago
-
Sr. Databricks Consultant - Top Secret Clearance USD 175K-250KARM | AWS S3 | Amazon Kinesis | Apache Spark | Azure DataSenior-level Full TimeFAIRFAX, VA, United States4h ago
-
Sr. Databricks Consultant - TS/SCI-CI Poly USD 175K-250KAWS S3 | Agile | Apache Spark | Azure Data | Azure Data LakeSenior-level Full TimeFAIRFAX, VA, United States4h ago
-
Sr. Databricks Consultant - TS/SCI-FSP USD 175K-250KAWS S3 | Apache Spark | Azure Data | Azure Data Lake | CI/CDComprehensive benefits | Travel opportunitiesSenior-level Full TimeFAIRFAX, VA, United States4h ago
-
Sr. Databricks Consultant - Public Trust USD 175K-250KAWS S3 | Agile | Amazon Kinesis | Apache Spark | Azure DataPublic Trust ClearanceSenior-level Full TimeWork from home, VA, United States4h ago
-
Senior-level Full TimeWork from home, VA, United States4h ago
-
Sr. Databricks Consultant - DHS Public Trust USD 175K-250KAWS S3 | Agile | Amazon Kinesis | Apache Spark | Azure DataHealth benefits | Remote work options | Travel requiredSenior-level Full TimeWork from home, VA, United States4h ago
-
Mid-Level Data Engineer USD 130K-140KAPI Design | AWS | Agile | Amazon Redshift | Big DataCareer growth | Learning opportunities | Public trust clearance support | Remote workMid-level Full TimeWork from home, VA, United States4h ago
-
Senior Data Engineer USD 170K-215KANSI SQL | AWS S3 | Amazon Redshift | Azure Container | Azure Container Apps401k match | Company paid life insurance | Dental insurance | Health insurance | Holiday paySenior-level Full TimeOrlando, FL, United States5h ago
-
Senior Software Engineer, Data Engineering USD 243K-295KAWS | Apache Airflow | Apache Spark | Azure | Cloud ComposerSenior-level Full TimeSan Mateo, CA, United States R7h ago
-
Data Engineer Stf - E4 USD 131K-243KAccess Control | Apache NiFi | Apache Spark | Data Modeling | Data Processing401k match | Dental insurance | Education assistance | Employee assistance program | Flexible spending accountsSenior-level Full TimeFort Worth, TX, US9h ago
-
Data Engineer USD 126K-208KAPI Integration | AWS | Airflow | Azure | BigQueryDEI initiatives | Dental insurance | Employee rewards and recognition | Medical insurance | Mental health supportMid-level Full TimeRemote, United States10h ago
-
Data Analytics Engineer USD 126K-208KAmazon Web Services | Azure Data | Azure Data Factory | Azure Data Lake | Cloud platformDEI initiatives | Mental health support | Retirement plans | Rewards and recognition | Sustainability initiativesSenior-level Full TimeRemote, United States10h ago
-
AWS | Automated testing | Cloud Native | Cloud-native development | Columnar DatabasesCompany stock plan | Educational reimbursement | Unlimited vacation | Wellness reimbursementSenior-level Full TimeReno, NV; San Ramon, CA; New … R12h ago