Senior Staff Data Engineer, Databricks (R5492)
Tasks
- Build Delta Lake ingestion with batch and streaming
- Build Silver layer cleansing standardization deduplication and conformance
- Build and maintain CI CD pipelines for data assets
- Create data quality checks for business meaning and integrity
- Define reusable ingestion and transformation patterns
- Design and build ingestion pipelines
- Implement data classification and segregation controls
- Implement secure Databricks platform constructs
- Maintain data documentation and transformation notes
- Own Bronze layer ingestion and recoverable loading
- Partner with analytics engineers for governed data for modeling
- Quarantine and route problematic datasets
Perks/Benefits
- N/A
Skills/Tech-stack
Access Control | Apache Spark | Batch Ingestion | CDC | CI/CD | Data Classification | Data Governance | Data Modeling | Data Quality | Data Security | Databricks | Delta Lake | Delta Lake Tables | Dimensional Modeling | PySpark | SQL | Spark SQL | Streaming Ingestion | Testing | Unity Catalog | Version control | Workflow Orchestration
Education
N/A
Roles
Data Engineer | Engineer | Senior Data Engineer | Staff Data Engineer
Related jobs
-
Data Visualization Engineer PowerBI Sr MXN 643K-966KData Modeling | Looker | Microstrategy | Power BI | SQLEnglish advanced language support | Hybrid workSenior-level Full TimeMexico R6h ago
-
Automated Speech Recognition | CI/CD | CUDA | Computer Vision | Containerization401k matching | Company events | Dental coverage | Flexible work schedules | Internet reimbursementSenior-level Full TimeAnnapolis Junction, MD R8h ago
-
AWS Athena | AWS Glue | Amazon ECS | Amazon EMR | Amazon EMR ServerlessBenefits | Bonus incentives | Flexible hours | MentoringMid-level Full TimeRa'anana, Center District, IL R10h ago
-
AWS | BigQuery | Customer 360 | Data Governance | Data LakeRemote work | Two factor authenticationMid-level Full TimeRemote R11h ago
-
Data Engineer USD 124K-124KAWS | Apache Spark | DBT Cloud | DBT Core | JSON401k | Employee assistance program | Health savings account | Life insurance | Medical, dental, and visionMid-level ContractWestbrook, Maine, USA R11h ago
-
Data Analytics Engineer GBP 53K-60KAWS | AWS Glue | Airflow | Amazon Redshift | DBTAnnual Holiday Vouchers | Company bonus scheme | Discounted flights | Flexible benefits | Free easyJet Plus membershipMid-level Full TimeUnited Kingdom-London-London Luton Airport R12h ago
-
Data Engineer GBP 51K-51KAgile | Cognos | Data Modeling | Data Warehousing | Data virtualizationBalanced lifestyle | Flexible working arrangements | Hybrid working | Learning opportunities | Support for financial wellbeingMid-level Full TimeGB-ENG-LAN-Warton R12h ago
-
Staff Data Engineer INR 1500K-2500KAPIs | AWS | Agentic Workflows | Automated testing | Cloud PlatformsSenior-level Full TimeHyderabad, Telangana , India R13h ago
-
Senior Data Engineer PLN 216K-300KAWS | Agile | Algorithms | Aster Data | AzureCompany paid premium medical package | Company-paid certifications | Conferences Workshops | Full remote working possibilities | Internal eventsSenior-level Full TimePoland R13h ago
-
Senior-level Full TimeRa'anana, Center District, IL R14h ago
-
Senior Data Engineer INR 3000K-4000KAWS | Airflow | Apache Spark | CI/CD | DBTBike to work | Career mentorship | Educational assistance | Employee assistance programme | Employee discountSenior-level Full TimeMumbai, MH, India R14h ago
-
AWS | Azure | Azure Synapse | Bicep | CI/CD100 percent remote workSenior-level Full TimeSzczecin, Poland R14h ago
-
Senior Data Engineer (Healthcare domain) PLN 204K-336KAzure | Data Governance | Data Lake | Data Modeling | Data WarehousingRemote workSenior-level Full TimeWarsaw, Masovian Voivodeship, Poland R14h ago
-
Data Engineer INR 3000K-4000KAmazon S3 | Databricks | ELT | ETL | PythonBike to work | Educational assistance | Employee assistance program | Employee discount | Enhanced parental leaveSenior-level Full TimeBengaluru, KA, India R15h ago
-
Sr Data Engineer_ITDE1I USD 102K-170KAPI Integration | AWS | Automation | Azure | CI/CD401k match | Dental insurance | Disability insurance | Hybrid work policy | Life insuranceSenior-level Full TimeHOUSTON, US, 77056 R16h ago
-
Senior Data Engineer (Healthcare domain) PLN 204K-336KAzure | Data Governance | Data Lake | Data Modeling | Data WarehousingRemote workSenior-level Full TimeWarsaw, Masovian Voivodeship, Poland R16h ago
-
Senior Data Engineer (ML/AI) - Alexandria USD 130K-239KApache Airflow | Apache Spark | Azure Data | Azure Data Factory | Azure SynapseContinuous learning | Hybrid work | MentorshipSenior-level Full TimeCairo, Cairo Governorate, Egypt R16h ago
-
Senior Data Engineer - Alexandria USD 120K-199KAzure | Cloud Computing | Data Compliance | Data Governance | Data ModelingCareer growth | Hybrid work schedule | Mentorship | Structured onboardingSenior-level Full TimeCairo, Cairo Governorate, Egypt R16h ago
-
Senior Data Engineer (ML/AI) - Alexandria USD 130K-231KAirflow | Apache Spark | Azure Data | Azure Data Factory | Azure SynapseCareer growth opportunities | Continuous learning | Hybrid work flexibility | Mentorship | Structured onboardingSenior-level Full TimeAlexandria, Alexandria Governorate, Egypt R16h ago
-
Senior Software Engineer INR 1163K-3000KAutomated testing | Batch Processing | CI/CD | DAX | Data GovernanceFlexible work arrangement | Hybrid work modelSenior-level Full TimeIndia, Telangana, Hyderabad R16h ago
-
API Design | API Integration | AWS | Alerting | AzureFlexible working hours | Hackathons | Partial remote work | Team-building tripsMid-level Contract FreelancePoland R16h ago
-
Assistant Director,Data and Analytics INR 2337K-5199KAnomaly Detection | BigQuery | Cloud Computing | Data Visualization | Fraud analyticsExecutive-level Full TimeBangalore, Karnataka, India R17h ago
-
Mid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL R17h ago
-
Apprentice - AI Engineer INR 300K-324KAWS | Anthropic SDK | Azure | Docker | EmbeddingsCareer development | Collaborative team | Exposure to global teams | Holiday allowance | Hybrid work environmentEntry-level Apprenticeship Full TimeWhitefield, Bangalore, India R18h ago
-
Solution Database Engineer INR 1500K-2600KAgile | Azure Data | Azure Data Factory | Azure DevOps | Common table expressionsSenior-level Full TimeBengaluru, KA, India R18h ago