Data Manager (Databricks)
Tasks
- Architect scalability high availability and disaster recovery
- Build and optimize data pipelines
- Collaborate with stakeholders to deliver roadmap
- Define data platform architecture
- Define observability and monitoring
- Design CI CD patterns
- Design data models
- Develop Spark processing modules
- Establish security and compliance controls
- Implement Infrastructure as Code and version control
- Implement data governance
- Implement ingestion batch and streaming
- Lead platform component integration
- Mentor data engineering teams
- Optimize storage partitioning and file compaction
- Troubleshoot data quality and performance issues
Perks/Benefits
- N/A
Skills/Tech-stack
AWS RDS | Airflow | Amazon Aurora | Apache Spark | Auto Loader | Azure Data | Azure Data Fabric | Azure Data Factory | Azure DevOps | Azure SQL | CI/CD | Data Contracts | Data Factory | Data Governance | Data Lineage | Data fabric | Databricks | Datadog | Delta | Delta Lake | Delta Live Tables | Delta Live) | Disaster Recovery | Encryption | Fivetran | GitHub Actions | Grafana | High Availability | IAM | Infrastructure as Code | Jenkins | Kappa Architecture | Lambda Architecture | Medallion Architecture | Metadata Management | Network Segmentation | Oracle | PostgreSQL | Prometheus | PySpark | RBAC | RPO | RTO | SQL Server | Spark SQL | Structured Streaming | Unity Catalog | Version control | “as-code”
Education
N/A
Related jobs
-
AWS | Agile | Athena | CI/CD | Code review401k with company match | Annual performance bonus | Baby bonding leave | Commuter benefits | Company holidaysExecutive-level Full TimeSan Carlos, CA22h ago
-
Benefits Administration | Business Intelligence | Compensation Management | Compliance reporting | Data GovernanceInclusion and Belonging Culture | Remote work flexibility | Training and documentation supportMid-level Full TimeRemote, United States R1d ago
-
Director of Data Platform & Engineering USD 185K-235KAPI Integration | Access Control | Azure Data | Azure Data Factory | Azure Data LakeDental insurance | Employee assistance program | Health insurance | Mental health benefits | Paid HolidaysExecutive-level Full TimeManhattan, New York R1d ago
-
Engineering Manager, Data USD 190K-230KAirflow | Cloud Computing | DICOM | Dagster | Data EngineeringRemote workSenior-level Full TimeUnited States1d ago
-
AWS | Agile | Airflow | Alteryx | AzureSenior-level Contract Full TimeChicago, Illinois, United States1d ago
-
AWS | Acceptance criteria | Agile | Analytics Platforms | AzureBackup childcare | Financial coaching | Health insurance | Mental health support | On-site wellness centerExecutive-level Full TimeNew York, NY, United States1d ago
-
Sr. Manager, Data Platforms USD 133K-195KAPIs | Access Controls | Batch Processing | Data Catalog | Data EngineeringContractor and partner support | Cross functional stakeholder collaboration | Hybrid work environmentSenior-level Full TimeCuyahoga Falls, OH, United States1d ago
-
Data Engineering Manager USD 150K-185KAccess Control | Batch Processing | CI/CD | Data Governance | Data Lineage401k match | Company funded HSA | Dental insurance | Flexible PTO | Health insuranceSenior-level Full TimeRemote (United States) R1d ago
-
Manager, Data Quality & Analytics USD 80K-172KCIBMTR FormsNet | Clinical data | Clinical data management | Data Auditing | Data ConsistencyOccasional weekend work | Telework options | Travel to worksite locationsMid-level Full TimePortland, OR, United States1d ago
-
Manager, Data Quality & Analytics USD 80K-172KBudget Management | Data Auditing | Data Quality | Data analytics | Data reportingTeleworking | Travel to worksite locations | Weekend work as neededMid-level Full TimePortland, OR, United States1d ago
-
Manager, Data Quality & Analytics USD 80K-172KAnalytical Software | Audit management | Clinical data | Clinical data management | Data ManagementOccasional weekend work | Telework options | Travel between worksite locationsMid-level Full TimePortland, OR, United States1d ago
-
Adobe Customer Journey | Adobe Customer Journey Analytics | Analytics validation | Campaign analytics | Cookie ConsentMid-level Full TimePlainsboro, NJ, US1d ago
-
Agile | Amazon Web Services | Anaconda | Angular | AnsibleExecutive-level Full TimeCorpus Christi, TX, United States1d ago
-
Manager, Compensation and People Analytics USD 110K-150KAdvanced Compensation | Compensation Management | Compensation benchmarking | Dashboarding | Data GovernanceChildcare resources | Disability insurance | Fertility assistance | Fitness reimbursement | Health insuranceMid-level Full TimeNew York, New York, United States1d ago
-
Palantir Lead USD 135K-200KAPI Integration | Access Control | Amazon Web Services | Apache Spark | Cloud PlatformsSenior-level Full TimeSan Jose, United States1d ago
-
Director, AI Analytics and Reporting Enablement USD 187K-312KAI | AWS | Apache Spark | Azure | Cloud DataExecutive-level Full TimePhiladelphia Walnut Street, United States1d ago
-
AVP, Business Analytics & AI USD 169K-241KAI Governance | AI Operations | AI Portfolio Management | AI portfolio | Amazon Web Services401k with profit sharing | Flexible work arrangements | Health and welfare benefits | Hybrid work | Paid time offExecutive-level Full TimeFarmington Hills, MI, United States1d ago
-
Head of Banking Data, AI and Analytics Platform USD 169K-175KAWS | Access Management | Airflow | Continuous integration | DBT401k | Life insurance | Medical, dental & vision coverage | Paid leaveExecutive-level Full TimeBuilding 400-Whippany Campus, Jefferson Park, United …1d ago
-
IT Product Manager (Data Analytics) Onsite USD 34K-34KAdvanced Analytics | Analytics roadmap | BigQuery | Business Requirements | Cloud platform401k plan | Dental insurance | Health insurance | Life insurance | Paid time offMid-level Full TimeTyson Emma - Springdale, Arkansas, United …1d ago
-
Data Engineering Leader, Supply Chain Digital Automation USD 116K-194KAsset Hierarchy | Automated testing | CI/CD | Cloud Computing | Data ContractsChallenging work | Equal opportunity employer | Professional developmentSenior-level Full TimeGreenville, United States1d ago
-
Senior Commercial Analytics Manager USD 140K-223KAdvanced Analytics | Data Governance | Data Visualization | Excel | KPI Tracking401k match | Education reimbursement | HSA contribution | PTO | Relocation not providedSenior-level Full TimeStamford - B2, United States1d ago
-
Senior Manager, Analytics Instrumentation USD 160K-180KAdobe Analytics | CSS | Data Governance | Data Layer | Data PrivacyHybrid work environment | On site attendance 3 days per weekSenior-level Full TimeUSA - CA - Los Angeles …1d ago
-
Data Engineering Leader, Supply Chain Digital Automation USD 116K-194KAccess Management | Asset Hierarchy | Auditability | Automated testing | CI/CDChallenging work environment | Equal opportunity workplace | Professional developmentSenior-level Full TimeGreenville, United States1d ago
-
Seasonal Data Engineer USD 150KAnalysis Services | Automation | Azure | Azure Analysis | Azure Analysis ServicesMid-level Full TimeSchwerin Office/Warehouse, United States1d ago
-
Sr Marketing Analytics Manager, Marketing Technology and Operations, Transportation Technology and Services USD 142K-193KAI analytics | Adobe Analytics | Anomaly Detection | Attribution | Audience SegmentationSenior-level Full TimeBellevue, Washington, USA1d ago