Data Manager (Databricks)
Tasks
- Architect scalability high availability and disaster recovery
- Build and optimize data pipelines
- Collaborate with stakeholders to deliver roadmap
- Define data platform architecture
- Define observability and monitoring
- Design CI CD patterns
- Design data models
- Develop Spark processing modules
- Establish security and compliance controls
- Implement Infrastructure as Code and version control
- Implement data governance
- Implement ingestion batch and streaming
- Lead platform component integration
- Mentor data engineering teams
- Optimize storage partitioning and file compaction
- Troubleshoot data quality and performance issues
Perks/Benefits
- N/A
Skills/Tech-stack
AWS RDS | Airflow | Amazon Aurora | Apache Spark | Auto Loader | Azure Data | Azure Data Fabric | Azure Data Factory | Azure DevOps | Azure SQL | CI/CD | Data Contracts | Data Factory | Data Governance | Data Lineage | Data fabric | Databricks | Datadog | Delta | Delta Lake | Delta Live Tables | Delta Live) | Disaster Recovery | Encryption | Fivetran | GitHub Actions | Grafana | High Availability | IAM | Infrastructure as Code | Jenkins | Kappa Architecture | Lambda Architecture | Medallion Architecture | Metadata Management | Network Segmentation | Oracle | PostgreSQL | Prometheus | PySpark | RBAC | RPO | RTO | SQL Server | Spark SQL | Structured Streaming | Unity Catalog | Version control | “as-code”
Education
N/A
Related jobs
-
SAP Data Governance Manager USD 116K-243KBI reporting | Business glossary | Data Catalog | Data Governance | Data LineageSenior-level Full TimeWashington, DC20h ago
-
Manager, Data Governance (Remote) USD 120K-130KAgile | Azure | Business glossary | Cloud analytics | Data Governance401k match | Dental insurance | Health insurance | Health savings account | Life insuranceSenior-level Full TimeDallas, TX R21h ago
-
Data Insights & Analytics Manager USD 115K-125KAdvanced Analytics | Data Architecture | Data Governance | Data Modeling | Data VisualizationAnnual bonus | Health insurance | Hybrid work | Paid Holidays | Paid time offMid-level Full TimeNorthbrook, IL, United States21h ago
-
Compute Intelligence Engineer USD 225K-300KAPIs | Apache Airflow | BI Dashboards | BigQuery | DBTConference attendance | Professional development budget | Relocation support | Remote work | Team offsitesSenior-level Full TimeSan Francisco21h ago
-
Manager, Advanced Analytics USD 100K-160KA/B | A/B Testing | B testing | BigQuery | Clustering401k | Dental insurance | Medical insurance | Paid leave | Tuition reimbursementMid-level Full TimeUniversal City, CALIFORNIA, United States21h ago
-
Manager, FAST Analytics USD 100K-120KAVOD | Data Governance | Data Infrastructure | Data Modeling | Data Visualization401k | Dental insurance | Discounts | Hybrid work schedule | Medical insuranceMid-level Full TimeNew York, NEW YORK, United States R21h ago
-
Director, Data Engineering & Architecture (Remote) USD 210K-272KAI | Analytics reporting | Cloud Native | Data Architecture | Data Governance401k matching | Cell phone stipend | Employee stock purchase plan | Flexible paid time off | Health benefitsSenior-level Full TimeSunnyvale, CA R22h ago
-
Manager of Data Platform Engineering USD 87K-119KAgile | Apache Airflow | Apache Kafka | Apache Spark | Backlog GroomingSenior-level Full TimeMorristown, NJ, United States23h ago
-
Principal Data Engineer/ Technical Lead USD 219K-298KAWS | Access Layer | Aggregation pipelines | Apache Kafka | Apache Spark401k match | Employer paid medical/dental/vision | Flexible spending account | Paid parental leave | Remote first work from homeSenior-level Full TimeUnited States (Remote) R1d ago
-
Data Analytics Manager (Health Research Team) USD 104K-199KClinical quality | Data Governance | Data Quality | Data Quality Metrics | Data Validation401k matching | Discretionary bonus program | Employee assistance program | Employee benefits package | Flexible work arrangementsMid-level Full TimeChicago, Illinois, United States R1d ago
-
Technical Project Manager - Data Science USD 110K-145KCross-functional | Cross-functional leadership | Data Architecture | Data Engineering | Data GovernanceMid-level Full TimeUnited States1d ago
-
Data Platform/Infrastructure Engineer USD 135K-180KAPI Gateway | AWS GovCloud | Access Control | Access Management | AirflowSecurity clearance processing supportSenior-level Full TimeFAIRFAX, VA, United States1d ago
-
Data Analysis Manager USD 85K-120KBusiness Process | Business process improvement | Data Governance | Data Management | Data analyticsMid-level Full TimeUnited States1d ago
-
Databricks Data Engineering Manager USD 151K-252KBusiness Intelligence | Data Engineering | Data Governance | Databricks | MLOpsTravel up to 25 percentMid-level Full TimeArlington/Rosslyn, Virginia, United States; Sacramento, California, …1d ago
-
Business Intelligence | CI/CD | Cloud Computing | Data Lake | PythonHealth insurance | Job security | Pension plan | Tax-deferred savings | Work from homeExecutive-level Full TimeLong Island City, NY, United States1d ago
-
APIs | Access Control | CI/CD | Cloud Data | Cloud Data PlatformsHealth insurance | Pension plan | Tax-deferred savings | Work From Home Up To Two Days | Worksite wellness programExecutive-level Full TimeLong Island City, NY, United States1d ago
-
Sr. Manager, Global Reporting & Analytics USD 106K-133KBusiness Intelligence | Chart of Accounts | DSO | Dashboards | Data Architecture401k match | Dental insurance | Long-term disability insurance | Medical insurance | Paid HolidaysSenior-level Full TimeSt. Louis, MO, 63146, US1d ago
-
API | Authentication | Authorization | CI/CD | Call Management401k | Discretionary vacation | Flexible spending account | Health savings account | Life and disability insuranceMid-level Full TimeSeattle, WA1d ago
-
Senior AI Engineering Team Manager USD 150K-220K.NET | Aerospike | Artificial Intelligence | C# | Cloud FoundryBonus Incentive Opportunities | On-site collaborationSenior-level Full TimeSouthlake, TX, United States1d ago
-
Data Governance | Data Modeling | Data Pipelines | Data Quality | Data Warehousing401k with company match | Comprehensive medical, dental, and vision insurance | Flexible spending account | Health savings account | Paid HolidaysSenior-level Full TimeLake Forest, IL, United States1d ago
-
Senior Manager, Data Engineering Lead USD 125K-165KAI in development | Access Management | Agile | Bamboo | BigQueryBonus incentives | Career developmentSenior-level Full TimeSouthlake, TX, United States1d ago
-
Head of Engineering, Data Platform USD 223K-270KAgentic Workflows | Cloud Computing | DBT | Data Architecture | Data Governance401k matching | Cell phone stipend | Dental insurance | Flexible time off | Internet stipendExecutive-level Full TimeRemote - USA R1d ago
-
Director, Data Engineering (AI Native) USD 216K-318KAmplitude | Anomaly Detection | Artificial Intelligence | Cloud Cost Management | Cost ManagementRemote work environmentExecutive-level Full TimeRemote, USA; Remote, Canada R1d ago
-
Data Platform Architect USD 100K-150KAccess Management | Alation | Amazon Redshift | Apache Flink | Apache KafkaSenior-level Full TimeUnited States - Remote R1d ago
-
Senior Analytics Manager - AI Model & Prompt Engineering USD 172K-258KA/B | A/B Testing | A2A protocols | AI Evaluation | AI Foundry401k | Adoption benefits | Career development | Disability benefits | Employee assistance programSenior-level Full TimeChicago, Illinois, United States1d ago