Data Manager (Databricks)
Tasks
- Architect scalability high availability and disaster recovery
- Build and optimize data pipelines
- Collaborate with stakeholders to deliver roadmap
- Define data platform architecture
- Define observability and monitoring
- Design CI CD patterns
- Design data models
- Develop Spark processing modules
- Establish security and compliance controls
- Implement Infrastructure as Code and version control
- Implement data governance
- Implement ingestion batch and streaming
- Lead platform component integration
- Mentor data engineering teams
- Optimize storage partitioning and file compaction
- Troubleshoot data quality and performance issues
Perks/Benefits
- N/A
Skills/Tech-stack
AWS RDS | Airflow | Amazon Aurora | Apache Spark | Auto Loader | Azure Data | Azure Data Fabric | Azure Data Factory | Azure DevOps | Azure SQL | CI/CD | Data Contracts | Data Factory | Data Governance | Data Lineage | Data fabric | Databricks | Datadog | Delta | Delta Lake | Delta Live Tables | Delta Live) | Disaster Recovery | Encryption | Fivetran | GitHub Actions | Grafana | High Availability | IAM | Infrastructure as Code | Jenkins | Kappa Architecture | Lambda Architecture | Medallion Architecture | Metadata Management | Network Segmentation | Oracle | PostgreSQL | Prometheus | PySpark | RBAC | RPO | RTO | SQL Server | Spark SQL | Structured Streaming | Unity Catalog | Version control | “as-code”
Education
N/A
Related jobs
-
Business Intelligence | Compliance | Data Governance | Data Lineage | Data ManagementSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Strategy & Operations Manager USD 126K-174KAI strategy | Budget Alignment | Business Intelligence | Change Management | Data GovernanceMid-level Full TimeFlorida, United States17h ago
-
Supervisor Data Engineering USD 78K-142KAgile | Artificial Intelligence | Azure | BI | Change Management401k matching | Career development support | Dental insurance | Health insurance | Life insuranceSenior-level Full TimeAkron, OH, United States21h ago
-
Data Integration & Pipeline Manager USD 96K-125KAWS | AWS Glue | Agile | Amazon Athena | Amazon EC2Mid-level Full TimeMiami Lakes, FL, United States21h ago
-
Credit Risk Analytics Manager-Finance USD 87K-132KAttribution | Backtesting | Benchmarking | CECL | Credit RiskMid-level Full TimeMiami Lakes, FL, United States21h ago
-
Director of AI & Data Analytics USD 240K-322KAWS | Anomaly Detection | Apache Airflow | Azure | DBT401k match | Autonomy | Collaborative culture | Generous PTO | Health and wellness benefitsExecutive-level Full TimeDallas, TX, US22h ago
-
Manager, Data Engineering USD 125K-153KAWS | Agile | Amazon RDS | Apache Kafka | Apache Spark401k administration with match | Disability insurance | Medical/Dental/Vision insurance | Paid parental leave | Paid sabbatical leaveMid-level Full TimeRemote, US R1d ago
-
Director, Business Planning & Analytics USD 160K-190KBusiness Intelligence | Cognos Analytics | Data Automation | Data Governance | Database Design401k plan | Paid Holidays | Paid time off | Retention bonusExecutive-level Full TimeGordonsville, Virginia, United States1d ago
-
ArcGIS Enterprise | BigQuery | BusinessObjects | Data Entry Management | Data GovernanceSenior-level Full TimeThornton, CO, USA; Atlanta, GA, USA1d ago
-
VP, Data Platform and Knowledge USD 252K-320KAI infrastructure | Data Architecture | Data Engineering | Data Governance | Data QualityExecutive-level Full TimePittsburgh, PA1d ago
-
Data Governance and AI Project Manager USD 123K-164KA/B | A/B Testing | AI Governance | Acceptance Testing | B testing401k match | Fitness reimbursement | Free fresh fruit | Learning platform training resources | On-site fitness centerMid-level Full TimeDes Moines, IA, United States1d ago
-
Sr. Data Quality & Governance Manager USD 170K-287KAI Automation | Account Hierarchy | Alteryx | CCPA | Contact DataCompassionate care leave | Health benefits | Learning and development | Paid parental leave | Paid time offSenior-level Full TimeSan Francisco, CA, United States1d ago
-
Total Rewards Systems & Analytics Senior Manager USD 116K-210KAnalytics | Artificial Intelligence | Automation | Data Governance | GovernanceSenior-level Full Time6314 Remote/Teleworker US, United States R1d ago
-
Slate Manager – Enrollment Management USD 66K-80KCRM Integrations | Dashboards | Data Governance | Data Integrity | Data ModelingHealth insurance | Paid time off | Retirement plan | Tuition benefitsMid-level Full TimePittsburgh, United States1d ago
-
AI/ML Deployment and Enablement Leader USD 170K-200KAmazon Web Services | Artificial Intelligence | Auditability | Automated rollback | Azure AI401k employer match | Adoption Assistance | Dental insurance | Health insurance | Life insuranceSenior-level Full TimeMinneapolis, MN, United States1d ago
-
HR Data Governance & Privacy Lead USD 92K-109KAudit Readiness | Change Management | DSAR | Data Governance | Data Management401k match | Adoption Assistance | Dental insurance | Disability insurance | Health insuranceSenior-level Full TimeMinneapolis, MN, United States1d ago
-
Senior Manager Analytics Consulting USD 121K-225KArtificial Intelligence | Business Intelligence | Client Management | Data Analysis | Data Governance401k | Disability insurance | Employee assistance program | Employee referral program | Hybrid workSenior-level Full TimeRichardson Texas HQ (1001 E. Lookout …1d ago
-
Palantir Data Pipeline & Metadata Lead USD 149K-248KAWS | Agile | Amazon S3 | Apache Spark | Auditability401k retirement plan | Adoption Assistance | Dental insurance | Disability insurance | Employee referral programSenior-level Full TimeGH Office: Tysons Corner, VA (Headquarters), …1d ago
-
340B | Analytics | Audit Readiness | Automation | ComplianceHealth insurance | Paid Holidays | Paid caregiver/parental leave | Paid medical leave | Paid personal daysSenior-level Full TimeUS CA San Diego, United States1d ago
-
Software Engineering Manager (Analytics & Innovation) USD 137K-206KAI machine learning | Advanced Analytics | Agile | Azure | Budgeting401k matching program | Dental insurance | Discretionary annual bonus | Educational reimbursement | Health insuranceMid-level Full TimeSan Antonio TX, United States1d ago
-
AWS | Amazon Kinesis | Amazon Redshift | Amazon S3 | Apache KafkaKnowledge sharing | Mentorship and career growth | Travel opportunitiesSenior-level Full TimeHerndon, Virginia, USA1d ago
-
Amazon Kinesis | Apache Kafka | Apache Spark | Cloud Data | Cloud data platformSenior-level Full TimeHerndon, Virginia, USA1d ago
-
Data Platform Engineer USD 77K-176KAWS | Agile | Amazon EMR | Amazon Redshift | Apache SparkDependent care | Disability insurance | Health insurance | Life insurance | Paid leaveMid-level Full TimeUSA, VA, Alexandria (600 Dulany St), …1d ago
-
Agency Data & Analytics Senior Manager (Hybrid) USD 113K-194KAI analytics | Business Intelligence | Cloud Migration | Cross-functional | Cross-functional leadership401k contribution | Annual incentive | Hybrid work | Medical, dental, vision benefits | Paid HolidaysSenior-level Full TimeAF-WI Madison Natl HQ, United States R1d ago
-
Solution Engineer - Data Engineering Specialist (FSI) USD 189K-248KAmazon Kinesis | Apache Flink | Apache Hadoop | Apache Hive | Apache IcebergSenior-level Full TimeUS-NY-New York1d ago