Data Architect - Databricks
Tasks
- Architect data storage using ADLS Gen2
- Automate code deployment processes
- Build Spark based data ingestion pipelines
- Build and govern medallion data layers
- Collaborate with global teams and mentor engineers
- Define data architecture standards and reference architectures
- Design CI CD pipelines and GitHub workflows
- Design data acquisition frameworks with CDC batch micro-batch and streaming
- Design data governance and security models
- Design domain and sub-domain storage
- Design solution architecture for enterprise data platforms on Azure
- Implement Unity Catalog
- Lead data and AI platform architecture
- Provide hands on technical leadership with Python and PySpark
Perks/Benefits
- Company-Paid Holidays
- Employee assistance program
- Life and disability insurance
- Medical, dental, and vision coverage
- Paid time off
- Retirement savings plan
Skills/Tech-stack
ADLS Gen2 | Azure | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | CDC | CI/CD | Data Architecture | Data Engineering | Data Lake Storage Gen2 | Databricks | Databricks Notebooks | Git | GitHub | GitHub Actions | Medallion Architecture | Micro-batch | PySpark | Python | Spark | Streaming | Unity Catalog
Education
N/A
Related jobs
-
Cloud & Data Engineer USD 83K-187KAWS | Azure | Big Data | CI/CD | Container Security401k savings plan | Disability insurance | Employee assistance program | Flexible vacation | Life insuranceEntry-level Full TimeLexington, MA, US2h ago
-
AI Solution Principal Systems Development Engineer USD 148K-222KCluster management | Containerization | GPU Computing | Generative AI | Hardware/Software Integration401k plan | Choice of medical coverage | Fitness reimbursement | Generous Time Off Programs | Team member discounts on Dell productsSenior-level Full TimeAustin, Texas, United States, United States10h ago
-
AI Search | Azure AI | Azure AI Search | CI/CD | Chroma401k | Dental insurance | Medical insurance | Paid sick hours | Vision insuranceSenior-level Contract Full TimeRidgefield Park, NJ, United States10h ago
-
API Security | AWS | Adversarial inputs | Agent Frameworks | Application SecurityCiti Bike membership | Flexible paid time off | Gym membership | Health and mental health services | Home office stipendSenior-level Full TimeSan Francisco, California, United States; Seattle, …11h ago
-
Enterprise Data Architect & Integration Lead USD 119K-213KAPI Design | AWS | Active Directory | Automation | Azure401k plan | Community and volunteering programs | Commuter benefits | Employee assistance program | Flexible spending accountSenior-level Full TimeProvidence, Rhode Island, United States13h ago
-
APIs | Artificial Intelligence | Automation | Decision support | Generative AI401k | Cafe Amenities | Fitness Center | Health insurance | Recreational facilitiesEntry-level Full TimeBroomfield, CO, United States13h ago
-
Applied AI Engineer (AI Adoption) USD 200K-250KAI Evaluation | Agent Design | Automation | LLM APIs | Machine LearningDental insurance | Group pension scheme | Income protection | Life assurance | Paid HolidaysMid-level Full TimeNew York, Chicago, London13h ago
-
Sr. Staff Embedded AI Engineer USD 165K-200KBare Metal | C# | C++ | CMSIS NN | Code generationEmployee resource groups | Flexible work environment | Hybrid schedule | Remote work optionSenior-level Full TimeColumbia, MARYLAND, United States13h ago
-
Senior Software Engineer, Robotics USD 141K-194KAlgorithms | C++ | CI/CD | Computer Vision | Constraint SatisfactionCompany holidays | Health insurance | Learning reimbursement | Life insurance | Long-term disabilitySenior-level Full TimeTorrance, California, United States13h ago
-
Senior Data Engineer USD 157K-218KAWS Athena | AWS EMR | AWS Glue | AWS Lambda | Amazon S3Beverages | Commuter stipend | Hybrid work schedule | Lunch stipend | Relocation stipendSenior-level Full TimeDenver, Colorado, United States; San Francisco, …14h ago
-
Principal Machine Learning Engineer USD 157K-223KA/B | A/B Testing | AI Agents | Airflow | B testingSenior-level Full TimeUS TX Austin14h ago
-
Senior Forward Deployed Engineer (AI/ML) USD 140K-174KBenchmarking | CUDA | Continuous batching | CrewAI | DatabasesConference reimbursement | Employee assistance program | Flexible time off | LinkedIn Learning access | Local Employee MeetupsSenior-level Full TimeSeattle R14h ago
-
Senior Forward Deployed Engineer (AI/ML) USD 140K-174KArtificial Intelligence | CUDA | Continuous batching | CrewAI | DatabasesConference reimbursement | Employee assistance program | Flexible time off | LinkedIn Learning access | Remote workSenior-level Full TimeSan Francisco R14h ago
-
Data Engineer USD 127K-175KAWS | Airbyte | Airflow | Alerting | CI/CD401k contribution | Dental insurance | Health insurance | Life insurance | Long-term disabilitySenior-level Full TimeUnited States - Remote R14h ago
-
Senior Software Engineer, Storage Content USD 198K-248KAlerting | Apache Spark | Bigtable | Cloud Computing | Content streamingSenior-level Full TimeRedwood City, California14h ago
-
Sr. AI Engineer USD 152K-183KAPI Development | AWS | Agent Frameworks | Agentic AI | Artificial Intelligence401k plan | Dental insurance | Hybrid work | Medical insurance | Tuition reimbursementSenior-level Full TimeUS - Orlando14h ago
-
Staff Engineer - AI Engineer USD 155K-175KERP | Hugging Face | Langchain | MES | ModbusContract extension possible | Remote workSenior-level Full TimeRemote, REMOTE, United States R14h ago
-
AI Data Infrastructure Engineer - Helix Team USD 150K-350KAWS | Ansible | Azure | Cloud platform | Data PipelinesOn-site collaborationSenior-level Full TimeSan Jose, CA14h ago
-
Associate, AI/ML Engineer USD 81K-106KAI Search | AWS | Azure | Azure AI | Azure AI Search401k match | Commuter benefits | Flexible spending account | Health savings account | Life and AD&D insuranceMid-level Full TimeSomerset, New Jersey, United States15h ago
-
AI Governance | AI Observability | AI Security | API | Access ManagementSenior-level Full TimeUS-CA-Menlo Park15h ago
-
AI Governance | AI Security | API | Access Control | Access ManagementSenior-level Full TimeUS-NY-New York15h ago
-
ARP | C# | C++ | Device Drivers | Embedded Linux401k match | Dental insurance | Employee assistance program | Health savings account | Medical insuranceSenior-level Full TimePittsburgh, PA, Palo Alto, CA, Detroit, …15h ago
-
Sr. Solutions Architect - BI and Analytics USD 219K-301KAWS | Apache Spark | Azure | Data Engineering | Data VisualizationSenior-level Full TimeUnited States16h ago
-
Machine Learning Engineer USD 125K-192K3D Reconstruction | C++ | CUDA | Camera Calibration | Computer VisionMid-level Full TimeSunnyvale, CA, United States16h ago
-
Forward Data Engineer USD 111K-221KAgile | Amazon EMR | Apache Iceberg | Apache NiFi | Apache SparkSenior-level Full TimeChantilly, VA16h ago