Databricks Senior Data & AI Engineer
Tasks
- Apply MLflow for experiment tracking
- Build RAG pipelines
- Build data pipelines
- Conduct knowledge transfer
- Configure row level and column level security
- Deploy model serving endpoints
- Design Lakehouse architecture
- Design data models for reporting
- Develop AI agents
- Implement Databricks SQL analytics
- Implement Delta Sharing
- Implement data governance with Unity Catalog
- Implement medallion architecture
- Integrate business intelligence dashboards
- Integrate external model APIs
- Lead code reviews
- Manage compute cost governance
- Optimize Databricks performance tuning
- Optimize data pipelines
- Support scoping and solutioning
- Use feature engineering in Unity Catalog
Perks/Benefits
- N/A
Skills/Tech-stack
AI Agent framework | AI BI | AI BI Genie | AWS | Agent Framework | Anthropic | Apache Airflow | Apache Spark | Azure | Column-Level Security | DBT | DLT | Data Governance | Data cloud | Databricks | Databricks Apps | Databricks SQL | Delta Lake | Delta Live Tables | Delta Live) | Delta Sharing | Feature Engineering | GCP | Gemini | LLM integration | Lakehouse Architecture | Looker | MLflow | Medallion Architecture | Model Serving | Mosaic AI | Mosaic AI Agent Framework | Mosaic Vector Search | OpenAI | Power BI | RAG | Row Level Security | Salesforce Data Cloud | Salesforce data | Snowflake | Tableau | Unity Catalog | Vector Search
Education
N/A
Related jobs
-
AI/ML Engineer USD 117K-171KA/B | A/B Testing | API Development | AWS | B testingDental insurance | Health insurance | Paid time off | Retirement plan | Sick leaveSenior-level Full TimeResidence Based, Residence Based, US2h ago
-
Cloud Data Engineer USD 92K-113KAgile | Apache Airflow | Beam | BigQuery | Cloud ComposerOnsite workMid-level Full TimeAda, MI, United States, 493554h ago
-
Forward Deployed Machine Learning Engineer USD 180K-300KAPI Design | Cloud Computing | Deep learning | Diffusion Models | Fine TuningIn-person collaboration days | Remote work flexibility | Travel cost coverageSenior-level Full TimeSan Francisco (USA) R10h ago
-
Senior Software Engineer, Knowledge Graph USD 196K-230KApache Flink | Apache Spark | Data Ingestion | Data Processing | DatabasesEmployee travel credits | Remote eligibleSenior-level Full TimeUnited States11h ago
-
Specialist Solutions Architect - AI/ML USD 180K-247KAI guardrails | Amazon Web Services | Apache Spark | Artificial Intelligence | Cloud ComputingMentorship | Remote work | Technical training | Travel up to 30 percentSenior-level Full TimeUnited States11h ago
-
Lead Data Engineer USD 115K-130KAgile | Apache Airflow | Azure Data | Azure Data Factory | Cloud Orchestration401k | Dental insurance | Medical insurance | Paid leave | Tuition reimbursementSenior-level Full TimeUniversal City, CALIFORNIA, United States12h ago
-
Machine Learning Engineer USD 168K-198KAPIs | Amazon SageMaker | Anthropic API | Asynchronous processing | CI/CDAdoption reimbursement | Commuter stipend | Fertility reimbursement | Flexible PTO | Learning and development allowanceSenior-level Full TimeSan Francisco, California, United States13h ago
-
AWS MSK | Amazon Kinesis | Apache Flink | Apache Kafka | AutomationSenior-level Full TimeUSA - Atlanta, GA; USA - …13h ago
-
Cyber Data Engineer USD 140K-145KAWS | ArcSight | Bash | Cribl | DHCP401k match | Accrued PTO | Health/Dental/Vision | Life insurance | Long-term disabilitySenior-level Full TimeSpringfield, VA14h ago
-
Staff Software Engineer - Container Platform USD 236K-339KAWS | Azure | Cloud automation | Cluster Lifecycle Management | Cluster lifecycleSenior-level Full TimeUS-CA-Menlo Park14h ago
-
Assistant Vice President, Data Engineer USD 142K-213KAPI Integration | Authentication | Authorization | Azure Data | Azure Data FactoryExecutive-level Full TimeChicago, IL14h ago
-
Mid-level Full TimeWashington, District of Columbia, United States15h ago
-
Sr/Staff Data Engineer - On-site in NYC! USD 175K-230KAWS | Apache Iceberg | Apache Kafka | Apache Spark | DBTEquity | Hybrid flexibility | Mission-driven workSenior-level Full TimeUSA16h ago
-
Applied AI Engineer USD 110K-163KAI Evaluation | API Integration | AWS Bedrock | Agentic Workflows | Agile ScrumGenerous benefits | Hybrid work | Incentives | Learning and trainingMid-level Full TimeKing of Prussia, Pennsylvania, United States16h ago
-
A/B | A/B Testing | APIs | Airflow | B testingSenior-level Full TimeUnited States16h ago
-
Senior Data Engineer (Information Technology) USD 81K-129KAlerting | Apache Spark | Automated testing | Azure | DBTDental and vision plans | Disability coverage | Employee Assistance Program (EAP) | Flexible Spending Accounts FSA | Health Savings Account (HSA)Senior-level Full TimeColorado Springs, US-CO, US17h ago
-
Full Stack Software Engineer - Robotics USD 125K-200KAWS | Datadog | Distributed Systems | Edge Computing | Grafana401k | Cell phone reimbursement | DC FSA | Employee assistance program | EquityMid-level Full TimeSan Francisco || Oakland, CA R17h ago
-
Senior-level Full TimeCosta Mesa, California, United States17h ago
-
Senior-level Full TimeCosta Mesa, California, United States17h ago
-
AI Platform Engineer, Training and Inference USD 150K-225KANN indexing | BF16 | DDP | Embeddings | FP8Career growth | Learning opportunitiesSenior-level Full TimeSan Francisco18h ago
-
API Gateway | AWS Glue | Access Control | Apache Airflow | Apache AtlasSenior-level Full TimeFAIRFAX, VA, United States18h ago
-
Senior Enterprise Data Operations Support Engineer (Tier-3 Expert Engineering Support) USD 125K-153KAWS GovCloud | Apache Airflow | Apache Kafka | Apache Spark | Azure GovernmentSenior-level Full TimeFAIRFAX, VA, United States18h ago
-
AI Developer - Model Creation & Full Stack USD 150K-175KAWS | Angular | Azure | CI/CD | D3.jsRemote work | USPS Public Trust Clearance eligibleMid-level Full TimeWork from home, VA, United States R18h ago
-
Enterprise Data Engineering & Operations Lead SME USD 107K-190KAWS GovCloud | Access Control | Apache Kafka | Apache Spark | Azure GovernmentSenior-level Full TimeFAIRFAX, VA, United States18h ago
-
Data/ML Scientist SME USD 105K-150KAWS GovCloud | Anomaly Detection | Apache Spark | Bayesian Causal Inference | Bayesian MethodsMid-level Full TimeFAIRFAX, VA, United States18h ago