Lead Engineer, Big Data - Databricks - Remote
Tasks
- Build data pipelines (batch and real-time)
- Create data quality frameworks and data validation
- Design azure data platform architecture
- Develop BI solutions with Power BI and Tableau
- Ensure HIPAA compliance and data governance
- Implement machine learning solutions
- Integrate Azure data platform with low code systems
- Lead end to end data engineering initiatives
- Mentor team members
- Model data for self service BI
- Plan and execute projects with Agile
- Resolve technical issues with support and vendor teams
Perks/Benefits
- N/A
Skills/Tech-stack
ARM Templates | Agile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Databricks | Azure DevOps | Azure Event | Azure Event Hubs | Azure Machine Learning | Azure Synapse | Batch Processing | CI/CD | Data Factory | Data Governance | Data Lake Storage | Data Quality | Data Validation | Event Hubs | GitHub Copilot | HIPAA | Infrastructure as Code | Kafka | Machine Learning | O365 | Pega | Power BI | PySpark | Python | Real Time | Real-time Streaming | SQL | Salesforce | Scala | Spark Streaming | Tableau | Terraform | “as-code”
Education
Bachelor of Arts | Bachelor of Engineering | Bachelor of Science
Roles
Related jobs
-
Data Engineer USD 220K-226KC++ | Data Exploration | Data Modeling | Data Visualization | Data pipelineHybrid scheduleMid-level Full TimeMountain View, CA, USA R5h ago
-
Senior Software Engineer USD 193K-253KC++ | Cause analysis | Debugging | Machine Learning | Model DeploymentHybrid scheduleSenior-level Full TimeNew York, NY, USA R5h ago
-
Software Engineer USD 149K-175KAlgorithms | C# | C++ | Cause analysis | Code ReviewsBenefits | Hybrid scheduleEntry-level Full TimeMountain View, CA, USA R5h ago
-
Senior AI Engineer USD 130K-150KAI Governance | AI Services | AI/ML | API Integration | AWS AI401k match | Dental insurance | Disability insurance | Employee assistance program | Health insuranceSenior-level Full TimeCentennial, CO; Westminster, CO R16h ago
-
Entry-level Full TimeOakland, CA, US, 94612 R16h ago
-
Machine Learning Engineer, Relevance and Personalization USD 166K-195KA/B | A/B Testing | Apache Airflow | Apache Kafka | Apache SparkMid-level Full TimeUnited States R16h ago
-
Data Platform Software Engineer USD 125K-194KCI/CD | Cloud Computing | Data Lakes | Data Modeling | Data Structures401k match | ESPP | Family leave programs | Flexible time off | Health plansEntry-level Full TimeSan Diego, CALIFORNIA, United States R16h ago
-
Data Engineer USD 150K-160KApache Iceberg | Cloud Data | Cloud Data Warehouses | Data Lakes | Data Warehouses401k | Employee assistance program | Health spending account | Life insurance | Long-term disabilityMid-level ContractSunnyvale, California, USA R17h ago
-
Senior Machine Learning (ML) Engineer USD 175K-200KAI Search | Artificial Intelligence | Attention Mechanisms | Autogen | Azure AI401k plan | Dental insurance | Flexible paid time off | Flexible work hours | Medical insuranceSenior-level Full TimeSeattle, WA R18h ago
-
Software Engineer, Data Runtime USD 29KAWS | Admission Policy | Arrow | Azure | Benchmarking401k | Additional company holidays | Enhanced parental leave | Generous vacation policy | Holiday breakSenior-level Full TimeSan Francisco, US (Hybrid) R18h ago
-
Staff Software Engineer, Fleet Optimization USD 251K-310KAlgorithm Design | Backend Development | C++ | Constraint Optimization | Control SystemsSenior-level Full TimeSan Francisco, CA ; Mountain View, … R18h ago
-
Senior AI Engineer USD 125K-197KAWS | AWS CDK | Agent Orchestration | Agentic Systems | Artificial IntelligenceHybrid workSenior-level Full TimeOverland Park, KS (Hybrid) R18h ago
-
Apache Iceberg | Apache Spark | Avro | Concurrency | Data Ingestion401k matching | ESPP | Family leave programs | Flexible spending accounts | Flexible time awaySenior-level Full TimeSan Diego, CALIFORNIA, United States R18h ago
-
AI Engineer I USD 90K-119KAPI Integration | Azure | CI/CD | Docker | Embeddings401k match | Dental insurance | Holiday pay | Medical insurance | Paid time offEntry-level Full TimeRemote, CA, USA R18h ago
-
Senior Machine Learning Engineer, Multimodal Perception USD 213K-263KClosed Loop | Closed-loop simulation | Computer Vision | Foundation Models | Imitation LearningSenior-level Full TimeMountain View, California, USA R19h ago
-
Senior Machine Learning Engineer, Multimodal Perception USD 213K-263KActive Learning | Audio | Auto-labeling | Camera | Data CurationSenior-level Full TimeMountain View, California, USA R19h ago
-
API | AWS | Automated testing | Cloud Computing | Code generation401k | Dental insurance | Discounts and perks | Fully remote | Medical insuranceSenior-level Full TimeUniversal City, CALIFORNIA, United States R19h ago
-
Staff Backend Engineer, Data Platform USD 140K-175KAsynchronous Service Design | Audit Logging | BigQuery | Cloud Run | Cloud Storage401k match | Company holidays | Dental insurance | Flexible paid time off | Health insuranceSenior-level Full TimeElizabeth, NJ R19h ago
-
Senior Software Engineer (Data Platform) USD 185K-185K.NET | AWS | Authentication | Authorization Code | Authorization Code GrantSenior-level Full TimeUnited States (EST) R19h ago
-
Lead GenAI Engineer USD 150K-170KAgile | Azure | Cloud platform | Containerization | Cybersecurity401k | Employee assistance program | Health savings account | Life insurance | Long-term disabilitySenior-level ContractMinneapolis, Minnesota, USA R19h ago
-
Staff, Analytics Engineer, GTM Data Science & Analytics USD 155K-194KCI/CD | Cause analysis | DBT | Data Governance | Data ModelingCorporate Bonus Plan | Equity plan | Generous time off | Healthcare | Paid personal time offSenior-level Full TimeRemote - US R20h ago
-
Senior Engineer, ML Data Services USD 149K-198KAPI Gateway | AWS | AWS Step Functions | Apache Airflow | Apache Beam401k match | Dental insurance | Health savings account | Life insurance | Medical insuranceSenior-level Full TimeBoston, Massachusetts, United States; Las Vegas, … R20h ago
-
A/B | A/B Testing | Agent workflows | B testing | Celery401k match | Family leave | Medical/Dental/Vision insurance | Unlimited vacationSenior-level Full TimeRemote (United States) R20h ago
-
Senior Analytics Engineer USD 155K-170KBigQuery | DBT | Data Governance | Data Modeling | Data Monitoring401k retirement savings | EAP | Fully remote | Hardware provided | Healthcare (Medical Dental Vision)Senior-level Full TimeUnited States R21h ago
-
AI Automation Engineer USD 85K-105KAPI Development | AWS | Autogen | Azure | ChromaDB401k employer match | Employee Assistance Program Tuition Reimbursement | Health, dental, vision coverage | Learning and development programs | Life insuranceMid-level Full TimeChicago, Illinois, United States R21h ago