Databricks Engineer
Milpitas, United States
USD 175K-250K (estimate) Senior-level Full Time
Tasks
- Build ELT workflows
- Design data pipelines with Databricks
- Develop ETL workflows
- Develop PySpark applications
- Document technical solutions
- Ensure data quality security governance compliance
- Implement CI/CD pipelines
- Implement Delta Lake transformations
- Implement medallion architecture
- Integrate data from multiple sources
- Monitor troubleshoot optimize production pipelines
- Optimize Spark jobs performance
- Process batch data
- Process streaming data
- Write Spark SQL queries
Perks/Benefits
- N/A
Skills/Tech-stack
Amazon Web Services | Apache Spark | Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure DevOps | Azure Key Vault | Azure Synapse | Azure Synapse Analytics | CI/CD | Cloud platform | Data Factory | Data Lake | Data Lake Architecture | Data Lake Storage Gen2 | Databricks | Delta Lake | Delta Live Tables | Delta Live) | ELT | ETL | Event Hubs | Git | GitHub Actions | Google Cloud | Google Cloud Platform | Kafka | Key Vault | MLflow | PySpark | Python | SQL | Spark SQL | Synapse Analytics | Terraform | Unity Catalog | Web Services
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Senior Data Engineer ID71671 USD 155K-170KAWS Kinesis | Apache Airflow | Apache Kafka | Automated testing | CI/CDAnnual learning budget | Flexible hours | Internal tech talks | Mentorship | People focused supportSenior-level Full TimeFort Lauderdale, United States3h ago
-
Senior Data Engineer ID71671 USD 155K-170KAWS Kinesis | Apache Airflow | Apache Kafka | Automated testing | CI/CDFlexible hours | Learning budget | Mentorship | People focused support | Remote workSenior-level Full TimeBaltimore, United States3h ago
-
Senior Data Engineer ID71671 USD 155K-170KAWS | AWS Kinesis | Apache Airflow | Apache Kafka | Automated testingFlexible hours | Internal TechTalks | Learning budget | Mentorship | Performance reviewsSenior-level Full TimePort Charlotte, United States3h ago
-
Senior Data Engineer ID71671 USD 155K-170KAmazon Web Services | Apache Airflow | Apache Kafka | Automated testing | Batch ProcessingFlexible hours | Internal TechTalks | Learning budget | Mentorship | No micromanagementSenior-level Full TimeAustin, United States3h ago
-
Project Data Scientist and Analytics USD 125K-208KAPI Development | APIs | Cloud Computing | Dashboards | Data ArchitectureSenior-level Full TimeSan Diego, California, United States4h ago
-
Data Generation | Data Processing | DeepSpeed | Direct Preference Optimization | Distributed TrainingEntry-level Full TimeSan Jose, California, United States4h ago
-
AI Software Engineer, Systems ML - Wearables AI USD 146K-208KBias Mitigation | C plus plus | C# | Deep learning | Device Machine LearningSenior-level Full TimeSeattle, WA5h ago
-
Data Engineer/Sr Data Engineer, IT Analytics USD 38K-55KAgile | Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake401 k program | Dental insurance | Discounts | Employee assistance program | Health benefitsSenior-level Full TimeFort Worth, TX, US8h ago
-
Embedded Software Engineer - E2 USD 91K-169KAssembly | C# | C++ | Cyber techniques | Cyber tools401k match | Dental | EAP | Education assistance | Flexible spending accountsMid-level Full TimeHanover, MD, US11h ago
-
Senior Power BI Software Engineer - Clearance Required USD 166K-309KBI Desktop | Containerization | DAX | Database Design | Database optimization401k match | EAP | Education assistance | Flexible spending accounts | Flexible work arrangementsSenior-level Full TimeAnnapolis Junction, MD, US11h ago
-
Embedded Software Engineer Stf - E4 USD 120K-224KAgile | Atlassian | C# | C++ | CI/CD401k match | EAP | Education assistance | Flexible work schedule | HolidaysMid-level Full TimeGrand Prairie, TX, US11h ago
-
Active Learning | Calibration | Causal Inference | Constrained optimization | Contextual BanditsSenior-level Full TimeAustin, TX14h ago
-
Staff Machine Learning Engineer USD 165K-236KClass imbalance | Docker | Error Analysis | Feature Engineering | Feature Labeling401k | Bonus | Equity | Generous vacation time | Medical/Dental/Vision insuranceSenior-level Full TimeRemote USA R15h ago
-
Business Intelligence - Data Engineer 133-2015 USD 80K-115KData Transformation | Data pipeline | Database Design | Dependency management | ETLBackground checkMid-level Full TimeTulsa, OK, US15h ago
-
Dynamic Scheduler DSP Algorithm Engineer USD 125K-175KCombinatorial Optimization | Convex Optimization | MATLAB | Optimization Solvers | PythonMid-level Full TimeLanham, Maryland, United States15h ago
-
AWS | Azure | CI/CD | Container Networking | Container networking interface401k company match | ESPP | Family leave programs | Flexible spending accounts | Flexible time away planSenior-level Full TimeSan Diego, CALIFORNIA, United States R16h ago
-
AWS | Azure | CI/CD | Container Networking | Containers401k match | ESPP | Family leave programs | Flexible spending accounts | Flexible time awaySenior-level Full TimeSanta Clara, California, United States R16h ago
-
Senior-level Full TimeNew York, NY, United States16h ago
-
Software Engineer - AIM Virtualization USD 157K-230KData Ingestion | Data egress | Database Internals | Functional Programming | Query OptimizationSenior-level Full TimeUS-CA-Menlo Park R16h ago
-
Principal Software Engineer- Embedded OS USD 125K-150KAgile | Artifactory | Atlassian Bamboo | Atlassian Confluence | BSPCareer growth opportunities | Comprehensive benefits | MentorshipSenior-level Full TimeDenver, Colorado, United States16h ago
-
Senior-level Full TimeDenver, CO - Hybrid; New York, … R16h ago
-
Staff+ Software Engineer, Safeguards Data USD 320K-485KAWS | Access Control | Cloud platform | Data Governance | Data LineageFlexible working hours | Generous vacation | Hybrid office policy | Optional equity donation matching | Parental leaveSenior-level Full TimeSan Francisco, CA | New York … R17h ago
-
Specialist Solutions Architect - AI/ML USD 180K-247KAI guardrails | AWS | Agent Orchestration | Artificial Intelligence | AzureSenior-level Full TimeUnited States17h ago
-
Software Engineer, Applied AI USD 170K-300KAWS | Agents | Benchmarking | Datadog | ECSFree coachingSenior-level Full TimeNew York R17h ago
-
Senior Solution Engineer USD 138K-181KAnalytics | Cloud Computing | Data Lake | Data Warehousing | Data fabricTravel opportunitiesSenior-level Full TimeUS-GA-Remote R18h ago