Database Engineer - Databricks
Tasks
- Administer Unity Catalog governance and access
- Develop and optimize Spark transformations
- Diagnose and resolve pipeline failures and data quality issues
- Document pipeline architecture and procedures
- Maintain orchestration across Databricks Workflows and Apache Airflow
- Maintain runbooks and operational documentation
- Manage and tune SQL Server source and target databases
- Monitor and control DBU consumption and cluster cost
- Own Databricks operations and reliability
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Airflow | Auto Loader | Databricks | Databricks Notebooks | Databricks SQL | Databricks Workflows | Delta Lake | PySpark | Python | SQL | SQL Server | SaaS API | Spark | Structured Streaming | Unity Catalog
Education
N/A
Roles
Related jobs
-
Telemetry Data Engineer USD 110K-128KAnomaly Detection | Azure Data | Azure Data Explorer | Data Engineering | Data Explorer401k retirement plan | Medical/Dental/VisionMid-level ContractRedmond, Washington, USA R8h ago
-
Automated testing | Build Automation | C++ | CI/CD | Firmware401k | Employee assistance program | Health savings account | Life insurance | Long-term disabilityMid-level ContractDallas, Texas, USA R9h ago
-
Principal, Data Analyst/BI Engineer USD 108K-145KAWS | BigQuery | Cloud platform | Data Governance | Data Modeling401k match | Healthcare Dental Vision | Paid Holidays | Paid time off | Parental leaveSenior-level Full TimeUnited States R9h ago
-
Senior AI Engineer USD 144K-236KA/B | A/B Testing | B testing | Classification | Data MiningFlexible work arrangement | Health and wellness programs | Time off for employeesSenior-level Full TimeMountain View, CA, United States R9h ago
-
Staff AI Engineer USD 175K-287KA/B | A/B Testing | B testing | Classification | Data MiningHealth and wellness programs | Hybrid work | Mentorship | Time offSenior-level Full TimeMountain View, CA, United States R10h ago
-
Staff AI Engineer USD 198K-260KAgentic Workflows | Containerization | Docker | Experiment design | Generative AI401k plan participation | Annual performance bonuses | Company equity grants | Disability insurance | Life insuranceSenior-level Full TimeNew York, New York, United States R10h ago
-
Automated testing | C++ | CI/CD | Embedded Linux | Firmware401k | Employee assistance program | Health savings account | Life insurance | Long-term disabilityMid-level ContractTampa, Florida, USA R10h ago
-
Automated testing | C++ | CI/CD | Firmware Build | Firmware Testing401k | Employee assistance program | Health savings account | Life insurance | Long-term disabilityMid-level ContractAtlanta, Georgia, USA R10h ago
-
Data Engineer USD 130K-150KAlerting | Apache Spark | Azure | Azure DevOps | CI/CDFree on-site gym | Free parking | Free pet insurance | Hybrid remote work | Medical/Dental/Vision insuranceMid-level Full TimeSeattle, United States R12h ago
-
AWS | AWS Glue | AWS Lambda | Amazon EMR | Amazon RedshiftCollaborative environment | Mentorship and growth | Mission-driven impactEntry-level Full TimeVaughan, Ontario; Remote, Canada; Remote, US … R14h ago
-
BE CAD Physical Impl and AI Engineer USD 401KAI for Design | AI for Design Optimization | ASIC | Automation | Back End CAD401k match | Dental plan | Educational assistance | Health benefits | Hybrid workSenior-level Full TimeAustin,Texas,United States R15h ago
-
Senior-level Full TimeAnnapolis Junction, United States R15h ago
-
Backend & Data Developer (.NET Core / Cloud) USD 115K-150KAWS | AWS Lambda | Alteryx | Amazon Athena | Amazon RedshiftSenior-level Full TimeCenter, Center District, IL R16h ago
-
Cloud infrastructure | Distributed Systems | GPU | JAX | Machine LearningMid-level Full TimeUnited States R1d ago
-
LLM Application Engineer USD 130K-175KAPIs | Agent Frameworks | Databases | Distributed Systems | EvaluationMid-level Full TimeUnited States R1d ago
-
Analytics Engineer USD 115K-149KAirflow | BI Semantic Layer | CI/CD | DAX | DBT401k plan | Catered lunches | Dental insurance | Medical insurance | Onsite fitness centerSenior-level Full TimeIndianapolis, IN R1d ago
-
Specialist Solutions Architect - Data Engineering & Warehousing (Digital Native Business) USD 180K-247KAWS | Anomaly Detection | Apache Kafka | Automated Ingestion | AzureConference presentations | Hackathons | Mentorship | Remote work | Training programsSenior-level Full TimeUnited States R1d ago
-
Senior AWS Data Engineer USD 153K-207KAPI Integration | AWS Batch | AWS CloudTrail | AWS Glue | AWS LambdaRemote workSenior-level Full TimeMountain View, CA, US R1d ago
-
Senior AWS Data Engineer USD 153K-207KAPI | AWS Batch | AWS Glue | AWS Lambda | AWS Step FunctionsContinuous learning | Flexible work hours | Mentorship | Remote workSenior-level Full TimeAtlanta, GA, US R1d ago
-
Machine Leaning Performance Engineer (Inference) USD 200K-300KC++ | CUBLAS | CUDA | CUDA C | CutlassFitness Events | Free meals | Hybrid work options | Paid time off | Volunteer opportunitiesMid-level Full TimeNew York R1d ago
-
Machine Learning Research Engineer USD 200K-300KAI Agents | API Design | Automation | Dask | Data pipelineHybrid work | Paid time off | Paid wellness expenses | Volunteer opportunities | Workshops and continuous learningMid-level Full TimeNew York R1d ago
-
Lead Analytics Engineer USD 138K-177KAWS Glue | AWS Glue Data Catalog | Amazon Athena | Amazon RDS | Amazon RedshiftRemote workSenior-level Full TimeAtlanta, GA, US R1d ago
-
Junior Data Engineer USD 85K-90KAWS EMR | AWS Glue | AWS Lambda | AWS Step Functions | Amazon AthenaContinuous learning | Flexible work arrangement | Remote workEntry-level Full TimeMountain View, CA, US R1d ago
-
Junior Data Engineer USD 85K-90KAWS Glue | AWS Lambda | AWS Step Functions | Amazon Athena | Amazon CloudWatchContinuous learning | Flexible schedule | Remote workEntry-level Full TimeAtlanta, GA, US R1d ago
-
General Pipeline - FullStack Applied AI Engineer USD 150K-250KAWS | Access Control | Asynchronous programming | Authentication | AuthorizationSenior-level Full TimeUnited States R1d ago