Azure DBX
Tasks
- Build CI CD pipelines for Databricks deployments using Azure DevOps
- Build ETL ELT workflows for large volume datasets
- Design data models for analytics reporting and AI ML initiatives
- Design scalable data pipelines using Azure Databricks
- Develop Spark solutions with PySpark and Scala on Azure Databricks
- Implement data governance security monitoring and compliance
- Implement medallion architecture
- Integrate data from APIs databases files and streaming sources
- Mentor junior engineers and conduct technical reviews
- Optimize Spark jobs and cluster configurations for performance and cost
- Troubleshoot production data issues
Perks/Benefits
- 401k plan
- Dental insurance
- Disability insurance
- Health insurance
- Life insurance
- PTO
- Paid Holidays
- Parental leave
- Vision insurance
Skills/Tech-stack
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure DevOps | Azure Key Vault | Azure Monitor | Azure SQL | Azure SQL Database | Azure Synapse | Azure Synapse Analytics | CI/CD | Data Factory | Data Lake Storage Gen2 | Data Modeling | Data Warehousing | Delta Lake | ELT | ETL | Event Hub | Git | Kafka | Key Vault | Log Analytics | PostgreSQL | PySpark | Python | Release Management | SQL | SQL Database | SQL Server | Snowflake | Spark | Spark SQL | Synapse Analytics | Terraform
Education
Related jobs
-
AI | ClickHouse | Confluence | ETL | Git401k match | Educational assistance | Federal Holidays | Health insurance | Paid time offMid-level Full TimeColumbia, Maryland5h ago
-
Data Governance | Data Modeling | Data Structures | Data pipeline | Java401k matching | Educational assistance | Federal Holidays | Healthcare coverage | Paid time offMid-level Full TimeAnnapolis Junction, Maryland5h ago
-
Software Engineer 2 - (Cloud, Analytics, Spark) USD 208K-254KAlgorithm Development | Data Querying | MapReduce | NoSQL | NumPy11 federal holidays | 24 days PTO | 401k 100 Percent Vested | Education assistance | Healthcare coverageSenior-level Full TimeAnnapolis Junction, Maryland; Annapolis Junction, Maryland5h ago
-
Software Engineer 1 - (Python, UI/UX, LLMs) USD 163K-199KAWS | Argo CD | CI/CD | Containerization | Docker401k match | Educational assistance | Federal Holidays | Health insurance | Paid time offMid-level Full TimeAnnapolis Junction, Maryland5h ago
-
Software Engineer 1 - (AI, Python, LLMs) USD 163K-199KAWS | Argo CD | CI/CD | Docker | Helm401k match | Federal Holidays | Health insurance | Paid time offMid-level Full TimeAnnapolis Junction, Maryland5h ago
-
Software Engineer, Machine Learning Infrastructure USD 140K-230KAPI Development | AWS | Airflow | Alerting | CI/CD401k access | Flexible PTO | Health, dental, vision coverage | Paid mealsMid-level Full TimeSan Francisco, CA, US7h ago
-
Anomaly investigation | Bitbake | Bitbucket Pipelines | C# | C++Travel up to 20 percentMid-level Full TimeHerndon, United States8h ago
-
Data Engineer, PDPO - Offense & Defense USD 147K-269KApache Spark | ClickHouse | Data Modeling | Data Warehousing | Distributed SystemsEntry-level Full TimeSan Jose, California, United States9h ago
-
Software Engineer Systems Machine Learning - Frameworks USD 112K-166KC# | C++ | CPU | CUDA | Compiler optimizationEntry-level Full TimeSunnyvale, CA | Bellevue, WA10h ago
-
Research Scientist, ML H/W-SW Codesign USD 177K-251KC# | C++ | Deep learning | GPU Architecture | Hardware accelerationMid-level Full TimeSunnyvale, CA10h ago
-
Robotics Verification & Validation Engineer USD 170K-230KAI Risk | AI Risk Assessment | Automated testing | Bias Mitigation | C++Senior-level Full TimeBurlingame, CA10h ago
-
Senior Software Developer, Machine Learning, Applied AI USD 174K-252KAI Agents | Agentic development | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA; Toronto, ON, Canada10h ago
-
Software Engineer, GenAI Silicon Automation, DeepMind USD 174K-252KABC | Application Programming | Application Programming Interfaces | C++ | CI/CDMid-level Full TimeMountain View, CA, USA10h ago
-
Senior Software Engineer, Search Quality, AI/ML USD 174K-252KAgentic Workflows | Algorithms | Backend Development | C++ | Data StorageSenior-level Full TimeNew York, NY, USA10h ago
-
Data Engineer, gTech Users and Products Engineering USD 106K-150KApache Flume | Data Modeling | Data Pipelines | Data Warehousing | Dimensional dataBonus target | Equity | Health insurance | Paid time off | Retirement planEntry-level Full TimeBoulder, CO, USA10h ago
-
Algorithms | Backend Development | C# | C++ | Code reviewSenior-level Full TimeSunnyvale, CA, USA10h ago
-
Software Engineer III, AI/ML, Keywordless Search Ads USD 147K-210KC++ | Data Processing | Data Storage | Debugging | Distributed ComputingSenior-level Full TimeMountain View, CA, USA10h ago
-
Software Engineer, Google Cloud Storage USD 147K-210KAccess Control | Best practices | Block Storage | C++ | Chaos EngineeringMid-level Full TimeRaleigh, NC, USA; Durham, NC, USA10h ago
-
AI Solutions Engineer USD 123K-178KAI Agents | Azure DevOps | Azure Machine Learning | Bias detection | CI/CDDental insurance | Disability insurance | Medical insurance | Paid time off | Supplemental insuranceSenior-level Full TimeReston, VA, US16h ago
-
Algorithm Engineer USD 150K-170KCI/CD | Code review | Data labeling | Deep learning | Digital SignalIn person office hubs | Remote work flexibilityMid-level Full TimeBoston, MA / Remote R18h ago
-
Analytics Engineer USD 130K-170KCI/CD | DBT | Data Modeling | Data Quality | Data Quality Testing401k match | Flexible time off | Health benefits day one | Paid caregiver leave | Paid parental leaveSenior-level Full TimeUnited States (Remote) R20h ago
-
Senior Principal Software Engineer, Machine Learning USD 222K-453KA/B | A/B Testing | B testing | Canary Deployment | Capacity PlanningHybrid work model | MentorshipSenior-level Full TimeRemote, US R20h ago
-
A/B | A/B Testing | B testing | Distributed Training | Embeddings401k matching | Commuter benefits | Dental insurance | FSA | HSASenior-level Full TimeMountain View, California, United States21h ago
-
CI/CD | Fastlane | Git | Multithreading | SOLIDAsynchronous culture | Bonus | Learning support resources | Remote work | Stock optionsSenior-level Full TimeKansas City, MO, USA R21h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Friendly team atmosphere | Remote distributed settingMid-level Full TimeDenver, CO, USA21h ago