Senior Data Engineer (Azure & Databricks)
Bloomington, Minnesota, United States
USD 120K-170K Mid-level Contract Full Time
Tasks
- Automate deployments using Azure DevOps CI CD
- Build reusable Databricks libraries and utilities
- Collaborate with analytics teams and stakeholders to deliver datasets
- Design data pipelines using Azure Databricks and PySpark
- Enforce schema validation and logging
- Implement medallion architecture bronze/silver/gold
- Ingest data from SQL Server Managed Instance
- Integrate external data sources using REST API and SFTP
- Manage Spark cluster and Databricks serverless compute
- Manage version control and environment deployments
- Monitor and troubleshoot Azure Data Factory pipelines
- Optimize data pipelines using Delta Lake and SQL
- Run parameter driven Databricks jobs
- Support AI and machine learning data readiness
Perks/Benefits
Skills/Tech-stack
Azure Data | Azure Data Factory | Azure Databricks | Azure DevOps | CI/CD | Data Factory | Databricks Serverless | Delta Lake | Managed Instance | PySpark | REST API | SFTP | SQL | SQL Server | SQL Server Managed Instance | Spark | Unity Catalog
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
Data Analytics & Engineering Opportunities USD 65K-105KHive | Microstrategy | MySQL | Oracle | Python401k with firm profit share | Dental insurance | Disability insurance | Firm paid holidays | Flexible spending accountEntry-level Full TimeWashington, DC, United States4h ago
-
Mid-Level Data Engineer USD 90K-98KAPI Development | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageRemote workMid-level Full TimeWork from home, VA, United States R6h ago
-
Senior Data Engineer USD 165K-180KAPIs | Anomaly Detection | Azure | Azure Data | Azure Data FactorySenior-level Full TimeWork from home, VA, United States R6h ago
-
Senior DevOps Engineer ID63545 USD 135K-185KAWS | Apache Airflow | ArgoCD | Azure | BigQueryFlextime | Growth roadmaps | Mentorship | Office work options | Remote work optionsSenior-level Full TimeMiami, United States8h ago
-
BigQuery | Curriculum Development | Data Governance | Data Lakehouse | Data ModelingAsynchronous hiring process | Part-time hoursSenior-level Part TimeBoston, US8h ago
-
Senior Director, AI / Machine Learning Software Engineer USD 136K-300KApache Flink | Apache Spark | CI/CD | Data Lineage | Data PrivacyHealth benefits | Paid leave | Paid volunteer timeSenior-level Full TimeNew York, NY, United States9h ago
-
Data Pipelines | Full Stack | Full-Stack Development | Machine Learning | PythonSenior-level Full TimeSan Jose, California, United States10h ago
-
Associate Principal, Trust and Safety, GenAI USD 142K-205KAbuse prevention | Cybersecurity | Dashboards | Data Transformation | Data VisualizationMid-level Full TimeWashington D.C., DC, USA; Seattle, WA, …10h ago
-
Customer Engineer, Data Analytics, Google Cloud USD 153K-222KBatch Processing | Big Data | Cloud Architecture | Cloud platform | Customer RequirementsSenior-level Full TimeSunnyvale, CA, USA11h ago
-
Cloud Data and AI Engineer, Professional Services USD 127K-183KC++ | Capacity Planning | Cloud Databases | Data Migration | Data PipelinesTravel up to 30 percentMid-level Full TimeReston, VA, USA11h ago
-
Data Scientist USD 67K-150KA/B | A/B Testing | B testing | Clustering | Drift monitoring401k plan | AD and D insurance | Child Life Insurance | Dental insurance | Educational Assistance PlanMid-level Full TimeUnited States12h ago
-
Senior Data Engineer USD 129K-165KAWS | Airflow | CI/CD | Data Modeling | Django401k | Half-day Fridays | Medical/Dental/Vision insurance | Paid Holidays | Remote workSenior-level Full TimeChicago, IL, US R20h ago
-
Chase Modeling - Applied AI ML Senior Associate USD 171K-215KClassification | Cloud Computing | Hadoop | Language Processing | Machine LearningBackup childcare | Financial coaching | Health care coverage | Mental health support | Onsite health and wellness centersSenior-level Full TimeChicago, IL, United States21h ago
-
Staff Machine Learning Engineer (Pricing) USD 215K-322KA/B Testing | API Design | AWS | Automated retraining | B testing401k matching | Dental insurance | Family planning assistance | Flexible time off | Healthcare benefitsSenior-level Full TimeSan Francisco, CA21h ago
-
Solutions Engineer USD 150K-180KAI | Apache Flink | Apache NiFi | Apache Spark | Applied ScienceContinued Career Development | Employee resource groups | Flexible work from home | Generous paid time off | Paid volunteer timeMid-level Full TimeUS-California-Remote, United States R22h ago
-
Senior-level Full TimeAEP Headquarters, United States22h ago
-
Software Engineer - Analytics & Development USD 103K-218KAmazon Web Services | Apache Airflow | Apache Spark | Bash | CI/CDContinuing education | Family support | Financial benefits | Flexible time off | HealthcareSenior-level Full TimeCXN ANNAPOLIS JUNCTION MD, United States22h ago
-
Senior Analytics Engineer USD 189K-316KAI | API Integration | C Sharp | Data Quality | Data WarehouseSenior-level Full TimeDurham Blackwell Street, United States22h ago
-
Sr Data Visualization Engineer USD 110K-155KAgile | Data Modeling | Data Normalization | Data Visualization | OracleChallenging projects | Professional developmentSenior-level Full TimeTerre Haute, United States22h ago
-
Alteryx | Dimensional Modeling | MS Excel | MS PowerPoint | MS SQLSenior-level Full TimeHeadquarters 4, United States22h ago
-
Full-Stack AI Software Engineer USD 120K-150KAWS | Agile | Algorithms | Azure | C#401k company match | Company holidays | Deferred compensation plan | Dental insurance | Disability insuranceMid-level Full TimeCRC - Alpharetta, GA 3460 Preston …22h ago
-
Digital Technical Specialist (Associate/Sr Associate) - Heathcare Data, Analytics & Automation - Remote USD 120K-171KAI | Automation | Data Transformation | Data integration | EHRDental insurance | Healthcare coverage | Remote work | Travel opportunity | Vision insuranceSenior-level Full TimeChicago - 550 Van Buren, United … R22h ago
-
Data Engineer USD 150K-175KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure SQLMid-level Full TimeNew York, United States22h ago
-
Senior Staff Data Engineer USD 155K-262KAWS | Airflow | Apache Flink | Apache Spark | Azure401k match | Accidental death and dismemberment | Dental insurance | Employee assistance program | Equity eligibilitySenior-level Full TimeUnited States, United States22h ago
-
Senior-level Full TimeAZ - Chandler, United States22h ago