Senior Data Engineer (Azure & Databricks)
Bloomington, Minnesota, United States
USD 120K-170K Mid-level Contract Full Time
Tasks
- Automate deployments using Azure DevOps CI CD
- Build reusable Databricks libraries and utilities
- Collaborate with analytics teams and stakeholders to deliver datasets
- Design data pipelines using Azure Databricks and PySpark
- Enforce schema validation and logging
- Implement medallion architecture bronze/silver/gold
- Ingest data from SQL Server Managed Instance
- Integrate external data sources using REST API and SFTP
- Manage Spark cluster and Databricks serverless compute
- Manage version control and environment deployments
- Monitor and troubleshoot Azure Data Factory pipelines
- Optimize data pipelines using Delta Lake and SQL
- Run parameter driven Databricks jobs
- Support AI and machine learning data readiness
Perks/Benefits
Skills/Tech-stack
Azure Data | Azure Data Factory | Azure Databricks | Azure DevOps | CI/CD | Data Factory | Databricks Serverless | Delta Lake | Managed Instance | PySpark | REST API | SFTP | SQL | SQL Server | SQL Server Managed Instance | Spark | Unity Catalog
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
AI Engineer USD 109K-140KAgent Orchestration | Computer Vision | Data Validation | Data extraction | Document ClassificationMid-level Full TimeMorristown, NJ, United States6h ago
-
Summer 2026 Data Engineer USD 41K-50KAPIs | Agile | Azure Data | Azure Data Factory | Azure Data LakeExposure to real-world projects | Learning and development opportunities | MentorshipEntry-level InternshipBoston, MA, United States6h ago
-
Summer 2026 Data Engineer USD 41K-50KAPIs | Agile | Azure Data | Azure Data Factory | Azure Data LakeMentorshipEntry-level InternshipBoston, MA, United States6h ago
-
Data Engineer USD 126K-208KAPI Integration | Airflow | Amazon Web Services | BigQuery | CCPADEI initiatives | Dental benefits | Employee rewards program | Medical benefits | Mental health supportMid-level Full TimeRemote, United States R6h ago
-
Alerting | Ansible | Bash | CI/CD | CephRemote workSenior-level Full TimeUnited States, United States R8h ago
-
Ansible | Bash | CI/CD | CentOS | CephContract-to-hire | No sponsorship | Remote workSenior-level Full TimeUnited States, United States R8h ago
-
HPG Big Data Engineer / Senior-Level USD 119K-164KAgile | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure FunctionsSenior-level Full TimeNashville, TN, United States8h ago
-
Machine Learning Engineer USD 131K-178KAWS | Cassandra | Convolutional Neural Networks | Data Lakes | Data PipelinesMid-level Full TimeRemote, NY, US R9h ago
-
Case design | Data Integrity | Database Testing | Database optimization | Defect ReportingSenior-level Full TimeAustin, TX R10h ago
-
Amazon S3 | Data Engineering | Data Modeling | Data Pipelines | Data QualitySenior-level Full TimeNew York11h ago
-
Amazon S3 | Automation | Data Engineering | Data Modeling | Data Pipelines401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceSenior-level Full TimePrinceton11h ago
-
Senior Databricks Forward Deployed Engineer - GPS USD 119K-198KAPI Integration | AWS | Airflow | Azure | CI/CDTravelSenior-level Full TimeArlington/Rosslyn, Virginia, United States; Atlanta, Georgia, …11h ago
-
Lead Databricks Forward Deployed Engineer - GPS USD 189K-372KAPI Integration | AWS | Airflow | Apache Spark | AzureSenior-level Full TimeArlington/Rosslyn, Virginia, United States; Atlanta, Georgia, …11h ago
-
Lead AI and Data Solutions Engineer II USD 137K-229KAmazon Web Services | Apache Spark | Application Programming | Application Programming Interfaces | Cloud ComputingSenior-level Full TimeSacramento, California, United States; Tempe, Arizona, …11h ago
-
AI & Data Supply Chain Engineer USD 116K-229KCapacity modeling | Data Modeling | Demand Planning | Enterprise Resource Planning | Inventory PlanningTravel 50%Mid-level Full TimeSeattle, Washington, United States11h ago
-
TikTok Shop - E-commerce Anti-Fraud Data Scientist USD 156K-296KA/B | A/B Testing | Analytics | B testing | Big DataMid-level Full TimeSeattle, Washington, United States11h ago
-
GTM Applied AI Architect, Google Cloud USD 153K-222KAgent Development | Agent Development Kit | Cloud platform | Function Calling | GeminiSenior-level Full TimeAustin, TX, USA; Boulder, CO, USA12h ago
-
Agentic AI Engineer USD 103K-158KAI Search | Agent Orchestration | Anthropic Claude | App Service | Autogen401k match | Flexible paid time off | Hybrid work flexibility | Medical/Dental/Vision | Paid parental leaveMid-level Full TimeGolden, CO, US14h ago
-
Senior Data Engineers USD 123K-215KAWS | CI/CD | Data Guard | Data Modeling | Data QualityCareer development training | Company retirement match | Counseling support | Disability benefits | Free financial coachingSenior-level Full TimePhoenix, AZ, United States19h ago
-
Senior AI Data Engineer USD 155K-185KApache Airflow | Apache Spark | Azure Synapse | BigQuery | ClickHouseEmployer paid Medical Dental Vision Insurance | Flexible paid time off | Manager check ins | Paid cell phone and service | Paid parental leaveSenior-level Full TimeRemote - United States R21h ago
-
Software Engineer, Data Infrastructure USD 155K-185KAWS | Apache Airflow | Apache Flink | Apache Kafka | Apache SparkMid-level Full TimeMountain View, CA21h ago
-
Senior Staff Software Engineer - Data Platform USD 200K-250KAWS Glue | AWS IAM | Amazon EMR | Amazon S3 | AmundsenDevelopment dollars | Employee stock purchase program | Family-forming benefits | Financial coaching | Flexible time offSenior-level Full TimeRemote, USA R21h ago
-
Senior Staff Software Engineer - Data Platform USD 200K-250KAWS EMR | AWS Glue | AWS IAM | AWS S3 | Apache AirflowDevelopment dollars | Financial coaching | Flexible remote work | Flexible time off | Free therapy sessionsSenior-level Full TimeRemote, USA R21h ago
-
AWS | Airflow | Ansible | Apache Spark | ArgoCDAdditional vacation days | English courses | Flexible remote options | Health insurance | Hybrid work optionsMid-level Full TimeGeorgia22h ago
-
Fullstack Software Engineer, Applied AI USD 130K-500KAsynchronous programming | Component design | Data Pipelines | JavaScript | Multi-tenantDental insurance | Free gym membership | Health insurance | Laundry reimbursement | Meal stipendMid-level Full TimeSan Francisco23h ago