Senior Data Engineer (Azure & Databricks)
Bloomington, Minnesota, United States
USD 120K-170K Mid-level Contract Full Time
Tasks
- Automate deployments using Azure DevOps CI CD
- Build reusable Databricks libraries and utilities
- Collaborate with analytics teams and stakeholders to deliver datasets
- Design data pipelines using Azure Databricks and PySpark
- Enforce schema validation and logging
- Implement medallion architecture bronze/silver/gold
- Ingest data from SQL Server Managed Instance
- Integrate external data sources using REST API and SFTP
- Manage Spark cluster and Databricks serverless compute
- Manage version control and environment deployments
- Monitor and troubleshoot Azure Data Factory pipelines
- Optimize data pipelines using Delta Lake and SQL
- Run parameter driven Databricks jobs
- Support AI and machine learning data readiness
Perks/Benefits
Skills/Tech-stack
Azure Data | Azure Data Factory | Azure Databricks | Azure DevOps | CI/CD | Data Factory | Databricks Serverless | Delta Lake | Managed Instance | PySpark | REST API | SFTP | SQL | SQL Server | SQL Server Managed Instance | Spark | Unity Catalog
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
Data Engineer-Secret Clearance Required USD 100K-127KAWS | AWS Glue | AWS Redshift | Azure | Azure Data401k match | Bereavement leave | Disability insurance | Employee assistance program | Employee discount programSenior-level Full TimeRemote - Nationwide, United States R9h ago
-
Lead Data Engineer – Snowflake USD 170K-216KAmazon S3 | Apache Airflow | Azure Data | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeUnited States13h ago
-
IT Data Engineer USD 60K-65KANSI X12 | AWS | Azure | Azure Data | Azure Data Factory401k retirement plan | Disability coverage | Employee assistance program | Flexible spending account | Medical/Dental/Vision insuranceMid-level Full TimeUS - Remote R14h ago
-
Data Engineer III USD 134K-161KAccess Control | Analytics Cloud | Data Governance | Data Marts | Data Quality401k | Adoption Assistance | Dental insurance | Education reimbursement | FSASenior-level Full TimeIrvine, CA14h ago
-
GenAI Engineer III USD 110K-218KArtificial Intelligence | Containerization | Data Analysis | Data Pipelines | DockerProfessional developmentSenior-level Full TimeArlington/Rosslyn, Virginia, United States14h ago
-
Generative AI Engineer III USD 110K-218KArtificial Intelligence | Data Analysis | Data Pipelines | Docker | KubernetesDiscretionary annual incentive program | Mentorship | Professional developmentSenior-level Full TimeAustin, Texas, United States; Boston, Massachusetts, …14h ago
-
Lead Generative AI Data Engineer III USD 159K-265KArtificial Intelligence | Data Pipelines | Docker | Generative AI | KubernetesSenior-level Full TimeAustin, Texas, United States; Boston, Massachusetts, …14h ago
-
Lead Generative AI Data Engineer III USD 159K-265KArtificial Intelligence | Data Pipelines | Generative AI | Language Models | Language ProcessingSenior-level Full TimeAtlanta, Georgia, United States; New York, …14h ago
-
Generative AI Data Engineer III USD 131K-218KAlgorithm Development | Artificial Intelligence | Data Analysis | Data Modeling | Data PipelinesMentorship | Travel opportunitiesSenior-level Full TimeAtlanta, Georgia, United States; New York, …14h ago
-
Integration Engineer II - AI and Python Developer USD 86K-170KAI Ops | AWS Bedrock | Azure | CI/CD | Fine TuningAgile team collaboration | Mentorship | Professional developmentMid-level Full TimeAtlanta, Georgia, United States; Baltimore, Maryland, …14h ago
-
Sr. AI Engineer USD 150K-175KAccess Control | Agentic AI | Auditability | CI/CD | Cloud platform401k | Dental insurance | Expense reimbursement for internet costs | Life insurance | Medical insuranceSenior-level Full TimeRemote, USA, United States R14h ago
-
Senior Data Engineer, Strategic Partnerships & IMPACT360 USD 154K-209KAWS Glue | AWS Lake Formation | AWS Lambda | Amazon Athena | Apache AirflowSenior-level Full TimeSeattle, Washington, USA1d ago
-
AI Engineer, Global Operations USD 107K-143KArtificial Intelligence | ETL | Langchain | Machine Learning | PythonContinuous learning | Flexible work options | Health, wellness, and retirement plans | Professional growth opportunitiesMid-level Full TimeCalifornia - Remote, United States R1d ago
-
Senior Cloud Database Engineer USD 107K-185KAWS | Amazon RDS | CI/CD | Capacity Planning | CassandraSenior-level Full TimeTwo Destiny Way, Westlake TX, United …1d ago
-
Sr. Data Engineer USD 130K-203KData Modeling | Data Pipelines | Databases | ETL | HDFS401k retirement savings plan | CVS store discount | Discount programs | Education assistance | Employee stock purchase planSenior-level Full TimeWoonsocket-1 CVS Drive, United States1d ago
-
AWS | Access Control | Apache Spark | Azure | CI/CD401k retirement plan | Comprehensive benefits | Dental insurance | Employee assistance program | Medical insuranceSenior-level Full TimeUSA, TX, Work-at-Home, United States R1d ago
-
Senior-level Full TimeMcLean, VA, United States1d ago
-
Lead Data Engineer USD 179K-225KAmazon Redshift | Amazon Web Services | Cassandra | Cloud platform | Data StreamingHealth benefits | Performance bonusSenior-level Full TimeMcLean, VA, United States1d ago
-
Associate Director, Data Science USD 160K-297KADME | Bioinformatics | Data Analysis | Data Governance | Data Mining401k match | Hybrid work | Paid time offMid-level Full TimeCambridge (USA), United States R1d ago
-
Sales Data & Analytics Product Manager USD 131K-190KAmazon Web Services | Analytics reporting | Apache Airflow | Apache Kafka | Apache Spark401k | Bonus | Dental coverage | Holidays | Medical coverageMid-level Full TimeUS, MA, Wilmington, United States1d ago
-
ML/ML Ops Principal Engineer USD 139K-190KAWS | BigQuery | Classification | Cloud Computing | Cloud platformSenior-level Full TimeGA250 - Atlanta GA, United States1d ago
-
Data Engineer USD 150K-190KAccess Control | Alerting | Audit Logging | Backfill | BigQueryDental insurance | Long-term disability | Medical insurance | Paid time off | Short-term disabilitySenior-level Full TimeColumbus, OH1d ago
-
Amazon SageMaker | Azure Machine Learning | CI/CD | Data Engineering | Data PipelinesOn-site workSenior-level Full TimeWashington, DC1d ago
-
Applied AI Engineer, Silicon Engineering USD 150K-275KC++ | CI/CD | Context engineering | Continuous integration | Coverage analysisDaily meals | Housing subsidy | Medical, dental & vision coverage | Relocation support | Unlimited compute budgetMid-level Full TimeSan Jose1d ago
-
Cloud Software Engineer USD 130K-250KAWS | Amazon Web Services | Apache Accumulo | Bigtable | C++Security ClearanceMid-level Full TimeAnnapolis Junction, MD1d ago