Sr Data Engineer
Des Moines, IA, United States
USD 130K-164K (estimate) Senior-level Full Time
Tasks
- Administer Databricks using Unity Catalog and cluster policies
- Build ETL/ELT pipelines
- Create runbooks and alerts
- Define lakehouse architecture and reference patterns
- Design data models for analytics and reporting
- Design scalable data pipelines
- Develop and optimize Databricks notebooks and workflows
- Enforce secure data access with RBAC
- Ensure compliance controls validation and documentation
- Ensure data lineage and metadata governance
- Implement Delta Lake patterns
- Implement automated data validation and profiling
- Implement encryption and secrets management
- Implement ingestion and transformation frameworks
- Lead Medallion architecture implementation
- Lead troubleshooting and root cause analysis
- Maintain architecture documentation and runbooks
- Monitor and optimize Spark jobs
- Perform code reviews and automated testing
- Perform performance tuning and cost optimization
- Support CI/CD and version control
- Translate business requirements into technical designs
Perks/Benefits
- N/A
Skills/Tech-stack
Automated testing | Azure Data | Azure Data Factory | Azure Databricks | CI/CD | Data Factory | Data Lineage | Data Modeling | Data Quality | Data Validation | Data profiling | Databricks | Delta Lake | ELT | ETL | Encryption | Lakehouse | Least Privilege | Medallion Architecture | Metadata Management | Performance Tuning | PySpark | RBAC | SQL | Secrets management | Snowflake Schema | Spark | Star Schema | Unity Catalog | Version control
Education
N/A
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R11d ago
-
ML Engineer, GenAI Ads, Search Personalization USD 207K-301KC++ | Clustering Algorithms | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeMountain View, CA, USA2h ago
-
Customer Engineer, Data/Analytics, Google Public Sector USD 152K-222KApache Spark | Batch Processing | Big Data | Cloud Architecture | Data GovernanceBonus target | Equity | Health benefits | Paid time off | Retirement benefitsSenior-level Full TimeReston, VA, USA; Atlanta, GA, USA2h ago
-
Forward Deployed AI Engineer USD 180K-230KAPI | AWS | Azure | CI/CD | Cloud platform5-day workweek | Collaborative work culture | Flexible working hours | Supportive work environmentSenior-level Full TimeNew York, New York, United States13h ago
-
Principal AI Engineer USD 200K-245KAgent-based | Agent-based architecture | Amazon Web Services | Azure | CI/CD5 days per week | Collaborative culture | Flexible working hours | Supportive work environmentSenior-level Full TimeNew York, New York, United States13h ago
-
Sr. Embedded Software/Firmware Engineer USD 180K-240KARM | ASPICE | AUTOSAR | C# | C++Competitive compensation | Dental insurance | Flexible hours | Medical insurance | Pre-IPO stock optionsSenior-level Full TimeUnited States - Remote R14h ago
-
Data Engineer USD 112K-160KAWS | AWS S3 | Airflow | Azure | CI/CD401k match | Family leave | Medical, dental, and vision insurance | PTO | Paid HolidaysMid-level Full TimeSanta Monica, CA, United States14h ago
-
Gen AI - CVS - Dallas USD 36K-126KAPI Gateway | Aiohttp | Amazon Web Services | Asyncio | CI/CD401k retirement plan | Medical, dental, vision benefits | Paid Holidays | Paid time off | Variable pay/incentivesMid-level Full TimeUnited States15h ago
-
Senior Numerical Optimization Engineer USD 191K-253KAPIs | C++ | CI/CD | CPLEX | Cloud ComputingComprehensive benefits | Health insurance | Paid on-call supportSenior-level Full TimeCosta Mesa, California, United States15h ago
-
Senior AI Engineer, Tools & Agents USD 200K-240KAPI Design | Agent Orchestration | Knowledge Base | LLM | Multi-tenantDental insurance | Flexible PTO | Health insurance | Parental leave | Visa supportSenior-level Full TimeSan Francisco R15h ago
-
Google Cloud Platform (GCP) Data Engineer USD 100K-203KApache Beam | BigQuery | Broadcast joins | CI/CD | CachingMid-level Full TimeArlington, VA15h ago
-
Google Cloud Platform (GCP) Data Engineer USD 100K-203KApache Beam | BigQuery | CI/CD | Cloud Storage | Cloud platformSecurity clearance providedMid-level Full TimeArlington, VA15h ago
-
Senior Data Engineer USD 180K-210KAirflow | Anomaly Detection | BigQuery | CI/CD | Cloud platform401k employer match | Annual Company Holidays | Catered lunches | Company retreat | Comprehensive health coverageSenior-level Full TimeNew York City15h ago
-
Sr. Engineer, Firmware & Embedded Software USD 79K-158KAgile | Azure DevOps | BDM | Bare Metal | BootloaderOccasional travel for onboarding | Remote work within the United StatesSenior-level Full TimeFlorida, United States R16h ago
-
API Design | Alerting | Data Modeling | Entitlement checks | Low LatencyHybrid work cultureSenior-level Full TimeHQ-San Francisco16h ago
-
API Design | Data Modeling | Low Latency | Low-Latency Systems | Rate LimitingHybrid work culture | In person meeting once per quarterSenior-level Full TimeHQ-San Francisco16h ago
-
API Design | Alerting | Data Modeling | Entitlement checks | Low LatencyHybrid work culture | In person once per quarter for remote rolesSenior-level Full TimeHQ-San Francisco16h ago
-
Software Engineer, Optimized Checkout & Link USD 161K-213KAPI Development | Automated testing | End to End | End-to-End Testing | GraphQL401k plan | Company bonus | Equity | Medical, dental, and vision benefits | TelecommutingMid-level Full TimeUS Remote R16h ago
-
Sr. Business Intelligence Engineer USD 110K-150KAzure Data | Azure Data Lake | Azure DevOps | CI/CD | Change Data CaptureDental benefits | Employee discounts | Employee recognition | Employee referral bonus | Medical benefitsSenior-level Full TimeHouston, Texas, United States16h ago
-
AI/ML Engineer USD 91K-184KAutomated Machine Learning | Continuous Deployment | Continuous integration | Data Pipelines | MLOpsMid-level Full TimeArlington, VA16h ago
-
Senior Software Engineer; Analytics Infrastructure USD 167K-230KAPI Design | Backend Engineering | Caching | Columnar Databases | Data ModelingBack up child care support | Employee stock purchase program ESPP | Flexible time off | Home office & equipment stipend | Learning and development stipendSenior-level Full TimeSan Francisco, CA17h ago
-
Data Solutions Engineer - HYBRID/NC RESIDENTS ONLY USD 119K-164KApache Spark | Azure | Azure Data | Azure Data Factory | Azure Data Lake401k match | Dental insurance | Employee referral program | Flexible spending account | Health insuranceEntry-level Full TimeDurham, North Carolina, United States R17h ago
-
Senior Data Engineer - Digital Pathology (Hybrid) USD 98K-163KAPI Integration | Agile | Apache Airflow | Apache Spark | Batch ProcessingSenior-level Full TimeRochester, MN, United States R17h ago
-
Principal Machine Learning Engineer USD 126K-264KCode review | Continuous Delivery | Continuous integration | Data Preprocessing | Data Privacy401k match | Adoption Assistance | Dental insurance | Health insurance | Life insuranceSenior-level Full TimeSeattle, WA, United States17h ago
-
Senior ECAD Library & Engineering Data Engineer USD 160K-225K3D Component Models | APIs | Altium 365 | Altium Designer | Altium ScriptingOnsite collaborationSenior-level Full TimeSouth San Francisco, California, USA17h ago