Sr Data Engineer
Des Moines, IA, United States
USD 130K-164K (estimate) Senior-level Full Time
Tasks
- Administer Databricks using Unity Catalog and cluster policies
- Build ETL/ELT pipelines
- Create runbooks and alerts
- Define lakehouse architecture and reference patterns
- Design data models for analytics and reporting
- Design scalable data pipelines
- Develop and optimize Databricks notebooks and workflows
- Enforce secure data access with RBAC
- Ensure compliance controls validation and documentation
- Ensure data lineage and metadata governance
- Implement Delta Lake patterns
- Implement automated data validation and profiling
- Implement encryption and secrets management
- Implement ingestion and transformation frameworks
- Lead Medallion architecture implementation
- Lead troubleshooting and root cause analysis
- Maintain architecture documentation and runbooks
- Monitor and optimize Spark jobs
- Perform code reviews and automated testing
- Perform performance tuning and cost optimization
- Support CI/CD and version control
- Translate business requirements into technical designs
Perks/Benefits
- N/A
Skills/Tech-stack
Automated testing | Azure Data | Azure Data Factory | Azure Databricks | CI/CD | Data Factory | Data Lineage | Data Modeling | Data Quality | Data Validation | Data profiling | Databricks | Delta Lake | ELT | ETL | Encryption | Lakehouse | Least Privilege | Medallion Architecture | Metadata Management | Performance Tuning | PySpark | RBAC | SQL | Secrets management | Snowflake Schema | Spark | Star Schema | Unity Catalog | Version control
Education
N/A
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R12d ago
-
Mid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL2h ago
-
Senior-level ContractRaleigh, United States3h ago
-
AI experimentation | Alerting | Artificial Intelligence | Data Annotation | Data ArchitectureSenior-level Full TimeNew York4h ago
-
Project Lead Data Engineer III USD 119K-198KBTEQ | Database Administration | Patch Management | Performance Tuning | Query OptimizationSenior-level Full TimeSacramento, California, United States4h ago
-
ML Engineer, GenAI Ads, Search Personalization USD 207K-301KC++ | Clustering Algorithms | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeMountain View, CA, USA7h ago
-
Customer Engineer, Data/Analytics, Google Public Sector USD 152K-222KApache Spark | Batch Processing | Big Data | Cloud Architecture | Data GovernanceBonus target | Equity | Health benefits | Paid time off | Retirement benefitsSenior-level Full TimeReston, VA, USA; Atlanta, GA, USA7h ago
-
Senior-level Full TimeUnited States13h ago
-
Lead Software Engineer - Applied AI ML Lead USD 170K-195KApache Airflow | Apache Iceberg | Apache Kafka | Apache Spark | Automated testingBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimePalo Alto, CA, United States16h ago
-
Forward Deployed AI Engineer USD 180K-230KAPI | AWS | Azure | CI/CD | Cloud platform5-day workweek | Collaborative work culture | Flexible working hours | Supportive work environmentSenior-level Full TimeNew York, New York, United States17h ago
-
Principal AI Engineer USD 200K-245KAgent-based | Agent-based architecture | Amazon Web Services | Azure | CI/CD5 days per week | Collaborative culture | Flexible working hours | Supportive work environmentSenior-level Full TimeNew York, New York, United States17h ago
-
Data Engineer, Consultant USD 125K-164KApache Airflow | Azure | Azure Data | Azure Data Factory | Azure Data LakeSenior-level Full TimeOakland, CA, United States18h ago
-
AI Engineer (Temporary) USD 85K-95KAWS | Agents SDK | Anthropic API | Azure | Data RiskAccess to training and enablement resources | Hybrid work environment | Temporary contract with benefitsEntry-level Full Time TemporaryIthaca (Main Campus), United States18h ago
-
Principal Embedded Software Engineer - OS Build USD 152K-229KArtifactory | BSP | Bamboo | Bash | BitbucketSenior-level Full TimeUSA-CO Lafayette Bldg 2, United States18h ago
-
Mid-level Full TimeUSA - Seattle, WA, United States18h ago
-
Principal Embedded Software Engineer - OS Build USD 152K-229KArtifactory | Atlassian | BSP | Bamboo | BashSenior-level Full TimeUSA-CO Lafayette Bldg 2, United States18h ago
-
Lead Software Engineer (Data)-7999 USD 129K-203KAmazon CloudWatch | Amazon EC2 | Amazon EMR | Amazon IAM | Amazon S3Comprehensive health benefits | Emotional wellbeing support | Financial support benefits | Personalized benefits | Remote work days per weekSenior-level Full TimeVA - Reston, 11951 Freedom Dr … R18h ago
-
Sr. Embedded Software/Firmware Engineer USD 180K-240KARM | ASPICE | AUTOSAR | C# | C++Competitive compensation | Dental insurance | Flexible hours | Medical insurance | Pre-IPO stock optionsSenior-level Full TimeUnited States - Remote R18h ago
-
Data Engineer USD 112K-160KAWS | AWS S3 | Airflow | Azure | CI/CD401k match | Family leave | Medical, dental, and vision insurance | PTO | Paid HolidaysMid-level Full TimeSanta Monica, CA, United States18h ago
-
Gen AI - CVS - Dallas USD 36K-126KAPI Gateway | Aiohttp | Amazon Web Services | Asyncio | CI/CD401k retirement plan | Medical, dental, vision benefits | Paid Holidays | Paid time off | Variable pay/incentivesMid-level Full TimeUnited States19h ago
-
Senior Numerical Optimization Engineer USD 191K-253KAPIs | C++ | CI/CD | CPLEX | Cloud ComputingComprehensive benefits | Health insurance | Paid on-call supportSenior-level Full TimeCosta Mesa, California, United States19h ago
-
Senior AI Engineer, Tools & Agents USD 200K-240KAPI Design | Agent Orchestration | Knowledge Base | LLM | Multi-tenantDental insurance | Flexible PTO | Health insurance | Parental leave | Visa supportSenior-level Full TimeSan Francisco R19h ago
-
Google Cloud Platform (GCP) Data Engineer USD 100K-203KApache Beam | BigQuery | Broadcast joins | CI/CD | CachingMid-level Full TimeArlington, VA19h ago
-
Google Cloud Platform (GCP) Data Engineer USD 100K-203KApache Beam | BigQuery | CI/CD | Cloud Storage | Cloud platformSecurity clearance providedMid-level Full TimeArlington, VA20h ago
-
Senior Data Engineer USD 180K-210KAirflow | Anomaly Detection | BigQuery | CI/CD | Cloud platform401k employer match | Annual Company Holidays | Catered lunches | Company retreat | Comprehensive health coverageSenior-level Full TimeNew York City20h ago