Sr Data Engineer
Des Moines, IA, United States
USD 130K-164K (estimate) Senior-level Full Time
Tasks
- Administer Databricks using Unity Catalog and cluster policies
- Build ETL/ELT pipelines
- Create runbooks and alerts
- Define lakehouse architecture and reference patterns
- Design data models for analytics and reporting
- Design scalable data pipelines
- Develop and optimize Databricks notebooks and workflows
- Enforce secure data access with RBAC
- Ensure compliance controls validation and documentation
- Ensure data lineage and metadata governance
- Implement Delta Lake patterns
- Implement automated data validation and profiling
- Implement encryption and secrets management
- Implement ingestion and transformation frameworks
- Lead Medallion architecture implementation
- Lead troubleshooting and root cause analysis
- Maintain architecture documentation and runbooks
- Monitor and optimize Spark jobs
- Perform code reviews and automated testing
- Perform performance tuning and cost optimization
- Support CI/CD and version control
- Translate business requirements into technical designs
Perks/Benefits
- N/A
Skills/Tech-stack
Automated testing | Azure Data | Azure Data Factory | Azure Databricks | CI/CD | Data Factory | Data Lineage | Data Modeling | Data Quality | Data Validation | Data profiling | Databricks | Delta Lake | ELT | ETL | Encryption | Lakehouse | Least Privilege | Medallion Architecture | Metadata Management | Performance Tuning | PySpark | RBAC | SQL | Secrets management | Snowflake Schema | Spark | Star Schema | Unity Catalog | Version control
Education
N/A
Roles
Related jobs
-
Delivery Senior Consultant, Software Engineering Solutions, Identity & Gen AI Engineer USD 155K-265KAI Agents | AWS | Access Management | Ansible | AuthenticationHybrid work model | Onsite up to 5 days per week | Professional training and development | Travel opportunitiesSenior-level Full TimeAtlanta, Georgia, United States; Charlotte, North …7h ago
-
AI Agents | AI Risk Management Framework | Access Management | Amazon Web Services | AnsibleSenior-level Full TimeArlington/Rosslyn, Virginia, United States; Baltimore, Maryland, …7h ago
-
AWS Data Engineer - Consultant USD 93K-170KAWS Glue | AWS Lambda | Amazon S3 | Cloud Engineering | Data ArchitectureMentorship | Professional developmentMid-level Full TimeHouston, Texas, United States7h ago
-
Data Engineer, Product Analytics USD 185K-204KAI workflows | C# | C++ | Data Compliance | Data ModelingCareer growthMid-level Full TimeLos Angeles, CA | Burlingame, CA8h ago
-
Business Support Engineer USD 159K-223KAPI Development | AWS | Agent Orchestration | Azure | Bias Mitigation24 7 Oncall RotationMid-level Full TimeMenlo Park, CA8h ago
-
Business Support Engineer USD 147K-203KAPI troubleshooting | AWS | Azure | Data Analysis | Debugging24 7 Oncall RotationMid-level Full TimeMenlo Park, CA8h ago
-
Business Support Engineer USD 141K-197KAI Agents | API troubleshooting | AWS | Agent Orchestration | Azure24 7 Oncall Rotation | Cross-functional collaborationSenior-level Full TimeMenlo Park, CA8h ago
-
AI Application Engineer USD 144K-209KAI Agents | API Development | Artificial Intelligence | Data Pipelines | Data QualityBonus | Employee benefits | Equity | Health insurance | Paid time offSenior-level Full TimeAustin, TX, USA8h ago
-
Staff Software Engineer, AI/ML, Search Ads USD 207K-301KC++ | Data Processing | Data Quality | Debugging | Distributed ComputingBonuses | Comprehensive health insurance | Equity | Paid time off | Retirement planSenior-level Full TimeMountain View, CA, USA8h ago
-
Senior Software Engineer, Database Internals, AlloyDB USD 174K-253KC# | C++ | Code optimization | Compute Technologies | Concurrency ControlEntry-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA8h ago
-
Staff Software Engineer, Network Health USD 207K-301KAnomaly Detection | Automated remediation | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA8h ago
-
Senior Data Engineer USD 90K-110KAgile | Amazon Web Services | Apache NiFi | Data Modeling | Data WarehousingAutonomy | Employee assistance program | Flexible working hours | Inclusive community | Online training videosSenior-level Full TimeNew York, United States R11h ago
-
Software Engineer, Data (L1) USD 75K-130KAI Platform | APIs | Apache Airflow | Apache Iceberg | Apache Spark401k match | Calm subscription | Career growth and learning opportunities | Commuter benefits options | Dental insuranceMid-level Full TimeAustin, TX15h ago
-
Machine Learning Research Engineer USD 161K-189KAWS | Azure | Bias Variance | Bias-Variance Tradeoff | C++Flexible hybrid work model | Mental health counseling | Mentorship programs | Paid parental leave | Paid volunteer time offMid-level Full TimeNew York, US, New York19h ago
-
Big Data | Cloud Cost Optimization | Cost Optimization | Data Governance | Data Security401k plan | Dental insurance | Health insurance | Life insurance | Paid time offMid-level Full TimeSpringdale, AR Terminal Office - Springdale, …19h ago
-
Senior Data Engineer - Logistics USD 38K-38KBig Data | Cloud Cost Optimization | Column-Level Security | Cost Optimization | Data Governance401k plan | Dental insurance | Health insurance | Life insurance | Paid time offSenior-level Full TimeSpringdale, AR Terminal Office - Springdale, …19h ago
-
Sr. Application Software Engineer, Data Analytics USD 160K-225KAngular | C# | CI/CD | Computer Vision | Continuous integrationExtended hours | Travel | Weekend workSenior-level Full TimeBastrop, TX20h ago
-
Mid-level Full TimeEl Segundo21h ago
-
Foundation Model Data, Software Engineer USD 213K-263KApache Beam | Apache Flume | C++ | CI/CD | Distributed Systems401k match | Baby bonding leave | Dental insurance | Disability insurance | Health insuranceSenior-level Full TimeMountain View, CA, USA; San Francisco, …21h ago
-
Senior-level Full TimeOnsite - Austin, TX21h ago
-
Machine Learning Engineer USD 196K-196KApache Airflow | CI/CD | Containerization | Docker | Experiment trackingFully remoteSenior-level Full TimeSan Francisco, CA R21h ago
-
Lead Cyber Risk & Analytics Engineer USD 130K-160KAgile | Data Visualization | Database querying | Econometrics | ExcelDependent care assistance | Flexible hours | Hybrid work | Learning and development | Mentorship programsSenior-level Full TimeNew York Office21h ago
-
Data Science Lead USD 133K-200KAI Foundry | API Development | Agile | Auditability | Automated testingKnowledge sharing | Mentoring | Technical oversightSenior-level Full TimeDes Moines, IA, United States21h ago
-
Applied AI/ML & Causal Inference - Senior Associate USD 177K-215KA/B | A/B Testing | Agentic AI | B testing | Causal InferenceSenior-level Full TimeJersey City, NJ, United States22h ago
-
Senior Data Engineer USD 90K-110KAWS | Agile | Apache NiFi | Data Architecture | Data ModelingAutonomy | Flexible working hours | Global employee assistance programme | Online training videos | Teambuilding eventsSenior-level Full TimeNew York, United States22h ago