ETL/Data Engineer
Indianapolis, IN, United States
USD 135K-165K (estimate) Senior-level Full Time
Tasks
- Build PySpark notebooks and jobs
- Define and enforce engineering coding standards
- Design and build data ingestion pipelines
- Design dimensional models and data vault patterns
- Develop and optimize ELT workflows
- Implement CI/CD for data pipelines
- Implement medallion architecture
- Implement slowly changing dimensions and change data capture
- Instrument pipelines with logging auditing and monitoring
- Lead legacy to cloud data migrations
- Monitor and troubleshoot data pipeline performance
- Perform workload assessment capacity planning and cost modeling
- Respond to production incidents for critical pipelines
- Tune distributed SQL workloads
Perks/Benefits
- N/A
Skills/Tech-stack
ACLs | Active Directory | Azure Active Directory | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure DevOps | Azure Key Vault | Azure Monitor | Azure Resource | Azure Resource Manager | Azure SQL | Azure Synapse | Azure Synapse Analytics | Bicep | CI/CD | Data Factory | Data Lake Storage Gen2 | Dedicated SQL Pool | Delta Lake | Entra ID | Hierarchical Namespace | KQL | Key Vault | Lifecycle Management | Log Analytics | Managed Identities | Microsoft Fabric | Network Integration | Pandas | Parquet | Private Endpoints | PySpark | Pytest | Python | RBAC | REST API | Resource manager | SQL | SQL Pool | Service principals | Spark SQL | Synapse Analytics | Synapse Dedicated SQL Pool | T-SQL | Teradata | Terraform | Unit Testing | Unity Catalog | Virtual Network | Virtual Network Integration | YAML
Education
N/A
Related jobs
-
Machine Learning Engineer, Ecommerce Risk Control USD 175K-266KData Analysis | Data Visualization | Experiment design | Machine Learning | Model DeploymentSenior-level Full TimeSan Jose, California, United States4h ago
-
Computer Vision | Fraud Detection | Language Processing | Machine Learning | Natural LanguageSenior-level Full TimeSeattle, Washington, United States4h ago
-
A/B | A/B Testing | B testing | Data Mining | Distributed ComputingMid-level Full TimeSeattle, Washington, United States4h ago
-
Machine Learning Engineer, Pricing - TikTok E-commerce Global Supply Chain and Logistics USD 129K-194KCausal Inference | Data Mining | Forecasting | Graph Databases | Knowledge graphsMid-level Full TimeSeattle, Washington, United States4h ago
-
Machine Learning Engineer, Ecommerce Risk Control USD 175K-266KA/B | A/B Testing | B testing | Data Visualization | Deep learningSenior-level Full TimeSeattle, Washington, United States4h ago
-
Mid-level Full TimeSan Jose, California, United States4h ago
-
Big Data | Click Through Rate | Click Through Rate Optimization | Conversion Rate | Conversion rate optimizationSenior-level Full TimeSeattle, Washington, United States4h ago
-
Machine Learning Engineer, Ecommerce Risk Control USD 175K-266KData Analysis | Data Pipelines | Data Visualization | Deep learning | Experiment designSenior-level Full TimeSeattle, Washington, United States4h ago
-
Site Reliability Engineer - Data Infrastructure USD 100K-215KAlerting | Bash | Change Management | Go | Incident ResponseRotational on-callEntry-level Full TimeSeattle, Washington, United States4h ago
-
Applied Scientist, Search, E-Commerce Alliance USD 136K-205KData Analysis | Language Processing | Machine Learning | Model Deployment | Natural LanguageMid-level Full TimeSan Jose, California, United States4h ago
-
Site Reliability Engineer - Data Infrastructure USD 100K-215KAlert Tuning | Automation | Bash | Change Management | DatabasesRotational on-callEntry-level Full TimeSan Jose, California, United States4h ago
-
Behavior Modeling | Data Analysis | Information Retrieval | Language Processing | Machine LearningMid-level Full TimeSeattle, Washington, United States4h ago
-
Senior Data Engineer USD 115K-150KAWS | AWS Glue | Airflow | Amazon Redshift | Amazon S3401k contribution | Health insurance | Life insuranceSenior-level Full TimeWashington, DC10h ago
-
Business Analysis | Customer Requirements | Data Modeling | Language Models | Language ProcessingHealth insurance | Mentorship | Paid time offMid-level Full TimeNew York, US, New York13h ago
-
Cost of ownership | Customer Success | Data Modeling | Data analytics | LLMCoaching | Flexible-hybrid work | Mental health counseling | Mentorship | Paid volunteer time offMid-level Full TimeRaleigh, US, North Carolina13h ago
-
AI Engineer USD 92K-138KAPI Design | Agent Frameworks | Anomaly Detection | Artificial Intelligence | Computer VisionSenior-level Full TimeOK PLAZA PL, United States16h ago
-
Senior Staff AI Data Engineer - Hybrid USD 135K-202KAWS | Alerting | Azure | Batch pipelines | ChunkingHybrid work schedule | Learning and development opportunities | MentorshipSenior-level Full TimeHartford CT- Home Office, United States R16h ago
-
Mid-level Full TimeUnited States - Remote R16h ago
-
Senior Machine Learning Engineer USD 112K-183KAWS | Data Ingestion | Data Modeling | Data Quality | Data Visualization401k savings plan | Adoption benefits | Career development | Disability benefits | Employee assistance programSenior-level Full TimeMossville, Illinois, United States16h ago
-
Quantitative Trader - 2027 USD 150K-250KC plus plus | Data Analysis | Data Modeling | Linux | Machine LearningContinuous learning workshops | Fitness Events | Free meals | Hybrid work | Paid time offMid-level Full TimeNew York, Chicago18h ago
-
Senior Embedded Software Engineer USD 120K-160KAtlassian Confluence | Bare Metal | Bitbucket | C++ | Cameo401k employer match | Career growth | Employee assistance program | Low-cost medical dental vision | Paid HolidaysSenior-level Full TimePalmdale, CA; Fort Worth, TX18h ago
-
API Development | AWS | Amazon SageMaker | Automated testing | CI/CDFlexible work arrangements | Health, dental, vision coverage | Hybrid work arrangement | Innovation opportunities | Internal mobilitySenior-level Full TimeCanada19h ago
-
BI tools | CI/CD | Data Modeling | Data Quality | Data WarehousingCareer growth | Flexible work arrangement | Greenfield project ownership | Occasional travel | Opportunity for diverse client engagementsMid-level Full TimeCanada19h ago
-
Alerting | BigQuery | CI/CD | Cloud Run | Cloud SQLCollaborative engineering team | Continuous learning | Flexible working arrangements | Fully remote | High autonomySenior-level Full TimeCanada R21h ago
-
AI Agents | Data Processing | Debugging | Distributed Systems | Language Processing401k matching | Career growth | Dental insurance | Health insurance | Life insuranceMid-level Full TimeSan Francisco, California, United States1d ago