ETL/Data Engineer
Indianapolis, IN, United States
USD 135K-165K (estimate) Senior-level Full Time
Tasks
- Build PySpark notebooks and jobs
- Define and enforce engineering coding standards
- Design and build data ingestion pipelines
- Design dimensional models and data vault patterns
- Develop and optimize ELT workflows
- Implement CI/CD for data pipelines
- Implement medallion architecture
- Implement slowly changing dimensions and change data capture
- Instrument pipelines with logging auditing and monitoring
- Lead legacy to cloud data migrations
- Monitor and troubleshoot data pipeline performance
- Perform workload assessment capacity planning and cost modeling
- Respond to production incidents for critical pipelines
- Tune distributed SQL workloads
Perks/Benefits
- N/A
Skills/Tech-stack
ACLs | Active Directory | Azure Active Directory | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure DevOps | Azure Key Vault | Azure Monitor | Azure Resource | Azure Resource Manager | Azure SQL | Azure Synapse | Azure Synapse Analytics | Bicep | CI/CD | Data Factory | Data Lake Storage Gen2 | Dedicated SQL Pool | Delta Lake | Entra ID | Hierarchical Namespace | KQL | Key Vault | Lifecycle Management | Log Analytics | Managed Identities | Microsoft Fabric | Network Integration | Pandas | Parquet | Private Endpoints | PySpark | Pytest | Python | RBAC | REST API | Resource manager | SQL | SQL Pool | Service principals | Spark SQL | Synapse Analytics | Synapse Dedicated SQL Pool | T-SQL | Teradata | Terraform | Unit Testing | Unity Catalog | Virtual Network | Virtual Network Integration | YAML
Education
N/A
Related jobs
-
Senior Data Engineer USD 115K-150KAWS | AWS Glue | Airflow | Amazon Redshift | Amazon S3401k contribution | Health insurance | Life insuranceSenior-level Full TimeWashington, DC9h ago
-
Business Analysis | Customer Requirements | Data Modeling | Language Models | Language ProcessingHealth insurance | Mentorship | Paid time offMid-level Full TimeNew York, US, New York11h ago
-
Cost of ownership | Customer Success | Data Modeling | Data analytics | LLMCoaching | Flexible-hybrid work | Mental health counseling | Mentorship | Paid volunteer time offMid-level Full TimeRaleigh, US, North Carolina12h ago
-
Quantitative Trader - 2027 USD 150K-250KC plus plus | Data Analysis | Data Modeling | Linux | Machine LearningContinuous learning workshops | Fitness Events | Free meals | Hybrid work | Paid time offMid-level Full TimeNew York, Chicago17h ago
-
Senior Embedded Software Engineer USD 120K-160KAtlassian Confluence | Bare Metal | Bitbucket | C++ | Cameo401k employer match | Career growth | Employee assistance program | Low-cost medical dental vision | Paid HolidaysSenior-level Full TimePalmdale, CA; Fort Worth, TX17h ago
-
API Development | AWS | Amazon SageMaker | Automated testing | CI/CDFlexible work arrangements | Health, dental, vision coverage | Hybrid work arrangement | Innovation opportunities | Internal mobilitySenior-level Full TimeCanada17h ago
-
BI tools | CI/CD | Data Modeling | Data Quality | Data WarehousingCareer growth | Flexible work arrangement | Greenfield project ownership | Occasional travel | Opportunity for diverse client engagementsMid-level Full TimeCanada17h ago
-
Alerting | BigQuery | CI/CD | Cloud Run | Cloud SQLCollaborative engineering team | Continuous learning | Flexible working arrangements | Fully remote | High autonomySenior-level Full TimeCanada R20h ago
-
APIs | AWS | Add-ons | Ansible | AppDynamicsSenior-level Full TimeHerndon, Virginia, United States1d ago
-
Data Engineer USD 101K-132KAWS Glue | Amazon DynamoDB | Amazon Kinesis | Apache Spark | Azure CosmosMid-level Full TimeCarmel, Indiana- Hybrid OR St. Louis, … R1d ago
-
AI/Machine Learning Engineer USD 130K-177KAWS | Apache Spark | Azure AI | Azure Machine Learning | DatabricksMid-level Full TimeCarmel, Indiana- Hybrid OR St. Louis, … R1d ago
-
Senior Machine Learning Engineer - Agentic Workflow USD 145K-196KAWS | Async API | CI/CD | Distributed Systems | DockerSenior-level Full TimeHouston, TX, United States1d ago
-
Robotics Software Engineer USD 155K-300KC++ | Device Drivers | EBPF | GPU Profiling | Kernel programmingSenior-level Full TimeSan Francisco1d ago
-
Embedded Engineer USD 175K-280KAOSP | Bare Metal | Bluetooth | Bootloader | C#401k match | Employee assistance program | Flexible spending account | Health, dental, vision coverage | Stock optionsSenior-level Full TimeSan Francisco1d ago
-
Software Engineer, Cooperative AI USD 230K-385KCI/CD | Cloud Platforms | Data Modeling | Docker | Information securitySenior-level Full TimeSan Francisco1d ago
-
Software Engineer, Analytics USD 99K-145KAWS | Amazon QuickSight | Automation | DBT | Data ModelingMid-level Full TimeRemote - US Only R1d ago
-
Space Operations Engineer (Embedded Software) USD 119K-160KARM | C# | C++ | Command and control | Communication ProtocolsMid-level Full TimeSan Francisco, CA1d ago
-
Senior Software Engineer (NIN-AE4-17.022425) USD 130K-225KAnsible | Apache NiFi | Java | Pig | PySparkSenior-level Full TimeLinthicum Heights, Maryland1d ago
-
Senior/Staff Machine Learning Engineer, Planning USD 160K-220KC++ | Cloud Computing | Data Ingestion | Deep learning | Experimentation401k plan | Free lunch | Professional development | Unlimited snacks and beveragesSenior-level Full TimeSanta Clara, CA1d ago
-
Mechatronic/Robotics Engineer or Systems Integration Engineer - Camera and Sensor Calibration USD 120K-190KAutomation systems | Bash | C++ | Camera Calibration | Control SystemsOn-site workMid-level Full TimeMountain View, CA1d ago
-
Senior-level Full TimeMountain View, CA1d ago
-
Staff AI engineer USD 170K-250KAI Model Evaluation | AI model | AWS | Agent Orchestration | CachingFlexible working hours | Hybrid work culture | Unlimited time offSenior-level Full TimeSan Francisco1d ago
-
Senior Quantum Embedded Engineer USD 142K-175K10G Ethernet | AMD Xilinx | Bash | C/C++ | CI toolsHybrid work | Remote workSenior-level Full TimeNew Haven, CT1d ago
-
Senior Quantum Applications Engineer - QEC USD 119K-258KCUDA-Q | Docker | End to End | End-to-End Testing | Error correctionSenior-level Full TimeNew Haven, CT1d ago
-
Quantum Engineer (Physicist) USD 136K-187KCircuit-QED | Cryogenics | Data Analysis | Error correction | Low temperature physicsMid-level Full TimeNew Haven, CT1d ago