Senior Data Engineer
Philadelphia, Pennsylvania, United States
USD 130K-165K (estimate) Senior-level Full Time
Tasks
- Build batch and streaming ETL and ELT workflows
- Deliver pipelines using software engineering standards documentation testing and CI/CD
- Design data pipelines for real time streaming
- Develop data services APIs and microservices
- Ensure data quality validation and privacy
- Implement streaming ingestion with Azure Event Hubs
- Optimize Databricks pipelines with PySpark and Spark SQL
- Process streams with Azure Stream Analytics
- Troubleshoot and resolve pipeline issues across systems
- Tune Spark for performance reliability and cost
Perks/Benefits
Skills/Tech-stack
Apache Spark | Azure | Azure Event | Azure Event Hubs | Azure Stream | Azure Stream Analytics | CI/CD | Databricks | Delta Lake | ELT | ETL | Event Hubs | Microservices | PySpark | REST APIs | Spark SQL | Stream Analytics
Education
Roles
Regions
Countries
States
Related jobs
-
Data Engineer II USD 145K-165KAWS | Agile | Apache Airflow | Apache Spark | CI/CD401k match | Company-provided phone | Extended leave | Generous PTO | Health insuranceMid-level Full TimeNew York, New York, United States12h ago
-
API | AWS Glue | Amazon Redshift | Amazon Web Services | Apache AirflowFully remote | Mentorship opportunities | On-call supportSenior-level Full TimeOrlando, FL, United States R14h ago
-
Senior Data Engineer USD 140K-160KAirflow | Apache Spark | Auto Loader | CI/CD | Change Data Capture401k | Paid leave | Tuition reimbursementSenior-level Full TimeNorth Hollywood, CA, United States14h ago
-
Senior Data Engineer – Hardware & Supply Chain USD 147K-203KAWS | Apache Iceberg | DBT | Data Governance | Data ModelingFlexible wellness time off | Paid Holidays | Paid sick leave | Paid time off | Parental leaveSenior-level Full TimeHybrid - San Francisco, California R14h ago
-
Mid-level Full TimeMiami Lakes, FL, United States16h ago
-
Senior AI Engineer USD 193K-240KAPI | Access Control | Audit Logging | CI/CD | Cloud Native401k employer match | Dental insurance | Dependent care FSA | Employer paid commuter benefit | Flexible time offSenior-level Full TimeDallas, Texas, United States17h ago
-
AWS | Autogen | Automated testing | Azure | CI/CDOnsite workSenior-level ContractDallas, United States21h ago
-
Databricks Engineer USD 97K-162KAPI Integration | Agile | Data Governance | Data Modeling | Data QualityMid-level Full TimeArlington/Rosslyn, Virginia, United States; Baltimore, Maryland, …21h ago
-
Data Scientist, Innovation Hub USD 108K-108KAudit Logging | Azure | CI/CD | Federated identity | Integration TestingCareer growth | Healthcare | On-site childcare | PTO | Paid HolidaysMid-level Full TimeTexas-Dallas-5323 Harry Hines Blvd21h ago
-
Databricks Engineer USD 107K-179KAPI Integration | Agile | Cloud Computing | Data Governance | Data ModelingTravel 10 to 15 percentMid-level Full TimeArlington/Rosslyn, Virginia, United States; Baltimore, Maryland, …21h ago
-
Senior Data Engineer USD 133K-158KAKS | Azure Data | Azure Data Factory | Azure Data Lake | Azure DatabricksSenior-level Full TimeCincinnati, Ohio, United States21h ago
-
AI Program Lead USD 159K-265KAI Governance | Agent-Based Artificial Intelligence | Agent-based | Artificial Intelligence | Business IntelligenceMentorship | Professional development | Travel up to 25 percentSenior-level Full TimeArlington/Rosslyn, Virginia, United States21h ago
-
Associate BI Analytics Engineer USD 98K-163KDAX | Data Modeling | Data Warehousing | Databricks | ELTMid-level Full TimeUSA-VA-Herndon21h ago
-
Senior Software Engineer - Data & Analytics Federation USD 160K-240KApache Arrow | Asynchronous frameworks | C++ | Distributed Systems | Linux401k match | Dental insurance | Life insurance | Medical insurance | Paid HolidaysSenior-level Full TimePrinceton21h ago
-
Data Engineer, Employee Technology Support USD 110K-130KCause analysis | Data Governance | Data Modeling | Data Monitoring | Data Quality401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceMid-level Full TimeNew York21h ago
-
Security Engineer III - AI/ML USD 170K-215KAPI Development | AWS | Anomaly Detection | Azure | BERTSenior-level Full TimePlano, TX, United States22h ago
-
Principal Data Engineer USD 160K-170KAWS | Amazon Redshift | Apache Airflow | BigQuery | Cloud platform401k match | Dental insurance | Health insurance | Paid time off | Remote workSenior-level Full TimeRemote (United States) R1d ago
-
AI/ML Engineer - School USD 101K-163KAI Safety | AI safety evaluation | AWS Bedrock | AWS ECS | AWS LambdaMid-level Full TimeVirtual US IL, United States R1d ago
-
AI Software Engineer - Fixed Income Technology USD 110K-150KAPI Integration | AWS | Azure | CI/CD | Data PipelinesHealth insurance | Hybrid work | Life insurance | Paid time off | Retirement planSenior-level Full TimeIL01 - Chicago, United States1d ago
-
Senior Cloud Database Engineer (Hybrid - Seattle, WA) USD 142K-220KASM | AWR | AWS DMS | AWS RDS | Amazon AuroraDisability insurance | EAP Resources | Hybrid work location | Life insurance | Medical/Vision/DentalSenior-level Full TimeSeattle WA, United States R1d ago
-
Senior Machine Learning Engineer USD 159K-295KAUC-ROC | AWS | Causal Inference | Clean room | Data clean roomSenior-level Full TimeCA San Francisco 153 Kearny Street, …1d ago
-
Principal AI Platform Engineer USD 117K-206KAI | AWS | Agent workflows | Ansible | Argo CDHealth insurance | Hybrid work | Paid time off | Remote work | Retirement savings planSenior-level Full TimeUSA - MN - Minneapolis, United …1d ago
-
Apache Iceberg | Atlan | BigQuery | Cloud Composer | Cloud RunSenior-level Full TimePA, Working at Home - Pennsylvania, …1d ago
-
AWS | Ab Initio | Artificial Intelligence | Data Governance | Data Quality401k matching | Health insurance | Mental health counseling | Paid Holidays | Paid time offMid-level Full TimeHartford - Tower, United States1d ago
-
Code review | Data Governance | Data Ingestion | Data Mastering | Data SharingSenior-level Full TimePrimary location: Dublin, Franklin, OH R1d ago