Principal Data Engineer
Philadelphia, Pennsylvania, United States
USD 120K-195K (estimate) Senior-level Full Time
Tasks
- Build and maintain data products APIs and microservices
- Define streaming reliability latency quality and observability
- Deliver real time IoT telemetry streaming ingestion
- Design Databricks lakehouse data platform architecture
- Design batch and streaming data pipelines
- Design data contracts and lineage for ML AI
- Design data quality checks validation rules and anomaly detection
- Develop ETL/ELT workflows
- Establish pipeline standards testing deployment monitoring documentation
- Implement data governance, privacy, and security practices
- Implement streaming solutions with Event Hubs and Stream Analytics
- Lead design reviews code reviews and production readiness reviews
- Lead stakeholder shaping for data requirements
- Manage mentor and develop data engineers
- Monitor data freshness schema and operational alerts
- Optimize Databricks pipelines with Spark SQL Delta Lake
- Optimize throughput partitioning retention and replay strategies
- Partner with ML AI analytics teams for model ready datasets
- Translate ambiguous business needs into data product definitions
- Troubleshoot production data pipeline issues
- Use Python PySpark and SQL to implement data systems
Perks/Benefits
- N/A
Skills/Tech-stack
API | Azure | Azure Event | Azure Event Hubs | Azure Stream | Azure Stream Analytics | CI/CD | Cause analysis | Data Modeling | Data Quality | Databricks | Delta Lake | ELT | ETL | Event Hubs | IoT | Medallion Architecture | Microservices | Observability | Partitioning | PySpark | Python | Root Cause Analysis | Root cause | SQL | Spark SQL | Stream Analytics | Streaming Data
Education
N/A
Roles
Data Engineer | Engineer | Principal | Principal Data Engineer
Regions
Countries
States
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R3d ago
-
Senior-level Full TimeUS - Remote; Canada - Remote R5h ago
-
Senior-level Full TimeMountain View, CA7h ago
-
Staff Software Engineer, AI Integrations USD 170K-240KAgile | Alerting | Amazon Web Services | C++ | CSSSenior-level Full TimeMountain View, CA7h ago
-
Senior Software Engineer Remote/Travel- Active Secret -Top Secret clearance(SCI-eligibility) USD 145K-204KAPI Testing | Cypher | Data Processing | DataOps | DevOpsCompetitive benefits | Growth opportunity | Remote work | TravelSenior-level Full TimeReston, VA, United States R17h ago
-
AWS | AWS CloudFormation | Amazon ECS | Automation | CI/CD401-k plan | Company match | Employee stock purchase plan | Healthcare benefits | Life insuranceMid-level Full TimeAtlanta, GA, United States19h ago
-
Software Engineer - X Data USD 125K-400KApache Flink | Apache Kafka | Apache Spark | BigQuery | Cause analysis401k retirement plan | Dental insurance | Discounts | Health insurance | Life insuranceMid-level Full TimePalo Alto, CA1d ago
-
Embedded Software Engineer Intern (Fall 2026) USD 108K-108KADC | Antenna Testing | C# | C++ | CANHousing stipend | Overtime pay | Paid sick time | Relocation supportEntry-level InternshipSouth San Francisco, California, USA1d ago
-
Site Reliability Engineer, Inference Infrastructure USD 165K-267KAmazon Web Services | C plus plus | Cloud platform | Distributed Systems | GPU401k matching | Co working Benefit | Company offsite | Education and learning stipend | Health and dental benefitsMid-level Full TimeToronto1d ago
-
Staff Software Engineer, Inference Infrastructure USD 160K-220KAmazon Web Services | Azure | C++ | Cloud platform | Distributed Systems401k | Annual company offsite | Arts and culture budget | Education and learning stipend | Health and dental benefitsSenior-level Full TimeSan Francisco1d ago
-
AI Engineer – Decision & Optimization Systems USD 80K-210KAMPL | C++ | CPLEX | Constraint Satisfaction | Go401k | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
AI Engineer - Allocation and Packing Systems USD 80K-210KA/B | A/B Testing | Approximation algorithms | Assignment Algorithms | B testing401k | Equity grant | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
AI Engineer – Routing & Network Optimization USD 80K-210KA Star | A Star Algorithm | AMPL | Bellman Ford Algorithm | CPLEX401k | Equity grant | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
AI Software Engineer USD 80K-210KAWS Kinesis | Apache Airflow | Apache Kafka | C# | C++401k | Equity grant | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
Deployment Strategist - North America USD 150K-245KAI Agent | AI Agent Frameworks | API Integration | Agent Frameworks | Agent OrchestrationAnnual company offsite | Annual travel stipend | Co-working stipend | Professional development stipendMid-level Full TimeUnited States R1d ago
-
Applied AI Engineer USD 204K-352KBackend Development | Cloud infrastructure | Deep learning | Docker | Fine TuningAnnual membership | Centralized coffee stipend | Flexible PTO | Health and wellness stipend | Health, dental, and vision insuranceSenior-level Full TimeNew York, NY (HQ)1d ago
-
ML Research Engineer - Hardware Codesign USD 185K-455KC++ | CUDA | Floating point | Floating point numerics | Functional simulationHybrid work schedule | Relocation assistanceSenior-level Full TimeSan Francisco1d ago
-
Staff AI Engineer USD 140K-210KAPI Development | AWS | Access Management | Bedrock AgentCore | CI/CD401k | All-hands meetings | Dental insurance | Employee equity program | Health insuranceSenior-level Full TimeNew York City1d ago
-
Senior Quantum Embedded Engineer USD 142K-175K10G Ethernet | AMD Xilinx | AMD Xilinx toolchain | AMD/Xilinx SoC | BashHybrid work options | Remote work optionsSenior-level Full TimeNew Haven, CT1d ago
-
Associate Quantum Engineer USD 130K-184KCryogenic Systems | Data Analysis | Data acquisition | Instrument Control | Laboratory Instrument ControlInterdisciplinary team | Mentorship | State-of-the-art facilitiesMid-level Full TimeNew Haven, CT1d ago
-
Quantum Engineer (Physicist) USD 136K-187KCircuit-QED | Cryogenics | Data Analysis | Error correction | Low temperature physicsMid-level Full TimeNew Haven, CT1d ago
-
Senior Quantum Applications Engineer - QEC USD 120K-258KCUDA-Q | Decoder algorithms | Docker | End to End | End-to-End TestingSenior-level Full TimeNew Haven, CT1d ago
-
Staff AI engineer USD 170K-250KAI Evaluation | AWS | Agent Orchestration | Caching | Distributed SystemsFlexible working hours | Hybrid work culture | Unlimited time offSenior-level Full TimeSan Francisco1d ago
-
Principal Business Consultant - Global Analytics USD 120K-250KArtificial Intelligence | Business Development | Call planning | Change Management | ConsultingBasic life insurance | Career progression | Charitable giving program | Company-Paid Holidays | Dental insuranceSenior-level Full TimeMassachusetts - Boston R1d ago
-
C++ | Deep learning | Imitation Learning | Machine Learning | PlanningHealth insurance | Life insurance | Long-term care insurance | Long-term disability insurance | Paid time offSenior-level Full TimeFoster City, CA1d ago