Principal Data Engineer (streaming)
Tasks
- Automate data quality checks
- Define SLAs and SLOs for latency and throughput
- Define schema contracts and schema evolution
- Design scalable streaming data pipelines
- Engineer real time data solutions
- Establish observability with metrics logging and distributed tracing
- Implement CI CD integration for data pipelines
- Implement event-driven architectures
- Implement production grade streaming pipelines
- Implement transformation tests and data assertions
- Improve data governance for streaming workloads
- Integrate alerting dashboards and incident response
- Mentor engineers and conduct peer reviews
- Modernize data ecosystem
- Optimize streaming performance tuning
Perks/Benefits
- Allyship and inclusion communities
- Caregiver leave
- Continuous development support program
- Employee assistance program
- Employee recognition
- Fertility and adoption support
- Flexible paid time off
- Parental leave
- Retirement plan matching
- Wellness support programs
Skills/Tech-stack
AWS | Alerting | Apache Flink | Apache Hudi | Apache Kafka | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | CI/CD | Cloud platform | DBT | Dashboards | Data Governance | Data Lake Storage Gen2 | Data Quality | Databricks | Distributed Systems | Distributed tracing | Docker | Event Driven | Event-driven architecture | Google Cloud | Google Cloud Platform | Great Expectations | Kubernetes | Lakehouse | Logging | Low Latency | Metrics | Microsoft Azure | Observability | Performance Tuning | Schema evolution | Spark Streaming
Education
N/A
Roles
Data Engineer | Engineer | Principal | Principal Data Engineer
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R5d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R18d ago
-
Senior Storage Engineer USD 145K-170KCapacity Planning | Cause analysis | Cloud Platforms | Data center | Distributed file systems401k match | Dental insurance | Employee assistance program | Health insurance | Health savings accountSenior-level Full TimeRemote, Texas R11h ago
-
Senior Software Engineer Remote/Travel- Active Secret -Top Secret clearance(SCI-eligibility) USD 145K-200KAPI Testing | Cloud Computing | Cypher | Data Processing | Data QualityProfessional growth | Remote work | Travel opportunitiesSenior-level Full TimeReston, VA, United States R12h ago
-
Staff Analytics Engineer USD 125K-254KBusiness Logic | DBT | Data Architecture | Data Modeling | Data WarehouseSenior-level Full TimeRemote, USA R13h ago
-
Apache Iceberg | CI/CD | Compaction | Cost Management | DBTEmployee Stock Purchase Plan (ESPP) | Flexible spending wallets | Health care coverage | Paid time off | Remote-first work environmentSenior-level Full TimeRemote US R1d ago
-
Agentic Workflows | CI/CD | Docker | Embeddings | Evaluation instrumentationEducational assistance | Health and wellness benefits | Hybrid work schedule | Paid Holidays | Paid maternity and parental bonding leaveSenior-level Full TimeNew Jersey Office - 210 Hudson … R1d ago
-
Junior Software Engineer- Cloud Cost Optimization USD 80K-110KApache Spark | Azure | Azure Databricks | Cloud Resource Utilization | Code review401k match | Career development opportunities | Employee charity matching | Health, dental, and vision insurance | Paid time offEntry-level Full TimeWork at Home - Kentucky, United … R1d ago
-
Bioinformatics Engineer (Hybrid) USD 95K-125KAWS CDK | Amazon Web Services | Batch computing | CI/CD | CWL401k match | Career advancement opportunities | Dental insurance | Disability insurance | Education assistanceEntry-level Full TimeBaltimore, MD, United States R1d ago
-
Senior Data Scientist, Gen AI Application USD 177K-329KA2A protocols | Angular | Cost Optimization | Evaluation | FigmaAnnual bonus | Health benefits | Hybrid work schedule | Relocation not availableSenior-level Full TimeSouth San Francisco, United States R1d ago
-
Principal Data Engineering USD 112K-193KAzure | Azure Data | Azure Data Factory | Batch Processing | CI/CD401k contribution | Comprehensive benefits package | Equity stock purchase | Incentive and recognition programs | Remote work flexibilitySenior-level Full TimeEden Prairie, Minnesota, UNITED STATES; REMOTE R1d ago
-
Airflow | CI/CD | Cause analysis | Data Modeling | Data PipelinesHealth insurance | Paid time off | Remote work | Retirement planSenior-level Full TimeRosemont, IL, United States of America R1d ago
-
GCP AI Engineer - Cambridge, MA USD 143K-194KBigQuery | Cloud Build | Cloud Operations | Containerization | Dataflow401k matching | Dental insurance | Disability insurance | Employee assistance program | Flexible spending accountMid-level Full TimeMassachusetts, United States R1d ago
-
Azure Data and MLOps Engineer USD 119K-222KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Databricks401k match | Dental insurance | Life insurance | Long-term disability | PTOSenior-level ContractUnited States - Remote R1d ago
-
Senior-level Full TimeTwo Destiny Way, Westlake TX, United … R1d ago
-
Senior Data Engineer - Power BI USD 97K-185KAWS Batch | AWS CloudFormation | Agile | Amazon EC2 | Amazon EKSSenior-level Full Time100 New Millennium Way, Bldg 2, … R1d ago
-
Data Engineer USD 94K-175KAWS | AWS Glue | Apache NiFi | Data Visualization | Document DatabasesCareer development | Employee assistance program | Fitness reimbursement | Flexible work-life balance | Headspace accessMid-level Full TimeUnited States of America, McLean, Virginia R1d ago
-
Azure Data Engineer USD 119K-261KAzure Data | Azure Data Factory | Azure Databricks | DLP | Data FactoryDental insurance | Disability insurance | Flexible spending account | Health insurance | Hybrid scheduleSenior-level Full Time5475 Rings Rd, United States R1d ago
-
Principal Data Science USD 126K-255KData Modeling | Data Quality | Data Visualization | Language Processing | Machine LearningEducational assistance | Employee assistance programs | Health care coverage | Learning resources | Paid time offSenior-level Full Time499 Washington Blvd., Jersey City NJ, … R1d ago
-
Principal Engineer - Data Platform USD 221K-387KAccess Control | Apache Spark | Audit Logging | CDC | CompactionSenior-level Full TimeSanta Clara, California, United States R1d ago
-
Software Engineer - Managed Kubernetes USD 225K-300KC plus plus | C# | CI/CD | CNI | CSIHybrid work schedule | On site office days 4 per week | Work from home day TuesdayMid-level Full TimeSan Francisco Office (Fremont St) R1d ago
-
Senior Data Engineer USD 155K-186KBigQuery | Cloud platform | DBT | Data Governance | Data Ingestion401k match | Accident insurance | Dental insurance | Disability insurance | Employee assistance programSenior-level Full TimeSan Francisco, CA R1d ago
-
Data Engineer USD 133K-159KBigQuery | Cloud platform | DBT | Data Governance | Data Modeling401k match | Employee assistance program | Employee stock purchase program | Flexible family planning support | Flexible vacation policyMid-level Full TimeSan Francisco, CA R1d ago
-
Finance Transformation AI Lead USD 106K-171KAPIs | Alteryx | Automation | Data analytics | DatabricksEmotional & mental wellness support | Employee resource groups | Fitness programs | Learning & development programs | Medical/Dental/VisionSenior-level Full TimeBoston, Massachusetts, United States R1d ago
-
Finance Transformation AI Lead USD 106K-171KAPIs | Alteryx | Artificial Intelligence | Cloud APIs | DashboardingDiscretionary paid time off | Emotional and mental wellness support | Employee resource groups | Fitness programs | Learning and development programsSenior-level Full TimeSeattle, Washington, United States R1d ago