Principal Data Engineer (streaming)
Tasks
- Automate data quality checks
- Define SLAs and SLOs for latency and throughput
- Define schema contracts and schema evolution
- Design scalable streaming data pipelines
- Engineer real time data solutions
- Establish observability with metrics logging and distributed tracing
- Implement CI CD integration for data pipelines
- Implement event-driven architectures
- Implement production grade streaming pipelines
- Implement transformation tests and data assertions
- Improve data governance for streaming workloads
- Integrate alerting dashboards and incident response
- Mentor engineers and conduct peer reviews
- Modernize data ecosystem
- Optimize streaming performance tuning
Perks/Benefits
- Allyship and inclusion communities
- Caregiver leave
- Continuous development support program
- Employee assistance program
- Employee recognition
- Fertility and adoption support
- Flexible paid time off
- Parental leave
- Retirement plan matching
- Wellness support programs
Skills/Tech-stack
AWS | Alerting | Apache Flink | Apache Hudi | Apache Kafka | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | CI/CD | Cloud platform | DBT | Dashboards | Data Governance | Data Lake Storage Gen2 | Data Quality | Databricks | Distributed Systems | Distributed tracing | Docker | Event Driven | Event-driven architecture | Google Cloud | Google Cloud Platform | Great Expectations | Kubernetes | Lakehouse | Logging | Low Latency | Metrics | Microsoft Azure | Observability | Performance Tuning | Schema evolution | Spark Streaming
Education
N/A
Roles
Data Engineer | Engineer | Principal | Principal Data Engineer
Related jobs
-
ACME | AWS CloudHSM | AWS KMS | Asymmetric Cryptography | Azure Key Vault401k plan with company matching | Bereavement leave | Employee assistance program | Employee discount program | Health, dental, and vision careSenior-level Full TimeRemote - Nationwide, United States R7h ago
-
Data Engineer-Secret Clearance Required USD 100K-127KAWS | AWS Glue | AWS Redshift | Azure | Azure Data401k match | Bereavement leave | Disability insurance | Employee assistance program | Employee discount programSenior-level Full TimeRemote - Nationwide, United States R7h ago
-
IT Data Engineer USD 60K-65KANSI X12 | AWS | Azure | Azure Data | Azure Data Factory401k retirement plan | Disability coverage | Employee assistance program | Flexible spending account | Medical/Dental/Vision insuranceMid-level Full TimeUS - Remote R12h ago
-
Sr. AI Engineer USD 150K-175KAccess Control | Agentic AI | Auditability | CI/CD | Cloud platform401k | Dental insurance | Expense reimbursement for internet costs | Life insurance | Medical insuranceSenior-level Full TimeRemote, USA, United States R12h ago
-
AI Research Engineer (Applied AI) USD 100K-150KAgent systems | Computer Vision | Data Quality | Data quality monitoring | Deep learningMid-level Full TimeUnited States - Remote R1d ago
-
Quantitative Developer (Fintech) USD 100K-150KAudit trails | Backtesting | C++ | Cloud Native | Cloud-native ArchitecturesBenefits | Career growth | Remote workMid-level Full TimeUnited States - Remote R1d ago
-
AWS | Access Control | Apache Spark | Azure | CI/CD401k retirement plan | Comprehensive benefits | Dental insurance | Employee assistance program | Medical insuranceSenior-level Full TimeUSA, TX, Work-at-Home, United States R1d ago
-
Associate Director, Data Science USD 160K-297KADME | Bioinformatics | Data Analysis | Data Governance | Data Mining401k match | Hybrid work | Paid time offMid-level Full TimeCambridge (USA), United States R1d ago
-
Senior-level Full TimeCanada R2d ago
-
Senior-level Full TimeUnited States R2d ago
-
Senior-level Full TimeUnited States R2d ago
-
Deep Learning Quality Specialist USD 72K-90KAnnotation Guidelines | Computer Vision | Confluence | Convolutional Neural Networks | Data Annotation401k plan | Commuter benefits | Employee assistance program | Flexible PTO | Fully paid medical/dental/visionMid-level Full TimeSeattle, WA R3d ago
-
Corporate AI Engineer USD 154K-200KAPI Integration | Access Control | Data Quality | Embeddings | Generative AIHybrid work schedule | Volunteer time offMid-level Full TimeAddison, TX (Hybrid); Bellevue, WA (Hybrid); … R3d ago
-
FDE Data Engineer- Space USD 123K-166KABAC | Apache Airflow | Apache Flink | Apache Hudi | Apache IcebergHealth benefits | Remote work | Travel for site workMid-level Full TimeUnited States - Remote R3d ago
-
Senior FDE Data Engineer USD 148K-201KABAC | Airflow | Apache Iceberg | Apache Kafka | Argo WorkflowsCompany paid premiums | Health savings account | Medical/Dental/Vision | Remote work | Travel to classified sitesSenior-level Full TimeUnited States - Remote R3d ago
-
AI Observability | AWS | Azure | CI/CD | Cost ControlCareer advancement | Fully remote work | Professional development opportunities | Work-life balanceSenior-level Full TimeCanada R3d ago
-
Senior Data Engineer 🇺🇸 USD 160K-200KAWS Glue | AWS Redshift | Amazon S3 | Apache Spark | Automated testingSenior-level Full TimeHybrid (New York, New York, US) R3d ago
-
AWS Data Engineer - Fully Remote - US Only USD 139K-210KAWS Glue | AWS Lambda | AWS Step Functions | Amazon DynamoDB | Amazon RedshiftAbility to work independently | Fully remote | US onlySenior-level Full TimePlano, Texas, United States - Remote R3d ago
-
AI Workflow Orchestration | AI workflow | AWS DynamoDB | AWS Lambda | AWS Step FunctionsArchitectural influence | Engineering Led Collaboration | High technical ownership | Learning opportunities | Remote-first work modelSenior-level Full TimeCanada R3d ago
-
Sr. AI Engineer USD 150K-175KAccess Control | Agentic Frameworks | Auditability | CI/CD | Cloud Native401-k match | Dental insurance | Expense Reimbursement for Home Office | Life insurance | Medical insuranceSenior-level Full TimeRemote, USA, United States R3d ago
-
Senior Analytics Engineer USD 159K-200KAWS | Airflow | DBT | Dagster | Data ObservabilityAutonomy | Fully remote | High-impact work | Use of AI toolsSenior-level Full TimeRemote US R3d ago
-
Senior-level Full TimeSan Jose, United States R4d ago
-
AEP Debugger | AEP Web SDK | Adobe Experience | Adobe Experience Platform | Adobe TagsSenior-level Full TimeLehi, United States R4d ago
-
Lead Data Engineer USD 188K-230KAirflow | Apache Spark | Azure Cosmos | Azure Cosmos DB | Azure DataDomestic travel up to 5 percent | Relocation not authorized | Remote workSenior-level Full TimeRemote - Minnesota, United States R4d ago
-
BigQuery | Cloud Data | Cloud data platform | Code review | DBTLong-term contract | Onsite work in Atlanta GA metro area | Potential conversion to full time | W2 employmentSenior-level Contract Full TimeAtlanta, Georgia, United States R4d ago