Principal Data Engineer (streaming)
Tasks
- Automate data quality checks
- Define SLAs and SLOs for latency and throughput
- Define schema contracts and schema evolution
- Design scalable streaming data pipelines
- Engineer real time data solutions
- Establish observability with metrics logging and distributed tracing
- Implement CI CD integration for data pipelines
- Implement event-driven architectures
- Implement production grade streaming pipelines
- Implement transformation tests and data assertions
- Improve data governance for streaming workloads
- Integrate alerting dashboards and incident response
- Mentor engineers and conduct peer reviews
- Modernize data ecosystem
- Optimize streaming performance tuning
Perks/Benefits
- Allyship and inclusion communities
- Caregiver leave
- Continuous development support program
- Employee assistance program
- Employee recognition
- Fertility and adoption support
- Flexible paid time off
- Parental leave
- Retirement plan matching
- Wellness support programs
Skills/Tech-stack
AWS | Alerting | Apache Flink | Apache Hudi | Apache Kafka | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | CI/CD | Cloud platform | DBT | Dashboards | Data Governance | Data Lake Storage Gen2 | Data Quality | Databricks | Distributed Systems | Distributed tracing | Docker | Event Driven | Event-driven architecture | Google Cloud | Google Cloud Platform | Great Expectations | Kubernetes | Lakehouse | Logging | Low Latency | Metrics | Microsoft Azure | Observability | Performance Tuning | Schema evolution | Spark Streaming
Education
N/A
Roles
Data Engineer | Engineer | Principal | Principal Data Engineer
Related jobs
-
Media Software Engineer, Speech (All Levels) USD 120K-180KAndroid | Artificial Intelligence | Audio Processing | C# | C++401k retirement savings plan | Company holidays | Complimentary lunch and snacks | Fertility support | Medical, dental, and vision insuranceEntry-level Full TimeSunnyvale R1d ago
-
Mid-level Full TimeUnited States R1d ago
-
Principal Machine Learning Engineer USD 180K-368KAWS Lambda | Amazon SQS | Amazon SageMaker | Automated testing | Backend EngineeringHybrid workSenior-level Full TimeRemote, USA R1d ago
-
Senior Machine Learning Engineer USD 198K-287KData Engineering | Fine Tuning | Foundation Models | GenAI | Incident ResponseOn-call rotationSenior-level Full TimeRemote - US R2d ago
-
Senior-level Full TimeRemote, US R2d ago
-
Sr. Staff Machine Learning Engineer, Content Ecosystem USD 227K-469KCausal Inference | Data Quality | Experimentation | Game theory | Language ModelsSenior-level Full TimeSan Francisco, CA, US; Remote, US R2d ago
-
Senior Data Platform Engineer USD 133K-197KAWS | Amazon IAM | Amazon Redshift | Ansible | Apache IcebergDental benefits | Free 1Password account | Generous paid time off | Health benefits | Maternity and Parental Leave Top-UpSenior-level Full TimeRemote (United States | Canada) R2d ago
-
AI Engineer I - Hybrid USD 125K-135KAI Services | API Development | Agentic Workflows | Azure | Azure AIHealth insurance | Hybrid work | Paid time off | Remote work options | Retirement planSenior-level Full TimeWindsor, Colorado, United States R2d ago
-
Senior ML Engineer USD 152K-228KAmazon SageMaker | Baseten | Batching | CI/CD | Google Vertex401k matching | Dental insurance | Fertility assistance | Flexible time off | Health insuranceSenior-level Full TimeRemote - US & Canada (see … R2d ago
-
Data Engineer - Assistant Vice President USD 100K-135KAWS | Apache Airflow | Apache Flink | Apache Spark | AzureEmployer-Matched Retirement Plan | Parental leave | Remote work Friday flexibility | Subsidized healthcare | Unlimited paid time offExecutive-level Full TimeSalt Lake City, Utah, United States R2d ago
-
Sr AI Engineer USD 124K-171KAPIs | Agentic Workflows | Event Driven | Event-driven architecture | JavaScriptFlexible time off | Learning and development stipend | Medical, dental, and vision insurance | Mental wellbeing resources | Paid HolidaysSenior-level Full TimeRemote - United States R2d ago
-
Lead Data Engineer - Data Platform Engineering (Hybrid) USD 128K-168KApache Airflow | Apache Hadoop | Apache Spark | BigQuery | Cloud Healthcare APISenior-level Full TimeMelville, Suffolk, United States R2d ago
-
Staff Machine Learning Engineer, Ads Conversion USD 222K-389KA/B | A/B Testing | B testing | Computer Vision | Data MiningSenior-level Full TimeSan Francisco, CA, US; Palo Alto, … R2d ago
-
Senior Analytics Engineer USD 145K-180KAnomaly Detection | DBT | Data Documentation | Data Modeling | Data TestingFull remote option | Hybrid scheduleSenior-level Full TimeAustin, Texas, United States R2d ago
-
Senior Analytics Engineer USD 135KAnomaly Detection | DBT | Data Documentation | Data Modeling | Data TestingFinancial benefits | Fully remote option | Hybrid schedule | Medical benefits | Stock optionsSenior-level Full TimeDenver, Colorado, United States R2d ago
-
Senior Analytics Engineer USD 135KAmazon Redshift | Anomaly Detection | DBT | Data Modeling | LLMFinancial benefits | Full remote options | Hybrid schedule | Medical benefits | Stock optionsSenior-level Full TimeNew York, New York, United States R2d ago
-
Full-Stack Engineer, AI Data Platform USD 130K-200KAWS DynamoDB | Cassandra | Cloud platform | GCS | Google CloudHybrid workMid-level Full TimeSan Francisco Bay Area R2d ago
-
Azure | Azure AI | Azure Data | Azure Data Factory | Azure DevOpsEU remote work days | Employee assistance program | Employer pension plan | Flexible working hours | JobradSenior-level Full TimeHomeoffice R2d ago
-
Senior Data Engineer II (Hybrid) USD 106K-171KAutomation | CI/CD | DBT | Data Governance | Data ModelingEmployee resource groups | Fitness programs | Learning and development programs | Medical/Dental/Vision | Paid parental leaveMid-level Full TimeScottsdale, Arizona, United States R2d ago
-
Senior Engineer - Ingestion & Streaming Frameworks USD 150K-190KAWS DMS | Airbyte | Amazon Web Services | Apache Airflow | Apache SparkSenior-level Full TimeRemote - United States R2d ago
-
Data Engineer II USD 110K-125KAPI | AWS Glue | AWS Lambda | Alerting | Amazon Redshift4 week paid sabbatical | 401k match | Flexible PTO | Fully remote | Health benefitsMid-level Full TimeRemote, US R2d ago
-
Principal Resident Architect USD 160K-230KAI Agent | Application deployment | High Availability | Infrastructure automation | KubernetesHealth and wellness resources | Wellness Fridays | Work-life balance supportSenior-level Full TimeRemote - United States R2d ago
-
AI-ML Engineer - Finance-Hybrid USD 155K-230KAgentic AI | CI/CD | Cloud Computing | Computer Vision | Data EngineeringContinuing education | Dental insurance | FSA | HSA | Health insuranceSenior-level Full TimeRochester, MN, United States R2d ago
-
DMLSS Business Data Analyst / Data Engineer USD 55K-65KConfluence | Data Governance | Data QA | Data QA/QC | Data QualityFederal clearance support | Remote workMid-level Full TimeRemote, United States R3d ago
-
Data Engineer USD 89K-141KAWS Glue | AWS Lambda | Access Control | Amazon Kinesis | Amazon QuickSightFully remote | Secret clearanceMid-level Full TimeUnited States R3d ago