R&D principal iot data pipeline architect
Tasks
- Define Azure and Databricks platform strategy and guardrails
- Define disaster recovery backup restore RTO RPO and recovery testing strategies
- Define event schemas data models and integration contracts
- Define governance for metadata stewardship classification retention lineage and auditability
- Design and implement IoT device to cloud pipelines
- Drive capacity planning workload isolation scale testing backfill and migration
- Enforce least privilege access managed identities secrets and encryption
- Ensure tenant isolation and security review and threat modeling
- Establish data reference architectures and engineering standards
- Establish observability and service objectives for ingestion and latency freshness availability performance recovery
- Implement Unity Catalog architecture and enterprise governance integration
- Implement data quality lineage observability and ownership
- Implement infrastructure as code and CI CD and automated testing
- Implement lakehouse and streaming data processing patterns
- Mentor engineers and communicate technical tradeoffs and risks
- Own platform roadmap and target architecture
- Own telemetry lifecycle from ingestion validation enrichment storage serving retention and deletion
- Perform integration and performance and security testing
- Provide technical authority for architecture reviews and exception controls
Perks/Benefits
Skills/Tech-stack
API Design | Apache Kafka | Automated testing | Azure | C# | CI/CD | Data Lineage | Data Modeling | Data Quality | Databricks | Delta Lake | Disaster Recovery | Disaster Recovery Testing | Distributed Systems | Encryption | Event Driven | Event Hubs | Event-driven architecture | Infrastructure as Code | IoT Hub | Java | Key Management | Lakehouse | Least Privilege | Managed Identities | Multi-tenant | Multi-tenant architecture | Observability | Python | Recovery testing | SQL | Scala | Security Engineering | Spark | Streaming processing | Structured Streaming | Tenant architecture | Threat modeling | Unity Catalog | “as-code”
Related jobs
-
Solutions Architect USD 180K-247KAccount Management | Artificial Intelligence | Big Data | Cloud Computing | Data ScienceSenior-level Full TimeSan Francisco, California R22h ago
-
Acquisition Funnel | Acquisition Funnel Analysis | Budgeting | Campaign Measurement | Data VisualizationSenior-level Full TimeUnited States R23h ago
-
Manager, Delivery Solutions Architects - Public Sector USD 197K-270KArchitecture | Artificial Intelligence | Data Engineering | Data Science | Data WarehousingSenior-level Full TimeMaryland; Remote - Virginia; Remote - … R23h ago
-
Agentic AI | BigQuery | Cloud Composer | Cloud SQL | Cloud Storage401k | Dental insurance | Employee assistance program | Health insurance | Paid time offSenior-level Full TimeSunnyvale, California, USA R23h ago
-
Agentic AI | BigQuery | Cloud Composer | Cloud SQL | Cloud Storage401k | Employee assistance program | Health savings account | Life insurance | Long-term disabilitySenior-level Full TimeNew York, New York, USA R23h ago
-
Director, GTM Systems & Applied AI USD 240K-282KAI Agents | Artificial Intelligence | CRM | Clay | ClearbitCommuter stipend | Flexible PTO | Learning and development allowance | Lunch stipend | Medical, dental, and vision coverageExecutive-level Full TimeSan Francisco, California, United States R23h ago
-
Principal, Data/Analytics and Benchmarking USD 129K-186KAI | Benchmarking | Dashboarding | Data Governance | Data ModelingSenior-level Full TimeUnited States R1d ago
-
Principal Data Scientist USD 96K-160KAnomaly Detection | Apache Spark | Bayesian Inference | Causal Inference | Classification401k company match | Dental insurance | Disability insurance | Medical insurance | Mileage reimbursementSenior-level Full TimeFrisco, Texas, United States R1d ago
-
AI Systems & ML Engineering Industry Expert USD 140K-209KA/B | A/B Testing | API Design | AWS | Agentic SystemsFlexible schedule | Fully remote | Hourly payment | International teamSenior-level Part TimeNew York City, NY, US R1d ago
-
Azure Practice Architect - AI/ML (Full-Time Remote) USD 156K-234KAKS | ARM Templates | Anomaly Detection | Azure | Azure CLI401k | Dental insurance | Employee assistance program | Fully remote | Health insuranceSenior-level Full TimeDallas, Texas, USA R1d ago
-
AI architecture | AI orchestration | API Design | Access Control | Agentic AIMid-level Full TimePrinceton Pike - NJ, United States R1d ago
-
Principal AI/ML Engineer (Large Language Model) USD 114K-252KAWS | BERT | CUDA | Computer Vision | DockerContinuing education | Family support | Flexible time off | Healthcare | Learning resourcesSenior-level Full Time602 AURORA CO, United States R1d ago
-
AKS | Agent Based Workflows | Agent systems | Agent-based | Anomaly DetectionRemote workSenior-level Full TimePlano, TX, US R1d ago
-
Senior Data Platform Architect USD 140K-225KAPI Integration | AWS | AWS Lambda | Amazon EC2 | Amazon RDSSenior-level Full TimeMcLean Office, United States R1d ago
-
Change Management | Cloud Data | Cloud data warehousing | DBT | Data EngineeringHybrid work scheduleEntry-level Full TimeSan Francisco, CA, United States R1d ago
-
Senior Solutions Architect - Consumer Product Goods USD 219K-301KArtificial Intelligence | Big Data | Databricks | Hadoop | JavaSenior-level Full TimeDallas, Texas; Missouri; Remote - Missouri; … R1d ago
-
A/B | A/B Testing | AI Safety | B testing | Data Pipelines401k | Community resource groups | Dental insurance | Disability insurance | Employee assistance programSenior-level Full TimeRemote - United States R2d ago
-
Principal Data Architect — Databricks Enablement USD 165K-185KAWS | Amazon Bedrock | Amazon SageMaker | Apache Spark | BI Dashboards100% remote work | 401k plan | Company laptop | Dental insurance | Equipment and office stipendSenior-level Full TimeUSA R2d ago
-
Agentic AI Architect-Anthropic-US East USD 275K-365KAPIs | AWS Lambda | AWS SageMaker | Access Control | Access ManagementClient travel | Conference travelSenior-level Full TimeUSA - Remote R2d ago
-
Agentic AI Architect-Anthropic-US West USD 270K-345KAPIs | AWS Lambda | AWS S3 | AWS SageMaker | Agentic AISenior-level Full TimeUSA - Remote R2d ago
-
Data Modeling | Data Quality | Data Visualization | Data Warehousing | Database DesignSenior-level Full TimeIowa City R2d ago
-
Alation | Amazon Redshift | Apache Kafka | BigQuery | ClusteringHybrid workSenior-level Full TimeNew York, NY, USA R2d ago
-
Principal Solutions Engineer, Data & AI USD 175K-200KAPIs | CRM | Databricks | Delta Sharing | HIPAA401k | Company on site visits | Dental insurance | Fully remote | Health and wellness stipendSenior-level Full TimeUnited States - Remote R2d ago
-
Principal Data Scientist USD 120K-231KAlteryx | Causal Inference | Data Visualization | Data Warehousing | Distributed ComputingAdoption Assistance | Dental insurance | Health insurance | Life insurance | Matched 401kSenior-level Full TimeOne Verizon Way, Basking Ridge, NJ … R2d ago
-
Domain Architect- AI/ML, Senior Specialist USD 140K-205KAPI Development | Agentic AI | Amazon Elastic Kubernetes Service | Artificial Intelligence | Distributed SystemsSenior-level Full TimeUSA - Majestic, United States R2d ago