Data Quality Lead

Tasks

• Align catalog metadata with AI features
• Build AI-ready data quality platform
• Build schema validation into CI/CD workflows
• Build self-service data monitoring tools
• Define and ship data quality capabilities
• Define data product performance metrics
• Define governed datasets for AI
• Design real-time anomaly detection
• Drive metadata lineage standardization
• Embed data contracts in pipelines
• Enable automated drift detection
• Establish data ownership models
• Implement observability for streaming and batch data
• Partner with ML teams on reproducibility
• Partner with data engineering to integrate dbt
• Productize policies and controls in Unity Catalog
• Promote data quality culture
• Support model lifecycle and dataset versioning

Perks/Benefits

• N/A

Skills/Tech-stack

AI/ML | AI/ML Infrastructure | CI/CD | DBT | Data Contracts | Data Governance | Data Lineage | Data Observability | Data mesh | Databricks Lakehouse | Feature Stores | Kafka | ML Infrastructure | Python | SQL | Scala | Schema evolution | Spark Structured Streaming | Streaming Frameworks | Structured Streaming | Unity Catalog