Lead Principal Data Engineer
NC.Durham Office, United States
USD 126K-261K (estimate) Senior-level Full Time
Tasks
- Apply security and governance for AI including row and column level security
- Build Lakehouse and Warehouse designs
- Define medallion bronze silver gold architecture
- Design and govern OneLake data lake
- Develop ingestion and transformation pipelines
- Establish monitoring, alerting, and observability
- Implement Git source control and CI CD pipelines
- Implement data quality and lineage testing
- Implement retrieval-augmented generation pipelines
- Integrate Azure AI services into data workflows
- Lead technical hiring and mentoring
- Manage outsourced engineering partners
- Optimize capacity and storage for performance and cost
- Own end to end Microsoft Fabric architecture
- Structure governed AI ready data foundation
Perks/Benefits
- N/A
Skills/Tech-stack
AI Foundry | AI Services | Azure AI | Azure AI Foundry | Azure AI Services | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure OpenAI | Azure SQL | Azure Service | Azure Service Bus | Azure Synapse | Azure Synapse Analytics | CI/CD | Data Factory | Data Lake Storage | Dataflows Gen2 | Delta Lake | Direct Lake | Git | Infrastructure as Code | Lakehouse | Microsoft Fabric | Microsoft Purview | OneLake | Parquet | PySpark | Python | RAG | SQL | Service Bus | Spark | Synapse Analytics | Warehouse | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer | Principal | Principal Data Engineer
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R12d ago
-
Mid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL1h ago
-
Senior-level ContractRaleigh, United States2h ago
-
SYSTEM ENGINEER - Computer Network Support - AI/ML - 6+ yrs of Experience - TS/SCI w/Poly clearance is required - ES - 032726-2 A USD 136K-140KAI/ML | Agile | Confluence | Jira | LLM401k retirement plan | Dental insurance | Disability insurance | Federal Holidays | Floating holidaysMid-level Full TimeFort George G Meade, United States2h ago
-
AI experimentation | Alerting | Artificial Intelligence | Data Annotation | Data ArchitectureSenior-level Full TimeNew York3h ago
-
Project Lead Data Engineer III USD 119K-198KBTEQ | Database Administration | Patch Management | Performance Tuning | Query OptimizationSenior-level Full TimeSacramento, California, United States3h ago
-
Software Engineer - TikTok AI Search Infrastructure USD 212K-389KAlgorithm Design | C++ | DAG | Data Architecture | Data ProcessingSenior-level Full TimeSan Jose, California, United States4h ago
-
Machine Learning Engineer / AI Model Developer (Mid to Senior)(Top Secret Clearance Required) (Hybrid) USD 78K-176KAPIs | Agile methodologies | Backend Development | Cloud Computing | Data Preparation401k | Employee discount program | Employee referral rewards | Flexible spending account | Flexible work scheduleMid-level Full TimeFort Belvoir, VA, US R5h ago
-
Software Engineer, Systems ML Engineering USD 170K-251KAlerting | Benchmarking | C++ | CUDA | DashboardingSenior-level Full TimeSunnyvale, CA | Bellevue, WA | …5h ago
-
AI/HPC Network Performance Engineer USD 147K-226KAI Training | Alerting | Auto-remediation | C++ | Configuration ManagementOncall rotationSenior-level Full TimeMenlo Park, CA5h ago
-
Forward Deployed Engineer III, Google Cloud, Applied AI USD 174K-253KAPI Integration | Agent systems | Agentic Workflows | Chatbots | Cloud platformBenefits | Bonus | Equity | Travel up to 50 percentSenior-level Full TimeSan Francisco, CA, USA; Atlanta, GA, …5h ago
-
Research Software Engineer, Multimodal AI USD 174K-253KAgent Orchestration | Audio | C++ | Few-Shot Learning | Few-shotMid-level Full TimeSan Jose, CA, USA5h ago
-
ML Engineer, GenAI Ads, Search Personalization USD 207K-301KC++ | Clustering Algorithms | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeMountain View, CA, USA5h ago
-
Customer Engineer, Data/Analytics, Google Public Sector USD 152K-222KApache Spark | Batch Processing | Big Data | Cloud Architecture | Data GovernanceBonus target | Equity | Health benefits | Paid time off | Retirement benefitsSenior-level Full TimeReston, VA, USA; Atlanta, GA, USA5h ago
-
Senior Software Engineer, AI/ML, Ads Training USD 174K-253KC++ | Distributed Systems | GPU | JAX | Machine LearningSenior-level Full TimeMountain View, CA, USA5h ago
-
Software Engineer, Product USD 208K-269KAgentic Frameworks | Anthropic | Claude Code | Cursor | DocumentationDental insurance | Health insurance | Paid time off | Retirement plan | Vision insuranceMid-level Full TimeSan Francisco, CA10h ago
-
C++ | Containers | Distributed Systems | Distributed Training | Distributed inferenceSenior-level Full TimeCupertino11h ago
-
Machine Learning Engineer USD 136K-215KBayesian Inference | Code review | Deep learning | Git | Machine LearningMid-level Full TimeSanta Barbara, CA, US12h ago
-
Senior-level Full TimeUnited States12h ago
-
Sr. Manager Data Engineering USD 125K-188KAutomated testing | Bitbucket | CI/CD | Cloud Functions | Cloud RunSenior-level Full TimeSouthlake, TX, United States13h ago
-
Lead Software Engineer - Applied AI ML Lead USD 170K-195KApache Airflow | Apache Iceberg | Apache Kafka | Apache Spark | Automated testingBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimePalo Alto, CA, United States15h ago
-
Data Analysis | Data Visualization | GenAI | Java | Jupyter NotebookSenior-level Full TimeCupertino16h ago
-
Forward Deployed AI Engineer USD 180K-230KAPI | AWS | Azure | CI/CD | Cloud platform5-day workweek | Collaborative work culture | Flexible working hours | Supportive work environmentSenior-level Full TimeNew York, New York, United States16h ago
-
Principal AI Engineer USD 200K-245KAgent-based | Agent-based architecture | Amazon Web Services | Azure | CI/CD5 days per week | Collaborative culture | Flexible working hours | Supportive work environmentSenior-level Full TimeNew York, New York, United States16h ago
-
Data Engineer, Consultant USD 125K-164KApache Airflow | Azure | Azure Data | Azure Data Factory | Azure Data LakeSenior-level Full TimeOakland, CA, United States16h ago