Lead Principal Data Engineer
NC.Durham Office, United States
USD 126K-261K (estimate) Senior-level Full Time
Tasks
- Apply security and governance for AI including row and column level security
- Build Lakehouse and Warehouse designs
- Define medallion bronze silver gold architecture
- Design and govern OneLake data lake
- Develop ingestion and transformation pipelines
- Establish monitoring, alerting, and observability
- Implement Git source control and CI CD pipelines
- Implement data quality and lineage testing
- Implement retrieval-augmented generation pipelines
- Integrate Azure AI services into data workflows
- Lead technical hiring and mentoring
- Manage outsourced engineering partners
- Optimize capacity and storage for performance and cost
- Own end to end Microsoft Fabric architecture
- Structure governed AI ready data foundation
Perks/Benefits
- N/A
Skills/Tech-stack
AI Foundry | AI Services | Azure AI | Azure AI Foundry | Azure AI Services | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure OpenAI | Azure SQL | Azure Service | Azure Service Bus | Azure Synapse | Azure Synapse Analytics | CI/CD | Data Factory | Data Lake Storage | Dataflows Gen2 | Delta Lake | Direct Lake | Git | Infrastructure as Code | Lakehouse | Microsoft Fabric | Microsoft Purview | OneLake | Parquet | PySpark | Python | RAG | SQL | Service Bus | Spark | Synapse Analytics | Warehouse | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer | Principal | Principal Data Engineer
Regions
Countries
States
Cities
Related jobs
-
AC DC Sizing | AC/DC | Arc Flash | Artificial Intelligence | AutomationMultiple project extensions | Paid on time | Remote work optionSenior-level Full TimeJuno Beach, FL R1d ago
-
AI Agents | API Development | Azure | CI/CD | LLMOpsCollaborative cross-functional team | Fully remoteMid-level Full TimeCanada, Ontario; Toronto, None; Toronto, Ontario; … R1d ago
-
API Integration | AWS Bedrock | Auditing | Authentication | AuthorizationMentoring | MentorshipMid-level Full TimeMenlo Park, CA, United States1d ago
-
Analytics and Insights Engineer III USD 131K-218KApache Spark | Data Governance | Data Modeling | Data Pipelines | Data QualitySenior-level Full TimeRichmond, Virginia, United States1d ago
-
3D Rendering | A/B | A/B Testing | Agentic AI | AlgorithmsSenior-level Full TimeSan Jose, California, United States1d ago
-
C++ | Computer Vision | Data Processing | Deep learning | Domain AdaptationEntry-level Full TimeSan Jose, California, United States1d ago
-
Senior Software Engineer, Google Cloud Dataproc USD 174K-252KApache Hive | Apache Spark | C++ | Data Warehouse | Data Warehouse InternalsSenior-level Full TimeSunnyvale, CA, USA1d ago
-
Agentic Orchestration | C++ | Data Onboarding | Data Pipeline Engineering | Data pipelineSenior-level Full TimeSan Jose, CA, USA1d ago
-
Senior Software Engineer, AI/ML, Core USD 174K-252KC++ | Data Processing | Data Structures | Data Structures and Algorithms | DebuggingSenior-level Full TimeMountain View, CA, USA1d ago
-
Staff Software Engineer, Search Local Ads Data Platform USD 207K-300KC plus plus | CSS | Data Storage | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeMountain View, CA, USA1d ago
-
Analytics Data Engineer USD 94K-130KAzure Data | Azure Data Factory | CI/CD | Data Analysis | Data FactoryRemote workMid-level Full TimeFlagstaff, AZ, US1d ago
-
AI Application & Cloud Operations Engineer (Mid level) USD 108K-117KAWS | AWS Bedrock | AWS CloudTrail | AWS CloudWatch | AWS PrivateLinkMid-level Contract Full TimePlano, TX, United States1d ago
-
Benchmarking | C plus plus | CUDA | Computational Engineering | Distributed Computing401k match | Generous paid time off | Medical, dental & vision coverage | Stock options | WFH Hardware ReimbursementSenior-level Full TimeSunnyvale, CA R1d ago
-
Software Engineer, Applied AI USD 173K-250KABAC | Agentic Frameworks | Apache Airflow | Audit trails | BIMDental insurance | Generous PTO | Health insurance | Retirement or Pension Plan | Vision insuranceMid-level Full TimeSan Francisco, CA1d ago
-
Staff Data Engineer USD 147K-184KAmazon Kinesis | Amazon Web Services | Apache Airflow | Apache Iceberg | Apache SparkDental insurance coverage | Health insurance coverage | Paid parental leave | Paid vacation | Relocation assistanceSenior-level Full TimeBerkeley, CA2d ago
-
Associate, Client Intelligence (Data Scientist) USD 98K-132KAWS | Apache Spark | Azure | Big Data | Cloud platformEmployee resource groups | Matching gift program | Professional development | Sustainability and Climate Change Initiatives | Volunteerism programMid-level Full TimeNew York, NY, US, 100012d ago
-
Apple Neural Engine | C++ | CPU | Compilers | Computer ArchitectureMid-level Full TimeSeattle2d ago
-
On-Device ML Infrastructure Engineer (CoreML Runtime), Graphics, Games and Machine Learning USD 170K-296KC++ | Core ML | GPU Programming | JAX | LLVMMid-level Full TimeCupertino2d ago
-
Apple Neural Engine | C plus plus | CPU performance | Compilers | Computer ArchitectureMid-level Full TimeCupertino2d ago
-
AI Developer - Model Creation & Full Stack USD 130K-145KAWS | Aiohttp | Angular | Asyncio | Azure DatabricksRemote work | USPS Public Trust Clearance supportMid-level Full TimeWork from home, VA, United States2d ago
-
GTM Data Analytics Engineer USD 120K-150KAmplitude | DBT | Hex | Metabase | PythonHealth insurance | Startup equityMid-level Full TimeSan Francisco2d ago
-
API Gateway | AWS Lambda | AWS Step Functions | Amazon SQS | Apache Airflow401k matching | Employee stock options | Health benefits | Hybrid work onsite 2 to 3 days per week | Paid time offExecutive-level Full TimeNew York, United States R2d ago
-
AIOps | Anomaly Detection | Anthos | CRD | Control Plane401k plan | Basic life insurance | Life insurance | Long-term disability | Medical/Dental/Vision insuranceSenior-level Full TimeResearch Triangle Park, North Carolina, United … R2d ago
-
Senior Machine Learning Engineer USD 160K-200KAWS | CI/CD | Computer Vision | Data Processing | DockerCommuter benefits | Dental insurance | Employee assistance program | Equity | Health insuranceSenior-level Full TimeArlington, VA R2d ago
-
Senior Computer Vision & Machine Learning Engineer USD 140K-206KAblation Studies | CI/CD | Data Pipelines | Docker | Error AnalysisAutonomy | Client project impact | Support for professional growthSenior-level Full TimeRemote, US R2d ago