Principal Engineer - Data Ingestion & AI Pipeline
USD 122K-173K (estimate) Senior-level Full Time
Tasks
- Build scalable ingestion patterns for enterprise sources
- Create reusable patterns reference architectures standards and guardrails
- Define data quality checks for completeness accuracy duplication stale content unsupported formats sensitive information
- Define engineering approach for program portfolio solutions
- Define technology tool stack within approved technologies
- Define transformation patterns for parsing text extraction normalization deduplication enrichment chunking classification metadata generation
- Design enterprise data ingestion pipelines
- Design incremental ingestion change detection delta processing and reprocessing
- Ensure ingestion compliance with security privacy retention and regulatory requirements
- Establish end to end test strategy and integration between teams
- Establish ingestion standards for lineage freshness versioning access control auditability
- Evaluate and implement OCR document intelligence parsing entity extraction classification and data quality
- Implement pipeline orchestration monitoring retry logic error handling dashboards
- Lead planning definition and design for complex features across teams
- Mentor senior engineers and influence technical direction
- Optimize ingested data for embedding indexing and retrieval for RAG
- Own architecture decisions impacting multiple teams systems or domains
- Provide technical leadership across data engineering AI platform cloud and application teams
- Provide technical oversight through design reviews and code within domain
- Structure and enrich data for effective LLM reasoning
Perks/Benefits
Skills/Tech-stack
AI Document Intelligence | Access Control | Apache Airflow | Apache Kafka | Apache Spark | Azure AI | Azure AI Document Intelligence | Azure Data | Azure Data Factory | Azure Event | Azure Event Hubs | Azure Fabric | Azure Synapse | Azure Synapse Analytics | Batch Processing | Change detection | Content Classification | DBT | DLP | Data Factory | Data Governance | Data Lineage | Data Quality | Data Security | Data freshness | Databricks | Delta Processing | Document Intelligence | Document parsing | ELT | ETL | Embeddings | Entity Extraction | Error Handling | Event Hubs | Incremental processing | LLM | OCR | PII Detection | Privacy Compliance | Python | RAG | Retry logic | SQL | Schema inference | Semantic Search | Streaming | Synapse Analytics | Text extraction | Vector Databases | Vector Search
Education
Bachelor of Engineering | Bachelor of Engineering in Computer Science | Bachelor of Science | Bachelor of Science in Computer Science
Roles
AI | AI Engineer | Data Engineering | Data Engineering Manager | Engineer | Engineering Manager | Manager | Principal | Principal Engineer
Regions
Countries
States
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R11d ago
-
Senior-level Full TimeOrlando, FL, United States7h ago
-
Data Analytics & Engineering Opportunities USD 65K-105KBusiness Intelligence | Microstrategy | Python | R | SQL401k match | Dental insurance | Disability insurance | Firm paid holidays | Health insuranceEntry-level Full TimeWashington, DC, United States7h ago
-
Artificial Intelligence | C plus plus | CUDA | Camera Calibration | Computer VisionVisa sponsorship not availableMid-level Full TimeKenner, LA, United States9h ago
-
Solutions Engineer USD 95K-130KAgile | Audit controls | Bug Fixing | Dashboards | Data LineageOnsite positionMid-level Full TimeNew Albany, OH, United States10h ago
-
Manager, Quantitative Consulting USD 116K-192KAudit Trail | Credit Risk | Data strategy | Expected shortfall | FRTBMid-level Full TimeUS-NC-Charlotte, US-NC-Charlotte, US-NY-New York, United States, …10h ago
-
AI and Analytics Delivery Project Manager USD 131K-218KAWS | Cloud Architecture | Data Analysis | Data Modeling | DatabricksHybrid work | Up to 10 percent travelSenior-level Full TimeGilbert, Arizona, United States; Lake Mary, …10h ago
-
Reindeer- Software Engineer (Infrastructure) USD 180K-250KAPI Integration | AWS | Authentication | Authorization | AzureSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL11h ago
-
Reindeer- AI Solutions Engineer (IL) USD 80K-170KApplication Integration | Cloud Computing | Communication | Compliance | Data AnalysisSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL11h ago
-
Agent systems | Alignment Training | C++ | CoT prompting | Computer VisionSenior-level Full TimeSeattle, Washington, United States12h ago
-
C++ | DPO | GNN | Graph Neural Networks | Heterogeneous GraphsEntry-level Full TimeSeattle, Washington, United States12h ago
-
Forward Deployed Engineer IV, Applied AI, Google Cloud USD 207K-301KAPI Integration | Agent systems | Cloud platform | Conversational AI | DebuggingTravel up to 50%Senior-level Full TimeSan Francisco, CA, USA; Austin, TX, …13h ago
-
Senior Software Engineer, AI/ML, Search Ecosystems USD 174K-253KAlgorithms | C++ | Data Processing | Data Structures | Data debuggingSenior-level Full TimeMountain View, CA, USA13h ago
-
C++ | Data Processing | Debugging | Distributed Computing | Machine LearningSenior-level Full TimeSunnyvale, CA, USA; New York, NY, …13h ago
-
Deep learning | Isaac Sim | Jetson | Language Models | Large Language ModelsSenior-level Full TimeMountain View, CA, USA13h ago
-
Senior Lead Data Architect USD 170K-205KAI-assisted validation | Access Controls | Agent Toolchains | Application development | Artificial IntelligenceBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimePlano, TX, United States17h ago
-
Data Analytics Architect USD 136K-220KArtificial Intelligence | Business Intelligence | Data Engineering | Data Governance | Data LakeHybrid work | One day per week remoteSenior-level Full TimeCenter, Center District, IL18h ago
-
Senior Manager - Analytics Engineering USD 155K-213KApache Airflow | Automation | CI/CD | Code review | DBTSenior-level Full TimeUnited States19h ago
-
Mid-level Full TimeRichardson, TX, United States21h ago
-
ASIL | Admittance control | Auto Docking | BLDC | C++Mid-level Full TimeIrvine, CA22h ago
-
Product Design & HRI Engineer, Robotics Hardware USD 119K-300K3D Printing | Acoustic Performance Evaluation | Audio Systems | CAD | CESenior-level Full TimeIrvine, CA23h ago
-
Analytics Engineer USD 118K-177KAWS | Airflow | Apache Spark | Census | DBT401k match | Dental insurance | Health insurance | Life insurance | Tuition reimbursementSenior-level Full TimeNewton Office , United States1d ago
-
Senior Director, Data Engineering USD 182K-291KAWS | Agile | CI/CD | Change Data Capture | ContainerizationSenior-level Full TimeNewton Office , United States1d ago
-
Lead AI Solutions Systems Engineer USD 142K-188KAPI Design | Agentic AI | Artificial Intelligence | Bias detection | CI/CDSenior-level Full TimeHerndon, VA, United States1d ago
-
Principal Algorithm Engineer – Imaging / Ultrasound (ML) USD 160K-241KAlgorithm Development | Applied Mathematics | C++ | Computer Vision | Geometric Reconstruction401k match | Dental insurance | Employee assistance program | Employee stock purchase plan | Health insuranceSenior-level Full TimeUSA-MA Newton, United States1d ago