AI Pipeline Engineer
Tasks
- Build high throughput data loading for GPU utilization
- Build ingestion systems for multimodal data
- Collaborate with ML researchers to align data systems with model needs
- Construct evaluation datasets with integrity and contamination controls
- Design data pipelines for AI training evaluation and continual improvement
- Design storage architectures for cost throughput and latency
- Develop dataset versioning lineage and provenance tracking
- Document data systems schemas and operating procedures
- Implement data cleaning deduplication filtering and quality assurance at scale
- Implement data privacy redaction and consent enforcement
- Implement labeling workflows active learning and human in the loop improvement
- Monitor data quality drift and pipeline health
- Optimize cost and performance using compression format selection and caching
Perks/Benefits
Skills/Tech-stack
Apache Beam | CI/CD | Caching | Code review | Compression | Data Drift | Data Lineage | Data Modeling | Data Privacy | Data Quality | Data loading | Data provenance | Data quality monitoring | Data redaction | Dataset versioning | Distributed Systems | High Throughput | High Throughput Data Loading | High-throughput data | Java | Observability | Python | Quality monitoring | Ray | Scala | Spark | Testing
Education
Roles
AI | AI Pipeline Engineer | Data Engineer | Engineer | Pipeline Engineer
Related jobs
-
Applied AI Architect, Strategic Enterprise Tech USD 240K-315KAPI Integration | Cloud Architecture | Data Science | Enterprise Integration | LLMFlexible work | Generous vacation | Parental leaveSenior-level Full TimeSan Francisco, CA | New York … R1d ago
-
Apache Flink | Apache Iceberg | Apache Spark | Avro | Concurrency401k match | ESPP | Family leave | Flexible time off | Flexible work scheduleSenior-level Full TimeSan Diego, CALIFORNIA, United States R1d ago
-
Senior Staff Data Platform Engineer - Data Access Team USD 201K-352KAlgorithms | CI/CD | Data Structures | Design Patterns | Distributed Systems401k match | ESPP | Family leave programs | Flexible spending accounts | Flexible time offSenior-level Full TimeSanta Clara, CALIFORNIA, United States R1d ago
-
Data Engineer – Databricks USD 152K-200KAWS | Apache Airflow | Azure | Azure Data | Azure Data FactoryContract-to-hire | Hybrid work onsite 3 days per weekSenior-level Contract Full TimeBeaverton, OR, United States R1d ago
-
Data Scientist - Foward Deployed Engineer USD 160K-170KAPIs | Agentic Workflows | Anomaly Detection | Context engineering | Experimentation401k | Employee assistance program | Health spending account | Life insurance | Long-term disabilityMid-level ContractPalo Alto, California, USA R1d ago
-
Data Engineer II USD 78K-117KCI/CD | Cortex AI | Data Lake | Data Warehouse | ETL401k retirement plan | Employee assistance program | Health savings account | Life insurance | Long-term disabilityMid-level Full TimeBaltimore, Maryland, USA R1d ago
-
Data Engineer USD 101K-130KBI | DBT | Data Modeling | Data Monitoring | Data QualityHybrid work | On site 3 to 4 days per week | Self service analytics enablementMid-level Full TimeDallas, TX R1d ago
-
Fullstack AI Engineer USD 117K-147KAWS | Anthropic | CI/CD | Go | JavaScriptDental insurance | Flexible time off | Medical insurance | Paid parental leave | SabbaticalMid-level Full TimeUnited States (Remote-First) R1d ago
-
Staff+ Software Engineer, RL Data Platform USD 320K-405KAPI | Crowdsourcing | Data Modeling | Data pipeline | LLMFlexible working hours | Generous vacation | Hybrid work policy | Optional equity donation matching | Parental leaveSenior-level Full TimeSan Francisco, CA | New York … R1d ago
-
Staff+ Research Engineer, RL Data Platform USD 500K-850KData pipeline | LLM Evaluation | Latency optimization | Machine Learning | MonitoringFlexible working hours | Generous vacation | Hybrid work policy | Optional equity donation matching | Parental leaveSenior-level Full TimeSan Francisco, CA | New York … R1d ago
-
Analytics Engineer USD 95K-105KBigQuery | Cursor | DBT | Data Contracts | Data Governance401k | Anniversary Milestone Celebrations | Flexible PTO | HSA contribution | Home internet reimbursementMid-level Full TimeRemote, United States R1d ago
-
Sr. Data Engineer (Hybrid - Addison, TX) USD 120K-150KAzure | Azure Data | Azure Data Factory | Data Cleansing | Data FactoryHybrid work schedule | Knowledge sharing culture | Onsite commute requirement 3 days per weekSenior-level Full TimeAddison, TX, United States R1d ago
-
Data Engineer II USD 150K-225KAmazon EKS | Amazon MSK | Amazon Web Services | Apache Flink | Apache Kafka401k match | Dental insurance | Employee discounts | Medical insurance | Paid time offMid-level Full TimeUnited States, Aliso Viejo, CA R2d ago
-
APIs | Access Control | Agentic Workflows | Cloud Computing | Context engineeringTravel opportunitiesSenior-level Full TimeUSA - Remote R2d ago
-
Sr. Software Engineer USD 150K-190KAI workflows | API | AWS | Agentic Workflows | Coordinate systems401k match | Dental insurance | Family plan benefits | Flexible time off | Hybrid work modelSenior-level Full TimeAtlanta, GA R2d ago
-
Apache Flink | Apache Iceberg | Apache Spark | Avro | Concurrency401k match | ESPP | Family leave | Flexible time away | Flexible-hybrid workSenior-level Full TimeSan Diego, CALIFORNIA, United States R2d ago
-
Senior Manager, AI Software Engineering USD 199K-235KAWS | Agile | Automated testing | CI/CD | Cloud Native401k match | Flexible time off | Hybrid work model | Medical insuranceSenior-level Full TimeNew York, NY R2d ago
-
Senior Manager, AI Software Engineering USD 177K-220KAWS | Agile | Automated testing | CI/CD | Claude401k match | Dental insurance | Family benefits | Flexible time off | Hybrid work modelSenior-level Full TimeAtlanta, GA R2d ago
-
API Integration | Access Control | Agent Packs | Agentic Workflows | Anthropic APICustomer facing delivery opportunities | Remote work | Travel 25 percentSenior-level Full TimeUSA - Remote R2d ago
-
Data and Analytics - AI Engineer II USD 100K-143KAI Foundry | AIOps | APIs | Azure | Azure AICollaborative team culture | Hybrid work environment | Professional developmentMid-level Full TimeOmaha, NE, 68102, USA R2d ago
-
Lead Full-Stack Engineer -- Advanced AI USD 132K-238KAPI Design | Agentic Systems | Application Architecture | CI/CD | Code review401k | Dental insurance | Employee discount | Health insurance | Life insuranceSenior-level Full Time7000 Target Pkwy N,NCD-0375 Brooklyn Park,MN … R2d ago
-
AI/ML Engineer USD 195K-264KAI RMF | AWS Bedrock | AWS SageMaker | Agentic AI | Azure Machine Learning401k matching | Career growth support | Paid time off | Remote work | Wellness packagesSenior-level Full TimeUSA LA Home Office (LAHOME), United … R2d ago
-
Data Platform Engineering, Student USD 165K-204KAzure Data | Azure Data Factory | Data Factory | Data Management | Data ModelingPart-time schedule | Remote workSenior-level Part Time(SUL) Remote - United States, United … R2d ago
-
AI Prompt Engineer USD 61K-122KAgile | Chain-of-Thought | Embeddings | Few-Shot Learning | Few-shotContinuing education | Financial benefits | Flexible time off | Healthcare benefits | Learning and developmentEntry-level Full Time405 ASHBURN VA (ASHBURN CACI/CLIENT REIMB … R2d ago
-
Mid-level Full TimeOhio WFH, United States R2d ago