AI Pipeline Engineer
Tasks
- Build evaluation dataset construction pipelines with integrity and contamination controls
- Build high throughput data loading systems to maximize GPU utilization
- Build ingestion systems for multimodal data
- Collaborate with ML researchers and engineers
- Design and operate large scale data pipelines for AI training and evaluation
- Design storage architectures balancing cost throughput and latency
- Develop dataset versioning lineage and provenance tracking
- Document data systems schemas and procedures
- Drive observability of data quality drift and pipeline health
- Implement data cleaning deduplication filtering and quality assurance
- Implement data privacy redaction and consent enforcement
- Implement labeling workflows active learning and human in the loop improvement
- Optimize cost and performance with compression format selection and caching
- Stay current with AI data infrastructure research and open source tools
Perks/Benefits
Skills/Tech-stack
Active Learning | Apache Beam | Apache Spark | CI/CD | Caching | Code review | Compression | Consent Management | Contamination control | Data Consent | Data Consent Management | Data Lineage | Data Modeling | Data Privacy | Data Quality | Data cleaning | Data labeling | Data loading | Data provenance | Data quality assurance | Data redaction | Dataset versioning | Deduplication | Distributed Systems | High Throughput | High Throughput Data Loading | High-throughput data | Human-in-the-loop | JVM | Observability | Python | Quality Assurance | Ray | Reproducibility | Schema | Storage Architecture | Testing | The Loop
Education
Roles
AI | AI Pipeline Engineer | Data Engineer | Engineer | Pipeline Engineer
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R15d ago
-
Senior DataOps Engineer USD 150K-180K800-171 | AWS CloudFormation | AWS Glue | AWS GovCloud | AWS IAMRemote position | US government clearance requiredSenior-level Full TimeRemote - United States R9h ago
-
Senior DataOps Engineer ( Remote US) USD 113K-163KAWS | AWS CloudFormation | Amazon EMR | Ansible | Apache Airflow401K employee contribution | Employee assistance program | Flexible Time Away From Work | Health & medical benefits | Learning platformSenior-level Full TimeRemote US R10h ago
-
Senior Software Engineer II - Ingestion AI Engineer USD 105K-155KAWS | Agile | Alerting | Azure | Causal InferenceMid-level Full TimeRemote, United States R11h ago
-
Lead Data Engineer USD 170K-198KAWS DMS | BI Semantic Layer | CDC | CI/CD | Contract EnforcementCareer development | Hybrid work environment | Performance reviewsSenior-level Full TimeNew York, NY R12h ago
-
Senior DevOps Engineer (Storage) - remote in the US USD 200K-250KAir-gapped | Alerting | Argo CD | ArgoCD | Bare MetalAttend conferences and working groups | Professional development and trainingExecutive-level Full TimeRemote, USA, United States R13h ago
-
AI/BI Analyst – Hybrid Role USD 74K-120KAWS | Amazon Bedrock | Data Governance | Data Modeling | DatabricksDental insurance | Educational assistance | Flexible spending account | Health savings account | Life insuranceMid-level Full TimeAtlanta, Georgia R17h ago
-
Explosives Research Experiment Lead USD 146K-222KAerospace Engineering | Bench testing | C++ | Computational modeling | Computer Aided Design401k | Education reimbursement | Flexible schedule | Hybrid work schedule | Relocation assistanceSenior-level Full TimeLivermore, CA, United States R1d ago
-
AI Engineer - ICAM USD 146K-197KAccess Management | Agile | Artificial Intelligence | Generative AI | Hugging Face100 percent remote workMid-level Full TimeUSA MD Home Office (MDHOME), United … R1d ago
-
APIs | AWS | CData | Change Data Capture | Cloud Native401k retirement plan | Adoption Assistance | Dental coverage | Education assistance | Employee Learning and DevelopmentSenior-level Full TimeNEO - Tamer Rashid Building OA, … R1d ago
-
AWS AI/ML Technical Practice Lead USD 155K-205KAWS | Agile | Amazon Bedrock | Amazon Rekognition | Amazon SageMakerSenior-level Full TimeUSA - Remote, United States R1d ago
-
Software Engineer, VP USD 169K-205KAPI Gateway | AWS Lambda | Agile | Amazon Bedrock | Amazon EC2Educational assistance | Health and wellness benefits | Paid Holidays | Paid maternity and parental bonding leave | Paid sick daysExecutive-level Full TimeNew Jersey Office - 210 Hudson … R1d ago
-
Vice President, Data Products & Analytics Engineer USD 184K-237KAWS | Agile | Alteryx | BI | CollibraExecutive-level Full TimeNew Jersey Office - 210 Hudson … R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Generative AI Specialist USD 100K-150KAgent systems | Agentic Workflows | Chunking | Deterministic Evaluation | EmbeddingsSenior-level Full TimeUnited States - Remote R1d ago
-
Large Language Model Specialist USD 100K-150KAdapter-Tuning | Attention Optimization | Benchmarking | DPO | DeepSpeed ZeRODirect W2 position | Remote workSenior-level Full TimeTempe, AZ R1d ago
-
Machine Learning Research Engineer USD 100K-150KAblation Studies | Accelerator hardware | Computer Vision | Data Quality | Data quality monitoringSenior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
ML Security Engineer USD 100K-150KAccess Controls | Access Management | Adversarial ML | Application Security | AuthorizationSenior-level Full TimeUnited States - Remote R1d ago
-
Hadoop Solutions Developer USD 100K-150KAWS EMR | Airflow | Apache Atlas | Apache Flink | Apache HBaseRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
ML Performance Engineer USD 100K-150KBenchmarking | C++ | Continuous batching | DeepSpeed | Distributed TrainingRemote workSenior-level Full TimeTempe, AZ R1d ago
-
Lead AI/ML Engineer - Remote USD 145K-249KAWS | Apache Spark | Azure | CI/CD | Cloud platform401k contribution | Career development opportunities | Comprehensive benefits package | Equity stock purchase | Incentive and recognition programsSenior-level Full TimeEden Prairie, Minnesota, UNITED STATES; REMOTE R1d ago
-
Associate AI/ML Engineer (RAG/LLM) USD 77K-137KAWS | Data Pipelines | LLM workflows | Language Models | Language Processing401k contribution | Career development opportunities | Comprehensive benefits package | Equity stock purchase | Hybrid workMid-level Full TimeBasking Ridge, New Jersey, UNITED STATES; … R1d ago
-
Portfolio Management Group Tech AI Engineer, Associate USD 125K-170KAWS SageMaker | Azure Machine Learning | Computer Vision | Deep learning | JavaComprehensive healthcare | Flexible time off | Leave benefits | Retirement benefits | Support for working parentsExecutive-level Full TimeAT8 - 725 Ponce de Leon … R1d ago