AI Pipeline Engineer
Tasks
- Build high throughput data loading systems for GPU utilization
- Build ingestion systems for text image audio video and structured signals
- Collaborate with machine learning researchers to align data systems with model needs
- Construct evaluation dataset pipelines with integrity and contamination controls
- Design and operate large scale data pipelines for AI training and evaluation
- Design storage architectures across data tiers
- Develop dataset versioning lineage and provenance tracking
- Document data systems schemas and operational procedures
- Drive observability for data quality drift and pipeline health
- Implement data cleaning deduplication filtering and quality assurance
- Implement data privacy redaction and consent enforcement
- Implement labeling workflows active learning and human in the loop improvement
- Optimize cost and performance using compression format selection and caching
Perks/Benefits
Skills/Tech-stack
Active Learning | Apache Beam | Apache Spark | CI/CD | Code review | Data Lineage | Data Modeling | Data Privacy | Data Quality | Data loading | Data redaction | Dataset versioning | Distributed Systems | High Throughput | High Throughput Data Loading | High-throughput data | Human-in-the-loop | Java | Python | Ray | Scala | Storage Optimization | Testing | The Loop
Education
Roles
AI | AI Pipeline Engineer | Data Engineer | Engineer | Pipeline Engineer
Related jobs
-
Senior-level Full TimeTel Aviv-Jaffa, Tel Aviv District, IL R8h ago
-
API Integration | APIs | Agent systems | Apache Spark | AutomationInternal talks | Learning platform | Remote days | Sports classes | Wellness benefitsEntry-level Full TimeBrooklyn, United States R12h ago
-
Senior Data Analytics Engineer USD 145K-206KAWS | CRM | DBT | Data Model | Data Modeling401k match | Disability insurance | Employee assistance program | Flexible spending account | Health Savings Account contributionSenior-level Full TimeBRIC, United States R22h ago
-
C++ | Configuration Management | Data Analysis | Data pipeline | Data pipeline managementPaid time off | Relocation assistanceSenior-level Full TimeUS-AL-HUNTSVILLE-401 ~ 401 Jan Davis Dr … R22h ago
-
C plus plus | Configuration Control | Data Analysis | Hardware-in-the-loop | IntegrationRelocation assistanceSenior-level Full TimeUS-AL-HUNTSVILLE-401 ~ 401 Jan Davis Dr … R22h ago
-
C++ | Defense radar | Discrimination | Hardware-in-the-loop | LinuxRelocation assistanceMid-level Full TimeUS-AL-HUNTSVILLE-401 ~ 401 Jan Davis Dr … R22h ago
-
Sr. Data Engineer USD 114K-185KAgile | Amazon Web Services | Apache Spark | Azure | Cloud platform401k retirement savings plan | CVS store discount | Disability benefits | Education assistance | Employee stock purchase planSenior-level Full TimeIrving-750 West John Carpenter, United States R22h ago
-
Data Engineer USD 116K-206KAWS Lambda | Airflow | Amazon Web Services | Apache Airflow | DatabricksSenior-level Full TimePA - West Chester, 1354 Boot … R22h ago
-
Data Scientist II USD 95K-142KAirflow | CI/CD | Data Governance | Data Validation | Data Visualization401k | Employee stock purchase plan | Paid time off | Parental leave | Tuition assistanceSenior-level Full TimeRemote, United States R22h ago
-
Senior Solutions Architect, Financial Services Banking USD 184K-287KApache Spark | C++ | CUDA | Dask | Deep learningSenior-level Full TimeUS, NY, Remote R22h ago
-
Software Engineer, AI Engineering and Enablement USD 96K-117KAPI Gateway | AWS | Agile | Automated testing | CI/CD401k plan | Disability insurance | Employee Assistance Program (EAP) | Flexible working schedule | Global shutdownMid-level Full TimeSeattle 400 Dexter - WA, United … R22h ago
-
Agent systems | Enterprise Data | Fine Tuning | LLM Fine-tuning | Language ModelsRemote workSenior-level Full TimeAustin R1d ago
-
Product Data Architect - Evinova USD 65K-217KAuditability | CSV | Computerized System | Computerized System Validation | Data EngineeringHealth benefits | Paid Holidays | Paid leaves | Paid vacation | Qualified retirement planSenior-level Full TimeUS - Gaithersburg - MD, United … R1d ago
-
Principal Data Engineer (Python, AWS, Redshift, EMR, Airflow, Databricks, Big data) | Contract to Hire | Irvine, CA (Hybrid) | A USD 147K-170KAWS RDS | Airflow | Amazon EMR | Amazon Redshift | Amazon Web ServicesContract-to-hire | Hybrid work | Mentorship opportunitiesSenior-level Full TimeIrvine, CA R2d ago
-
Data Engineer USD 99K-146KAirflow | Amazon Redshift | Amazon Web Services | Apache Kafka | Atlassian JiraMid-level Full TimeDearborn, MI, United States R2d ago
-
Principal Machine Learning Engineer USD 172K-250KApache Arrow | Apache Spark | Bias Mitigation | DPO | Data-parallelSenior-level Full TimeUnited States R2d ago
-
Technical Lead, Machine Learning USD 172K-220KBias Evaluation | DPO | Data Pipelines | Evaluation | GPU OptimizationSenior-level Full TimeUnited States R2d ago
-
Staff Machine Learning Engineer USD 130K-200KBias Evaluation | DPO | Data Pipelines | GPU Optimization | Inference architectureSenior-level Full TimeUnited States R2d ago
-
Staff Machine Learning Engineer USD 165K-236KClass imbalance | Docker | Error Analysis | Feature Engineering | Feature Labeling401k | Bonus | Equity | Generous vacation time | Medical/Dental/Vision insuranceSenior-level Full TimeRemote USA R2d ago
-
Staff Data Engineer USD 135K-180KAI coding | AI coding tools | AWS EKS | Alerting | Amazon EC2401k match | Career development | Parental leaveSenior-level Full TimeTempe, Arizona R2d ago
-
AWS | Azure | CI/CD | Container Networking | Container networking interface401k company match | ESPP | Family leave programs | Flexible spending accounts | Flexible time away planSenior-level Full TimeSan Diego, CALIFORNIA, United States R2d ago
-
AWS | Azure | CI/CD | Container Networking | Containers401k match | ESPP | Family leave programs | Flexible spending accounts | Flexible time awaySenior-level Full TimeSanta Clara, California, United States R2d ago
-
Senior-level Full TimeDenver, CO - Hybrid; New York, … R2d ago
-
Staff+ Software Engineer, Safeguards Data USD 320K-485KAWS | Access Control | Cloud platform | Data Governance | Data LineageFlexible working hours | Generous vacation | Hybrid office policy | Optional equity donation matching | Parental leaveSenior-level Full TimeSan Francisco, CA | New York … R2d ago
-
Software Engineer, Applied AI USD 170K-300KAWS | Agents | Benchmarking | Datadog | ECSFree coachingSenior-level Full TimeNew York R3d ago