Senior AI Data Infrastructure/Pipeline Engineer
Tasks
- Build ETL systems for data transformation and loading
- Build distributed data processing systems with low latency
- Deploy and operate services with Kubernetes and Docker
- Design and build data closed loop pipelines
- Develop data cleaning and annotation quality inspection toolchains
- Develop data management platform for data lake ingestion to model training
- Implement data version control and data lineage tracing
- Implement log event tracking and data synchronization
- Manage metadata and enable fast data retrieval
- Optimize data pipeline performance for collection cleaning conversion
- Perform offline and real-time data processing
- Troubleshoot large scale distributed system performance bottlenecks
Perks/Benefits
Skills/Tech-stack
Apache Iceberg | Apache Kafka | Apache Pulsar | Caching | Data Lake | Data Lineage | Data Warehouse | Data version control | Distributed Systems | Docker | ETL | Feature Engineering | Go | I/O | I/O Optimization | Java | Kubernetes | Lance | Memory Management | Metadata Management | MongoDB | MySQL | PostgreSQL | Python | RabbitMQ | Redis | Stream processing | Version control
Education
Bachelor of Engineering | Bachelor of Science | Master of Science | PhD
Roles
AI | AI Data Engineer | Data Engineer | Engineer | Senior Data Engineer
Regions
Countries
States
Cities
Related jobs
-
Software Engineer III, Generative AI USD 147K-211KComputer Vision | Data Processing | Debugging | Language Models | Language ProcessingSenior-level Full TimeKirkland, WA, USA4h ago
-
Senior Software Engineer - Database Engineering USD 200K-287KAutomated testing | Debugging | Distributed Systems | Distributed key-value stores | Failure recoverySenior-level Full TimeUS-CA-Menlo Park9h ago
-
Data Analyst - Forecasting and Optimization USD 124K-187KBacktesting | Deep learning | Feature Engineering | Gurobi | HiGHS401k matching | Disability insurance | Health insurance | Life insurance | Medical savings accountMid-level Full TimePhiladelphia, PA, United States10h ago
-
AWS Glue | AWS Lambda | AWS S3 | Access Control | Data GovernanceCareer growth opportunities | Collaborative and inclusive work environment | Diverse and inclusive culture | Flexible work arrangements | Permanent remote working modelSenior-level Full TimeCanada R11h ago
-
Data Modeling | Data analytics | Language Models | Large Language Models | Machine LearningCoaching | Hybrid work model | Mental health counseling | Mentorship | Paid volunteer timeMid-level Full TimeRaleigh, US, North Carolina11h ago
-
Applied AI Engineer USD 120K-158KA/B | A/B Testing | API Integration | Anthropic API | B testingCareer growth | Fully remote | Global Engineering Organization | High ownership culture | Learning and development budgetMid-level Full TimeUnited States R1d ago
-
Lead AI Engineer (AI Systems & Automation) USD 130K-260KAlerting | Anthropic API | Automation | Distributed Systems | DockerFully remote | Global Engineering Organization | High ownership culture | Learning and development budget | Modern engineering practicesSenior-level Full TimeUnited States R1d ago
-
Supervisor of AI Software Engineering USD 185K-195KAPI Development | Agile | Azure DevOps | CI/CD | CORS401k plan | Disability insurance | Health insurance | Life insurance | PTO programMid-level Full TimeLos Angeles, CA, United States1d ago
-
AI Engineer USD 200K-250KAWS | Automated testing | CI/CD | Deployment Pipelines | Embedding Models401k match | Frequent In Person Collaboration | Generous benefitsSenior-level Full TimeNew York1d ago
-
Senior AI Engineer USD 153K-259KAgent Frameworks | Embeddings | Evaluation | Graph Databases | Human-in-the-loop401k plan | Flexible vacation policy | Flexible work policy | Health and wellness benefits | Paid HolidaysSenior-level Full TimeRemote - US R1d ago
-
AI Research Scientist USD 240K-350KBenchmarking | Convolutional Neural Networks | Diffusion Models | Distributed Training | Federated Learning401k match | Continuing education support | Equity options | Flexible time off | Free parkingMid-level Full TimeAustin, TX1d ago
-
Member of the Technical Staff - Machine Learning USD 350K-400KBigQuery | Computer Vision | Explore Exploit Tradeoff | Explore/Exploit | GPU memorySenior-level Full TimeSan Francisco HQ1d ago
-
Senior-level Full TimeMountain View, CA1d ago
-
Mechatronic/Robotics Engineer or Systems Integration Engineer - Camera and Sensor Calibration USD 132K-200KAutomation systems | Bash | C plus plus | Calibration Techniques | Camera systemsOn-site workMid-level Full TimeMountain View, CA1d ago
-
DAX | Data Governance | Data Modeling | Data Visualization | Data Warehousing401k match | Dental insurance | Disability insurance | Life insurance | Medical insuranceSenior-level Full TimeCincinnati, Ohio, United States1d ago
-
Senior Quantum Embedded Engineer USD 142K-175K10G Ethernet | AMD Xilinx | Bash | C# | C++Hybrid work | Remote workSenior-level Full TimeNew Haven, CT1d ago
-
Senior Quantum Applications Engineer - QEC USD 119K-258KCUDA-Q | Decoder algorithms | Docker | End to End | End-to-End TestingSenior-level Full TimeNew Haven, CT1d ago
-
Quantum Engineer (Physicist) USD 136K-187KCircuit-QED | Cryogenics | Data Analysis | Error correction | Low temperature physicsMid-level Full TimeNew Haven, CT1d ago
-
Associate Quantum Engineer USD 130K-184KCryogenics | Data Analysis | Data acquisition | High-vacuum | Instrument ControlInterdisciplinary team environment | Mentorship | State-of-the-art facilitiesMid-level Full TimeNew Haven, CT1d ago
-
Staff AI engineer USD 170K-254KAI Evaluation | AWS | Agent Orchestration | Caching | Data PipelinesFlexible working hours | Hybrid work culture | Unlimited time offSenior-level Full TimeSan Francisco1d ago
-
Research Scientist - Distributed Machine Learning USD 180K-287KBF16 | CUDA | CUDA kernels | DeepSpeed | Distributed Training401k | Dental insurance | Disability insurance | Employee assistance program | Health insuranceMid-level Full TimeSunnyvale, CA1d ago
-
Machine Learning Infrastructure Engineer USD 216K-330KCUDA | DeepSpeed | Distributed Systems | Distributed Training | FSDPMid-level Full TimeSunnyvale, CA1d ago
-
AI Research Internship - LLM USD 84K-156KBias detection | Data Preparation | Data Processing | Dataset Management | Deep learningEntry-level InternshipSunnyvale, CA1d ago
-
Data Engineer USD 120K-175KAPIs | AWS | Apache Spark | Data Pipelines | Data Processing401k plan | Dental insurance | Disability insurance | Employee assistance program | HolidaysMid-level Full TimeSunnyvale, CA1d ago
-
Forward Deployed AI Engineer, Operations USD 112K-300KAnalytics | C++ | Data Processing | Data Processing Pipelines | JavaDental insurance | Equity compensation | Medical insurance | Paid time off | Travel opportunitiesSenior-level Full TimeSouth San Francisco, California, USA1d ago