Senior AI Data Infrastructure/Pipeline Engineer
Tasks
- Build ETL systems for data transformation and loading
- Build distributed data processing systems with low latency
- Deploy and operate services with Kubernetes and Docker
- Design and build data closed loop pipelines
- Develop data cleaning and annotation quality inspection toolchains
- Develop data management platform for data lake ingestion to model training
- Implement data version control and data lineage tracing
- Implement log event tracking and data synchronization
- Manage metadata and enable fast data retrieval
- Optimize data pipeline performance for collection cleaning conversion
- Perform offline and real-time data processing
- Troubleshoot large scale distributed system performance bottlenecks
Perks/Benefits
Skills/Tech-stack
Apache Iceberg | Apache Kafka | Apache Pulsar | Caching | Data Lake | Data Lineage | Data Warehouse | Data version control | Distributed Systems | Docker | ETL | Feature Engineering | Go | I/O | I/O Optimization | Java | Kubernetes | Lance | Memory Management | Metadata Management | MongoDB | MySQL | PostgreSQL | Python | RabbitMQ | Redis | Stream processing | Version control
Education
Bachelor of Engineering | Bachelor of Science | Master of Science | PhD
Roles
AI | AI Data Engineer | Data Engineer | Engineer | Senior Data Engineer
Regions
Countries
States
Cities
Related jobs
-
Data and AI Solution Engineer USD 128K-214KAWS | AWS Bedrock | Agentic AI | Azure | Cloud ComputingSenior-level Full TimeUSA-VA-Herndon7h ago
-
Delivery Senior Consultant, Software Engineering Solutions, Identity & Gen AI Engineer USD 155K-265KAI Agents | AWS | Access Management | Ansible | AuthenticationHybrid work model | Onsite up to 5 days per week | Professional training and development | Travel opportunitiesSenior-level Full TimeAtlanta, Georgia, United States; Charlotte, North …7h ago
-
AI Agents | AI Risk Management Framework | Access Management | Amazon Web Services | AnsibleSenior-level Full TimeArlington/Rosslyn, Virginia, United States; Baltimore, Maryland, …7h ago
-
AWS Data Engineer - Consultant USD 93K-170KAWS Glue | AWS Lambda | Amazon S3 | Cloud Engineering | Data ArchitectureMentorship | Professional developmentMid-level Full TimeHouston, Texas, United States7h ago
-
Data Engineer, Product Analytics USD 185K-204KAI workflows | C# | C++ | Data Compliance | Data ModelingCareer growthMid-level Full TimeLos Angeles, CA | Burlingame, CA8h ago
-
Business Support Engineer USD 159K-223KAPI Development | AWS | Agent Orchestration | Azure | Bias Mitigation24 7 Oncall RotationMid-level Full TimeMenlo Park, CA8h ago
-
Business Support Engineer USD 147K-203KAPI troubleshooting | AWS | Azure | Data Analysis | Debugging24 7 Oncall RotationMid-level Full TimeMenlo Park, CA8h ago
-
Business Support Engineer USD 141K-197KAI Agents | API troubleshooting | AWS | Agent Orchestration | Azure24 7 Oncall Rotation | Cross-functional collaborationSenior-level Full TimeMenlo Park, CA8h ago
-
Software Engineer, Machine Learning RecSys USD 170K-240KAgent Orchestration | BERT | Bias Mitigation | C++ | Code ReviewsSenior-level Full TimeSunnyvale, CA8h ago
-
Senior Software Engineer, Sensor AI/ML, Watch Software USD 174K-253KC# | C++ | Data Processing | Debugging | Deep learningSenior-level Full TimeMountain View, CA, USA8h ago
-
Staff Software Engineer, Cooling Optimization USD 207K-301KC++ | Control Theory | Data Structures | Data center | Data center architectureSenior-level Full TimeSunnyvale, CA, USA8h ago
-
Staff Software Engineer, AI/ML, Search Ads USD 207K-301KC++ | Data Processing | Data Quality | Debugging | Distributed ComputingBonuses | Comprehensive health insurance | Equity | Paid time off | Retirement planSenior-level Full TimeMountain View, CA, USA8h ago
-
Senior Software Engineer, Database Internals, AlloyDB USD 174K-253KC# | C++ | Code optimization | Compute Technologies | Concurrency ControlEntry-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA8h ago
-
Software Engineer III, Generative AI, Search Health USD 147K-211KA/B | A/B Testing | B testing | Benchmarking | Computer VisionSenior-level Full TimeMountain View, CA, USA8h ago
-
Senior Developer Relations Engineer, AI Infrastructure USD 163K-237KC++ | Deep learning | Developer Content | GPU | Google KubernetesSenior-level Full TimeSunnyvale, CA, USA; Cambridge, MA, USA8h ago
-
Senior Data Engineer USD 90K-110KAgile | Amazon Web Services | Apache NiFi | Data Modeling | Data WarehousingAutonomy | Employee assistance program | Flexible working hours | Inclusive community | Online training videosSenior-level Full TimeNew York, United States R11h ago
-
Associate Director, AI Enablement & Machine Learning USD 212K-318KCloud Computing | Data Engineering | Data Governance | Deep learning | Generative AIAnnual incentive program | Healthcare coverage | Retirement benefitsMid-level Full TimeCambridge, MA13h ago
-
Software Engineer, Data (L1) USD 75K-130KAI Platform | APIs | Apache Airflow | Apache Iceberg | Apache Spark401k match | Calm subscription | Career growth and learning opportunities | Commuter benefits options | Dental insuranceMid-level Full TimeAustin, TX15h ago
-
Solutions Engineer – Robotics & Autonomous Driving USD 152K-500K2D Imaging | 3D Imaging | Autonomous Driving | C++ | CameraCommuter stipend | Generous PTO | Global travel | Health, dental, vision coverage | Learning and development stipendSenior-level Full TimeSan Francisco, CA; New York, NY16h ago
-
Senior AI Engineer I USD 123K-215KAWS | Agent Orchestration | Evaluation and monitoring | GCP | GRPCSenior-level Full TimePhoenix, AZ, United States18h ago
-
Senior Software Engineer, Storage USD 217K-303KC++ | Caching | Cassandra | Data Storage | Distributed Systems401k employer match | Dental insurance | Equity compensation | Generous time off | Medical insuranceSenior-level Full TimeRemote - United States R18h ago
-
Machine Learning Research Engineer USD 161K-189KAWS | Azure | Bias Variance | Bias-Variance Tradeoff | C++Flexible hybrid work model | Mental health counseling | Mentorship programs | Paid parental leave | Paid volunteer time offMid-level Full TimeNew York, US, New York19h ago
-
Big Data | Cloud Cost Optimization | Cost Optimization | Data Governance | Data Security401k plan | Dental insurance | Health insurance | Life insurance | Paid time offMid-level Full TimeSpringdale, AR Terminal Office - Springdale, …19h ago
-
Senior Data Engineer - Logistics USD 38K-38KBig Data | Cloud Cost Optimization | Column-Level Security | Cost Optimization | Data Governance401k plan | Dental insurance | Health insurance | Life insurance | Paid time offSenior-level Full TimeSpringdale, AR Terminal Office - Springdale, …19h ago
-
AI/ML Software Developer (req-264) USD 123K-149KAlgorithms | Data Structures | Java | JavaScript | Language Models401k plan | Community outreach initiatives | Commute benefits | Dental insurance | Federal HolidaysSenior-level Full TimeTysons, Virginia, United States19h ago