Senior AI Data Infrastructure/Pipeline Engineer
Tasks
- Build ETL systems for data transformation and loading
- Build distributed data processing systems with low latency
- Deploy and operate services with Kubernetes and Docker
- Design and build data closed loop pipelines
- Develop data cleaning and annotation quality inspection toolchains
- Develop data management platform for data lake ingestion to model training
- Implement data version control and data lineage tracing
- Implement log event tracking and data synchronization
- Manage metadata and enable fast data retrieval
- Optimize data pipeline performance for collection cleaning conversion
- Perform offline and real-time data processing
- Troubleshoot large scale distributed system performance bottlenecks
Perks/Benefits
Skills/Tech-stack
Apache Iceberg | Apache Kafka | Apache Pulsar | Caching | Data Lake | Data Lineage | Data Warehouse | Data version control | Distributed Systems | Docker | ETL | Feature Engineering | Go | I/O | I/O Optimization | Java | Kubernetes | Lance | Memory Management | Metadata Management | MongoDB | MySQL | PostgreSQL | Python | RabbitMQ | Redis | Stream processing | Version control
Education
Bachelor of Engineering | Bachelor of Science | Master of Science | PhD
Roles
AI | AI Data Engineer | Data Engineer | Engineer | Senior Data Engineer
Regions
Countries
States
Cities
Related jobs
-
Forward Deployed AI Engineer, Aviation Regulatory USD 112K-300KAirspace Tracking | Analytics | C++ | Data Pipelines | Data ProcessingSenior-level Full TimeSouth San Francisco, California, USA3h ago
-
Forward Deployed AI Engineer, Expansion USD 112K-300KAnalytics | C++ | Data Pipelines | Decision Support Systems | Decision supportDental insurance | Equity compensation | Medical insurance | Overtime pay | Paid time offSenior-level Full TimeSouth San Francisco, California, USA3h ago
-
Forward Deployed AI Engineer, Community Engagement USD 112K-300KC++ | Data Pipelines | Evaluation | Issue Tracking | JavaSenior-level Full TimeSouth San Francisco, California, USA3h ago
-
Analytics | C++ | Data Pipelines | Document Intelligence | EvaluationDental insurance | Health insurance | Paid time off | Travel up to 25% | Vision insuranceSenior-level Full TimeSouth San Francisco, California, USA3h ago
-
Analytics | C++ | Data Pipelines | Evaluation | JavaDental insurance | Health insurance | Paid time off | Travel opportunities | Vision insuranceSenior-level Full TimeSouth San Francisco, California, USA3h ago
-
Forward Deployed AI Engineer, Legal Real Estate USD 112K-300KAnalytics | Approval Routing | C++ | Contract automation | Data PipelinesDental insurance | Equity compensation | Medical insurance | Paid time off | Performance bonusesSenior-level Full TimeSouth San Francisco, California, USA3h ago
-
Forward Deployed AI Engineer, Operations USD 112K-300KAnalytics | C++ | Data Processing | Data Processing Pipelines | GenAIDental insurance | Equity compensation | Medical insurance | Overtime pay | Paid time offSenior-level Full TimeSouth San Francisco, California, USA3h ago
-
Forward Deployed AI Engineer, People USD 112K-300KAnalytics | C++ | Data Pipelines | Data Processing | GenAIDental insurance | Medical insurance | Paid time off | Travel opportunities | Vision insuranceSenior-level Full TimeSouth San Francisco, California, USA3h ago
-
Analytics | C++ | Data Pipelines | Data Processing | GeospatialDental insurance | Medical insurance | Paid time off | Travel opportunity | Vision insuranceSenior-level Full TimeSouth San Francisco, California, USA3h ago
-
Forward Deployed AI Engineer, Site Acquisition USD 112K-300KAnalytics | C++ | CRM | Data Processing | Data Processing PipelinesDental insurance | Medical insurance | Paid time off | Travel up to 25 percent | Vision insuranceSenior-level Full TimeSouth San Francisco, California, USA3h ago
-
Forward Deployed AI Engineer, Talent USD 112K-300KAnalytics | C++ | Data Processing | Data Processing Pipelines | Deep learningSenior-level Full TimeSouth San Francisco, California, USA3h ago
-
Principal Machine Learning Engineer USD 205K-230KAWS Lambda | BigQuery | C# | CI/CD | Cloud Functions401k | Dental insurance | Health insurance | Life insurance | Paid HolidaysSenior-level Full TimeUnited States of America - Remote … R4h ago
-
Senior Software Engineer - Internal Observability USD 200K-287KAWS | Anomaly Detection | Azure | C++ | Cause analysisEntry-level Full TimeUS-CA-Menlo Park5h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Flexible remote work | Laid-back atmosphereMid-level Full TimeAtlanta, GA, USA5h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Friendly relaxed atmosphere | Opportunity to work on impactful productMid-level Full TimeAustin, TX, USA5h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeMid-level Full TimeNew York, NY, USA5h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerMid-level Full TimeBoston, MA, USA5h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Career growth | Laid-back atmosphere | Remote-friendlyMid-level Full TimePortland, OR, USA5h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Bonus opportunities | Equity opportunities | Friendly work environment | Remote-friendlyMid-level Full TimeMinneapolis, MN, USA5h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Bonus | Equity | Flexible remote setting | Friendly work environmentMid-level Full TimeTempe, AZ, USA5h ago
-
Bash | Cloud infrastructure | Data Processing | Docker | GCPAsynchronous culture | Competitive compensation not applicable | Laid-back atmosphere | Remote-friendlyMid-level Full TimeFrisco, TX, USA5h ago
-
Bash | Cloud Computing | Cloud platform | Data Processing | DockerAsynchronous culture | Friendly and laid-back atmosphere | Opportunity for career growth | Remote distributed settingMid-level Full TimeLas Vegas, NV, USA5h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous work culture | Flexible management approachMid-level Full TimeDetroit, MI, USA5h ago
-
Bash | Data Processing | Docker | GCP | Large Scale DataAsynchronous culture | Flexible management approach | Remote/distributed workMid-level Full TimeRaleigh, NC, USA5h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Flexible management approach | Remote/distributed teamMid-level Full TimeKansas City, MO, USA5h ago