Senior Data Pipeline & AI Engineer
Tasks
- Define evaluation frameworks for LLM outputs
- Design data pipelines end to end
- Develop dbt models for data transformation
- Implement pipeline observability and fault tolerance
- Integrate LLM evaluation tools with Vertex AI and Langsmith
- Investigate and resolve data pipeline production incidents
- Lead system reliability improvements and operational excellence
- Maintain and extend Go microservices for data transformation
- Optimize pipeline throughput latency and reliability
- Orchestrate data migrations across customer destinations
- Write technical design documents for architecture changes
Perks/Benefits
- Annual company meet-up
- Annual learning subsidy
- Bereavement leave
- Flexible PTO
- Learning opportunities
- Monthly productivity stipend
- Paid parental leave
- Sponsored benefits packages
- Team offsites
Skills/Tech-stack
Batch Processing | BigQuery | DBT | Data Modeling | Data pipeline | Data pipeline orchestration | Databricks | Fault Tolerance | Go | Incident Management | Java | LLM | Langsmith | Pipeline Orchestration | Pipeline observability | Python | Snowflake | Streaming processing | System Architecture | Technical documentation | Vertex AI
Education
N/A
Related jobs
-
Forward Deployed AI Engineer, Aviation Regulatory USD 112K-300KAirspace Tracking | Analytics | C++ | Data Pipelines | Data ProcessingSenior-level Full TimeSouth San Francisco, California, USA4h ago
-
Forward Deployed AI Engineer, Expansion USD 112K-300KAnalytics | C++ | Data Pipelines | Decision Support Systems | Decision supportDental insurance | Equity compensation | Medical insurance | Overtime pay | Paid time offSenior-level Full TimeSouth San Francisco, California, USA4h ago
-
Forward Deployed AI Engineer, Community Engagement USD 112K-300KC++ | Data Pipelines | Evaluation | Issue Tracking | JavaSenior-level Full TimeSouth San Francisco, California, USA4h ago
-
Analytics | C++ | Data Pipelines | Document Intelligence | EvaluationDental insurance | Health insurance | Paid time off | Travel up to 25% | Vision insuranceSenior-level Full TimeSouth San Francisco, California, USA4h ago
-
Analytics | C++ | Data Pipelines | Evaluation | JavaDental insurance | Health insurance | Paid time off | Travel opportunities | Vision insuranceSenior-level Full TimeSouth San Francisco, California, USA4h ago
-
Forward Deployed AI Engineer, Legal Real Estate USD 112K-300KAnalytics | Approval Routing | C++ | Contract automation | Data PipelinesDental insurance | Equity compensation | Medical insurance | Paid time off | Performance bonusesSenior-level Full TimeSouth San Francisco, California, USA4h ago
-
Forward Deployed AI Engineer, Operations USD 112K-300KAnalytics | C++ | Data Processing | Data Processing Pipelines | GenAIDental insurance | Equity compensation | Medical insurance | Overtime pay | Paid time offSenior-level Full TimeSouth San Francisco, California, USA4h ago
-
Forward Deployed AI Engineer, People USD 112K-300KAnalytics | C++ | Data Pipelines | Data Processing | GenAIDental insurance | Medical insurance | Paid time off | Travel opportunities | Vision insuranceSenior-level Full TimeSouth San Francisco, California, USA4h ago
-
Analytics | C++ | Data Pipelines | Data Processing | GeospatialDental insurance | Medical insurance | Paid time off | Travel opportunity | Vision insuranceSenior-level Full TimeSouth San Francisco, California, USA4h ago
-
Forward Deployed AI Engineer, Site Acquisition USD 112K-300KAnalytics | C++ | CRM | Data Processing | Data Processing PipelinesDental insurance | Medical insurance | Paid time off | Travel up to 25 percent | Vision insuranceSenior-level Full TimeSouth San Francisco, California, USA4h ago
-
Forward Deployed AI Engineer, Talent USD 112K-300KAnalytics | C++ | Data Processing | Data Processing Pipelines | Deep learningSenior-level Full TimeSouth San Francisco, California, USA4h ago
-
Principal Machine Learning Engineer USD 205K-230KAWS Lambda | BigQuery | C# | CI/CD | Cloud Functions401k | Dental insurance | Health insurance | Life insurance | Paid HolidaysSenior-level Full TimeUnited States of America - Remote … R5h ago
-
Firmware Lead, Robotics USD 185K-268KARM Cortex | ARM Cortex-M | ARM Cortex-R | Asynchronous programming | Bring-upIn office 5 days per week | Relocation assistanceSenior-level Full TimeSan Francisco5h ago
-
Senior Software Engineer - Internal Observability USD 200K-287KAWS | Anomaly Detection | Azure | C++ | Cause analysisEntry-level Full TimeUS-CA-Menlo Park6h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Flexible remote work | Laid-back atmosphereMid-level Full TimeAtlanta, GA, USA6h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Friendly relaxed atmosphere | Opportunity to work on impactful productMid-level Full TimeAustin, TX, USA6h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeMid-level Full TimeNew York, NY, USA6h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerMid-level Full TimeBoston, MA, USA6h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Career growth | Laid-back atmosphere | Remote-friendlyMid-level Full TimePortland, OR, USA6h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Bonus opportunities | Equity opportunities | Friendly work environment | Remote-friendlyMid-level Full TimeMinneapolis, MN, USA6h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Bonus | Equity | Flexible remote setting | Friendly work environmentMid-level Full TimeTempe, AZ, USA6h ago
-
Bash | Cloud infrastructure | Data Processing | Docker | GCPAsynchronous culture | Competitive compensation not applicable | Laid-back atmosphere | Remote-friendlyMid-level Full TimeFrisco, TX, USA6h ago
-
Bash | Cloud Computing | Cloud platform | Data Processing | DockerAsynchronous culture | Friendly and laid-back atmosphere | Opportunity for career growth | Remote distributed settingMid-level Full TimeLas Vegas, NV, USA6h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous work culture | Flexible management approachMid-level Full TimeDetroit, MI, USA6h ago
-
Bash | Data Processing | Docker | GCP | Large Scale DataAsynchronous culture | Flexible management approach | Remote/distributed workMid-level Full TimeRaleigh, NC, USA6h ago