Lead Robotics Foundation Model Engineer

Position Summary
As the Lead Robotics Foundation Model Engineer, you will design, train, and deploy large-scale multimodal models that integrate vision, language, and action components for real-world robotic applications. Leveraging data from our teleoperation systems, you will create generalizable policies for our robots to perform complex tasks autonomously and reliably—beyond lab-scale or proof-of-concept demos. You will guide the end-to-end pipeline, from data processing and model design to on-robot deployment and performance optimization.

Key Responsibilities

• 
  
Model Architecture & Implementation

• Design Vision-Language-Action Models
• Develop and refine network architectures (transformers, multimodal encoders) that integrate vision data, language instructions, and robot control signals to output intelligent action policies.


• Scalable Training Pipelines
• Set up robust machine learning pipelines (distributed training, large-batch processing) to handle extensive teleoperation datasets.


• Real-Time Control Integration
• Work closely with our robotics control team to ensure model outputs align with real-time actuation requirements, bridging deep learning inference with embedded controllers.

Teleoperation & Data Utilization

• Data Collection & Curation
• Collaborate with the teleoperation software team to design data-collection strategies, ensuring we capture high-quality vision and operator-action sequences for model training.
• Multimodal Annotation & PreprocessingImplement processes for labeling or inferring language-based instructions, sensor metadata, and contextual cues from unstructured teleoperation logs.
• Domain Adaptation & Continuous LearningGuide methods to adapt VLA models as new teleoperation data is collected, ensuring models remain robust across varying tasks, operators, and environments.

Real-World Robot Deployment
• On-Robot Inference & OptimizationPackage and deploy trained policies onto embedded compute platforms (NVIDIA Jetson or similar), ensuring low-latency inference and reliable control signals.
• Performance Evaluation & Safety ChecksEstablish rigorous evaluation protocols (safety, accuracy, and autonomy metrics) to validate VLA models in real industrial or field environments, not just in simulation.
• Continuous Field OptimizationWork hand-in-hand with hardware teams and site operators to diagnose issues, refine model hyperparameters, and optimize inference for new or unexpected scenarios.

Collaboration & Stakeholder Management
• Cross-Functional CollaborationLiaise with internal and external robotics researchers, control engineers, and teleoperation specialists to align on objectives, share findings, and integrate best practices.
• External PartnershipsRepresent the VLA team in collaborations with external research institutes or technology partners, advocating for our approach to building robust production models.
• Continuous Optimization & Innovation
• Metrics & Model HealthDefine key performance indicators (accuracy, success rate, real-time efficiency) for model-driven robot autonomy and continuously track improvements.
• Research & Knowledge SharingStay up-to-date with advancements in multimodal deep learning, large-scale model optimization, and robotic control research; share breakthroughs internally.

Qualifications

• Technical Skills


• Deep Learning Expertise
Demonstrated track record building and training large-scale multimodal or transformer-based models (e.g., vision-language transformers, reinforcement learning pipelines).
• Robotics Integration
Experience deploying AI/ML solutions onto physical robots with real-time constraints; proficiency using robotics middleware (e.g., ROS1/2) and embedded edge hardware (e.g., Jetson).
• Data Engineering for ML
Proficiency in constructing data-processing pipelines (Python, C++, or similar. Training using high-performance GPU) for large, complex datasets (images, video, text, sensor logs).
• Distributed Systems
Familiarity with distributed training paradigms (PyTorch Distributed or similar) for large-scale model development.
• Control & Actuation
Solid understanding of control theory and how high-level AI actions map to low-level motors, actuators, and physical robot systems.