Research Scientist Intern, Robust Tracking via 3D Gaussian Splatting (PhD)

The Reality Labs (RL) Research Team brings together a world-class team of researchers, developers, and engineers to create the future of AR and VR. A key problem in processing input data of a human subject is finding dense correspondences (registration). Recently, a new class of methods has been opened up via 3D Gaussian Splatting: GitHub - JonathonLuiten/Dynamic3DGaussians. However, when applied in more challenging scenarios of longer-term correspondences, previous methods can easily get trapped in suboptimal solutions. Various heuristic methods have been utilized in the past, such as the usage of image pyramids or blurring, but these do not fully solve the problem and require ad-hoc tuning. The goal of this internship is to dive into this problem including its theoretical aspects, e.g., is there a useful class of quasi-convex loss functions that would yield more robust tracking?

Research Topics:
* Virtual humans, animatable avatar creation
* Dense motion tracking (as opposed to sparse feature/keypoint methods)
* Differentiable rendering via 3D Gaussian Splatting
* Machine learning / AI

Our internships are twelve (12) to twenty four (24) weeks long and we have various start dates throughout the year.Research Scientist Intern, Robust Tracking via 3D Gaussian Splatting (PhD) Responsibilities

• Design and execution of algorithms.
• Investigate, design and develop novel algorithms in areas such as body capture, tracking, reconstruction, generative models for clothed humans and deformable objects (soft tissue, clothes, hair).
• Prototyping, building and analysis of experimental systems.
• Collaboration with and support of other researchers across various disciplines.
• Communication of research agenda, progress and results.

Minimum Qualifications

• Currently has, or is in the process of obtaining a PhD in Computer Vision, Graphics, Machine Learning, or related field.
• Experience with computer vision, machine learning, 3D geometry and linear algebra.
• Experience or interest to learn 3D Gaussian Splatting.
• 2+ years of experience with Python and PyTorch.
• Must obtain work authorization in the country of employment at the time of hire, and maintain ongoing work authorization during employment.

Preferred Qualifications

• Intent to return to a degree-program after the completion of the internship.
• 2+ years of experience with one or more of the following areas: developing models for neural human reconstruction, human body learning, dynamic shape representation, numerical optimization, deep learning.
• Experience working and communicating cross functionally in a team environment.
• Proven track record of achieving results as demonstrated in accepted papers at top computer vision, graphics, machine learning or robotics related journals and conferences such as CVPR, ICCV, ECCV, NeurIPS, SIGGRAPH, SIGGRAPH Asia etc.
• Experience with C++.
• Interpersonal skills: cross-group and cross-culture collaboration.

LocationsAbout Meta Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today—beyond the constraints of screens, the limits of distance, and even the rules of physics. Meta is committed to providing reasonable support (called accommodations) in our recruiting processes for candidates with disabilities, long term conditions, mental health conditions or sincerely held religious beliefs, or who are neurodivergent or require pregnancy-related support. If you need support, please reach out to accommodations-ext@fb.com.