Rishi G Sharma, 46Fremont, CA

2021008, Mar 18, 2021

Sep 14, 2020

17/020565

- San Francisco CA, US
Alex Kuefler - London, GB
William D. Richards - San Francisco CA, US
Christopher Correa - San Francisco CA, US
Rishi Sharma - San Francisco CA, US
Sulabh Kumra - San Francisco CA, US

G06N 3/08
G05B 19/4155

A computer system trains a neural network to predict, for each pixel in an input image, the position that a robot's end effector would reach if a grasp (“poke”) were attempted at that position. Training data consists of images and end effector positions recorded while a robot attempts grasps in a pick-and-place environment. For an automated grasping policy, the approach is self-supervised, as end effector position labels may be recovered through forward kinematics, without human annotation. Although gathering such physical interaction data is expensive, it is necessary for training and routine operation of state of the art manipulation systems. Therefore, the system comes “for free” while collecting data for other tasks (e.g., grasping, pushing, placing). The system achieves significantly lower root mean squared error than traditional structured light sensors and other self-supervised deep learning methods on difficult, industry-scale jumbled bin datasets.