Lifeng Liu, 5554 Patricia Rd, Sudbury, MA 01776

8600192, Dec 3, 2013

Dec 8, 2010

12/962918

Lifeng Liu - Sudbury MA, US
Aaron S. Wallack - Natick MA, US
Cyril C. Marrion - Acton MA, US

Cognex Corporation - Natick MA

G06K 9/36
G06K 9/00

382285, 382154

This invention provides a system and method for determining correspondence between camera assemblies in a 3D vision system implementation having a plurality of cameras arranged at different orientations with respect to a scene, so as to acquire contemporaneous images of a runtime object and determine the pose of the object, and in which at least one of the camera assemblies includes a non-perspective lens. The searched 2D object features of the acquired non-perspective image, corresponding to trained object features in the non-perspective camera assembly, can be combined with the searched 2D object features in images of other camera assemblies (perspective or non-perspective), based on their trained object features to generate a set of 3D image features and thereby determine a 3D pose of the object. In this manner the speed and accuracy of the overall pose determination process is improved. The non-perspective lens can be a telecentric lens.

2010016, Jul 1, 2010

Dec 29, 2008

12/345130

Cyril C. Marrion - Acton MA, US
Nigel J. Foster - Natick MA, US
Lifeng Liu - Arlington MA, US
David Y. Li - West Roxbury MA, US
Guruprasad Shivaram - Chestnut Hill MA, US
Aaron S. Wallack - Natick MA, US
Xiangyun Ye - Framingham MA, US

COGNEX CORPORATION - Natick MA

G06K 9/00

382154

This invention provides a system and method for determining the three-dimensional alignment of a modeledobject or scene. After calibration, a 3D (stereo) sensor system views the object to derive a runtime 3D representation of the scene containing the object. Rectified images from each stereo head are preprocessed to enhance their edge features. A stereo matching process is then performed on at least two (a pair) of the rectified preprocessed images at a time by locating a predetermined feature on a first image and then locating the same feature in the other image. 3D points are computed for each pair of cameras to derive a 3D point cloud. The 3D point cloud is generated by transforming the 3D points of each camera pair into the world 3D space from the world calibration. The amount of 3D data from the point cloud is reduced by extracting higher-level geometric shapes (HLGS), such as line segments. Found HLGS from runtime are corresponded to HLGS on the model to produce candidate 3D poses. A coarse scoring process prunes the number of poses. The remaining candidate poses are then subjected to a further more-refined scoring process. These surviving candidate poses are then verified by, for example, fitting found 3D or 2D points of the candidate poses to a larger set of corresponding three-dimensional or two-dimensional model points, whereby the closest match is the best refined three-dimensional pose.

2011028, Nov 17, 2011

May 14, 2010

12/780119

Aaron S. Wallack - Natick MA, US
Lifeng Liu - Arlington MA, US
Xiangyun Ye - Framingham MA, US

G06T 7/00

382153, 901 14

A system and method for robustly calibrating a vision system and a robot is provided. The system and method enables a plurality of cameras to be calibrated into a robot base coordinate system to enable a machine vision/robot control system to accurately identify the location of objects of interest within robot base coordinates.

2012014, Jun 14, 2012

Dec 8, 2010

12/963007

Lifeng Liu - Sudbury MA, US
Aaron S. Wallack - Natick MA, US

COGNEX CORPORATION - Natick MA

H04N 13/02

348 50, 348E13074, 348E13001

This invention provides a system and method for training and performing runtime 3D pose determination of an object using a plurality of camera assemblies in a 3D vision system. The cameras are arranged at different orientations with respect to a scene, so as to acquire contemporaneous images of an object, both at training and runtime. Each of the camera assemblies includes a non-perspective lens that acquires a respective non-perspective image for use in the process. The searched object features in one of the acquired non-perspective image can be used to define the expected location of object features in the second (or subsequent) non-perspective images based upon an affine transform, which is computed based upon at least a subset of the intrinsics and extrinsics of each camera. The locations of features in the second, and subsequent, non-perspective images can be refined by searching within the expected location of those images. This approach can be used in training, to generate the training model, and in runtime operating on acquired images of runtime objects. The non-perspective cameras can employ telecentric lenses.

2023005, Feb 16, 2023

Sep 19, 2022

17/947937

- Shenzhen, CN
Lifeng Liu - Sudbury MA, US
Jian Li - Waltham MA, US

HUAWEI TECHNOLOGIES CO., LTD. - Shenzhen

G06N 20/20
G06K 9/62
G06N 5/02
G06F 11/34

This disclosure relates to recommendations made to users based on learned behavior patterns. User behavior data is collected and grouped according labels. The grouped user behavior data is labeled and used to train a machine learning model based on features and tasks associated with the classification. User behavior is then predicted by applying the trained machine learning model to the collected user behavior data, and a task is recommended to the user.

2020006, Feb 27, 2020

May 13, 2019

16/410672

- Natick MA, US
Lifeng Liu - Arlington MA, US
Tuotuo Li - Newton MA, US

G06T 7/80
B25J 9/16
G06T 7/73
G06T 7/33
G06K 9/46
H04N 5/247

This invention provides a system and method that ties the coordinate spaces at the two locations together during calibration time using features on a runtime workpiece instead of a calibration target. Three possible scenarios are contemplated: wherein the same workpiece features are imaged and identified at both locations; wherein the imaged features of the runtime workpiece differ at each location (with a CAD or measured workpiece rendition available); and wherein the first location containing a motion stage has been calibrated to the motion stage using hand-eye calibration and the second location is hand-eye calibrated to the same motion stage by transferring the runtime part back and forth between locations. Illustratively, the quality of the first two techniques can be improved by running multiple runtime workpieces each with a different pose, extracting and accumulating such features at each location; and then using the accumulated features to tie the two coordinate spaces.

2019008, Mar 21, 2019

Nov 15, 2018

16/192233

- NATICK MA, US
Lifeng Liu - Arlington MA, US
Xiangyun Ye - Framingham MA, US

B25J 9/16
G06T 7/80

A system and method for robustly calibrating a vision system and a robot is provided. The system and method enables a plurality of cameras to be calibrated into a robot base coordinate system to enable a machine vision/robot control system to accurately identify the location of objects of interest within robot base coordinates.

2019001, Jan 17, 2019

Jul 12, 2017

15/647748

- Plano TX, US
Lifeng Liu - Sudbury MA, US
Xiaotian Yin - Belmont MA, US
Jun Zhang - Cambridge MA, US
Jian Li - Austin TX, US

G06K 9/62
G06K 9/00
G05D 1/00
B60W 40/08
B60W 50/14

Methods, apparatus, and systems are provided for integrated driver expression recognition and vehicle interior environment classification to detect driver condition for safety. A method includes obtaining an image of a driver of a vehicle and an image of an interior environment of the vehicle. Using a machine learning method, the images are processed to classify a condition of the driver and of the interior environment of the vehicle. The machine learning method includes general convolutional neural network (CNN) and CNN with adaptive filters. The adaptive filters are determined based on influence of filters. The classification results are combined and compared with predetermined thresholds to determine if a decision can be made based on existing information. Additional information is requested by self-motivated learning if a decision cannot be made, and safety is determined based on the combined classification results. A warning is provided to the driver based on the safety determination.