Nathan C Silberman, 42Needham, MA

2021022, Jul 22, 2021

Apr 5, 2021

17/222782

- Mountain View CA, US
Nathan Silberman - Brooklyn NY, US
Dilip Krishnan - Arlington MA, US
George Trigeorgis - Nicosia, CY
Dumitru Erhan - San Francisco CA, US

G06K 9/46
G06K 9/62
G06N 3/04
G06N 3/08
G06T 7/73

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing images using an image processing neural network system. One of the system includes a shared encoder neural network implemented by one or more computers, wherein the shared encoder neural network is configured to: receive an input image from a target domain; and process the input image to generate a shared feature representation of features of the input image that are shared between images from the target domain and images from a source domain different from the target domain; and a classifier neural network implemented by the one or more computers, wherein the classifier neural network is configured to: receive the shared feature representation; and process the shared feature representation to generate a network output for the input image that characterizes the input image.

2021014, May 13, 2021

Dec 23, 2020

17/132901

Gabriel Fine - Salt Lake City UT, US
Nathan Silberman - Brooklyn NY, US

G06T 7/70
G06T 7/00
G06K 9/66
A61B 34/10
A61B 34/20
G16H 30/20
G06K 9/62
G06T 7/73
G16H 50/50
G06N 3/04
G16H 40/63
G16H 30/40
G06N 3/08

A system and method is disclosed for guiding an invasive medical device with radiation absorbing markers. The invasive medical device can include markers with different radiation absorbing properties relative to other portions of the invasive medical device. An imaging device can generate images of the invasive medical device within a patient. A trained model for the invasive medical device can be trained on annotated images of the invasive medical device annotated with marker information identifying the markers and spatial information for the invasive medical device. An imaging computer system can apply the trained model to images of the invasive medical device within the patient including depictions of the markers to determine current spatial information of the invasive medical device inside the patient. The images of the invasive medical device and visual spatial information representing the spatial information of the invasive medical device can be outputted to a display.

2021013, May 6, 2021

Dec 22, 2020

17/131066

Gabriel Fine - Salt Lake City UT, US
Nathan Silberman - Brooklyn NY, US

G06T 7/70
G06T 7/00
G06K 9/66
A61B 34/10
A61B 34/20
G16H 30/20
G06K 9/62
G06T 7/73
G16H 50/50
G06N 3/04
G16H 40/63
G16H 30/40
G06N 3/08

A system and method for augmenting unlabeled imaging data of invasive medical devices is disclosed. An imaging device can generate images of the invasive medical device inside a patient. A trained model for the invasive medical device can be trained on annotated images of the invasive medical device having labels indicating orientation and position information. An imaging computer system can apply the trained model to unlabeled images of the invasive medical device within the patient to determine a current orientation and a current position of the invasive medical device inside the patient. The images of the invasive medical device, visual orientation information representing the current orientation of the invasive medical device, and visual position information representing the current position of the invasive medical device inside the patient can be outputted to a display.

2021013, May 6, 2021

Dec 22, 2020

17/131409

Gabriel Fine - Salt Lake City UT, US
Nathan Silberman - Brooklyn NY, US

G06T 7/70
G06T 7/00
G06K 9/66
A61B 34/10
A61B 34/20
G16H 30/20
G06K 9/62
G06T 7/73
G16H 50/50
G06N 3/04
G16H 40/63
G16H 30/40
G06N 3/08

A system and method is disclosed for augmenting image data of an invasive medical device having non-medical structures. An invasive medical device having markers with distinct physical shapes relative to other portions of the invasive medical device can be inserted in a patient. An imaging device can generate images of the invasive medical device within the patient. A trained model for the invasive medical device can be trained on annotated images of the invasive medical device annotated with marker and spatial information. An imaging computer system can apply the trained model to images of the invasive medical device having depictions of the markers to determine current spatial information. The depictions of the markers can be identified and correlated with the spatial information from the annotated images. The images and visual spatial information representing the spatial information of the invasive medical device can be outputted to a display.

2021011, Apr 15, 2021

Dec 22, 2020

17/130938

Gabriel Fine - Salt Lake City UT, US
Nathan Silberman - Brooklyn NY, US

G06T 7/70
G06T 7/00
G06K 9/66
A61B 34/10
A61B 34/20
G16H 30/20
G06K 9/62
G06T 7/73
G16H 50/50
G06N 3/04
G16H 40/63
G16H 30/40
G06N 3/08

A system and method is disclosed for augmenting image data of an invasive medical device using optical imaging. An optical imaging sensor, separate from the invasive medical device, can generate images of the medical device within a patient. A trained model for the invasive medical device can be trained on annotated images of the invasive medical device with orientation and distance information of the invasive medical device. An imaging computer system can apply the trained model to images of the invasive medical device within the patient to determine a current orientation and a current distance of the invasive medical device. The images of the invasive medical device as captured by the optical imaging sensor, visual orientation information representing the current orientation of the invasive medical device, and visual distance information representing the current distance of the invasive medical device within the patient can be displayed.

2021011, Apr 15, 2021

Dec 22, 2020

17/130949

Gabriel Fine - Salt Lake City UT, US
Nathan Silberman - Brooklyn NY, US

G06T 7/70
G06T 7/00
G06K 9/66
A61B 34/10
A61B 34/20
G16H 30/20
G06K 9/62
G06T 7/73
G16H 50/50
G06N 3/04
G16H 40/63
G16H 30/40
G06N 3/08

A system and method is disclosed for automatically determining position information for an invasive medical device based on imaging data. An imaging device can generate 2D images of the invasive medical device within the patient from a vantage point relative to the patient. A trained model for the invasive medical device can be trained on annotated 2D images of the invasive medical device with position information. An imaging computer system can apply the trained model to unannotated 2D images of the invasive medical device within the patient to determine a current position of the invasive medical device. The 2D images of the invasive medical device and visual position information representing the current position of the invasive medical device can be outputted to a display.

2021011, Apr 15, 2021

Dec 22, 2020

17/131341

Gabriel Fine - Salt Lake City UT, US
Nathan Silberman - Brooklyn NY, US

G06T 7/70
G06T 7/00
G06K 9/66
A61B 34/10
A61B 34/20
G16H 30/20
G06K 9/62
G06T 7/73
G16H 50/50
G06N 3/04
G16H 40/63
G16H 30/40
G06N 3/08

A system and method is disclosed for displaying augmented image data for invasive medical devices. A current orientation and a current position of the invasive medical device within a patient can be determined by applying a trained model of the invasive medical device to unannotated images of the invasive medical device as captured by an imaging device. The images of the invasive medical device can be displayed and overlaid with the current orientation and current position of the invasive medical device. User input can be received to initialize tracking of an orientation and a position of the invasive medical device as the invasive medical device is moved within the patient.

2021011, Apr 15, 2021

Dec 22, 2020

17/131390

Gabriel Fine - Salt Lake City UT, US
Nathan Silberman - Brooklyn NY, US

G06T 7/70
G06T 7/00
G06K 9/66
A61B 34/10
A61B 34/20
G16H 30/20
G06K 9/62
G06T 7/73
G16H 50/50
G06N 3/04
G16H 40/63
G16H 30/40
G06N 3/08

A system and method for guiding invasive medical devices relative to other medical devices is disclosed. An imaging device can generate images of the invasive medical device within a body. A trained model for the invasive medical device can be trained on annotated images of the invasive medical device with at least one of orientation and position information. An imaging computer system can apply the trained model to unannotated images of the invasive medical device to determine at least one of a current orientation and a current position of the invasive medical device relative to another medical device within the body. At least one of visual orientation information representing the current orientation of the invasive medical device relative to the other medical device and visual position information representing the current position of the invasive medical device relative to the other medical device within the body can be outputted.