Gokhan Tur, 51285 Quinnhill Rd, Los Altos, CA 94024

7295981, Nov 13, 2007

Jan 9, 2004

10/755014

Narendra K. Gupta - Dayton NJ, US
Mazin G. Rahim - Warren NJ, US
Gokhan Tur - Morris Plains NJ, US
Antony Van der Mude - Hackettstown NJ, US

AT&T Corp. - New York NY

G10L 15/18

704257, 704 9, 704270

A method of generating a natural language model for use in a spoken dialog system is disclosed. The method comprises using sample utterances and creating a number of hand crafted rules for each call-type defined in a labeling guide. A first NLU model is generated and tested using the hand crafted rules and sample utterances. A second NLU model is built using the sample utterances as new training data and using the hand crafted rules. The second NLU model is tested for performance using a first batch of labeled data. A series of NLU models are built by adding a previous batch of labeled data to training data and using a new batch of labeling data as test data to generate the series of NLU models with training data that increases constantly. If not all the labeling data is received, the method comprises repeating the step of building a series of NLU models until all labeling data is received. After all the training data is received, at least once, the method comprises building a third NLU model using all the labeling data, wherein the third NLU model is used in generating the spoken dialog service.

7412383, Aug 12, 2008

Apr 4, 2003

10/407965

Tirso M. Alonso - New Providence NJ, US
Ilana Bromberg - Murray Hill NJ, US
Barbara B. Hollister - Mountainside NJ, US
Mazin G. Rahim - Warren NJ, US
Giuseppe Riccardi - Hoboken NJ, US
Lawrence Lyon Rose - Basking Ridge NJ, US
Daniel Leon Stern - Princeton NJ, US
Gokhan Tur - Morris Plains NJ, US
James M. Wilson - Berkeley Heights NJ, US

AT&T Corp - New York NY

G10L 15/06
G10L 15/22

704236, 704243, 704257

Systems and methods for annotating speech data. The present invention reduces the time required to annotate speech data by selecting utterances for annotation that will be of greatest benefit. A selection module uses speech models, including speech recognition models and spoken language understanding models, to identify utterances that should be annotated based on criteria such as confidence scores generated by the models. These utterances are placed in an annotation list along with a type of annotation to be performed for the utterances and an order in which the annotation should proceed. The utterances in the annotation list can be annotated for speech recognition purposes, spoken language understanding purposes, labeling purposes, etc. The selection module can also select utterances for annotation based on previously annotated speech data and deficiencies in the various models.

7562014, Jul 14, 2009

Sep 26, 2007

11/862008

Mazin G Rahim - Warren NJ, US
Giuseppe Riccardi - Hoboken NJ, US
Gokhan Tur - Parsippany NJ, US

AT&T Intellectual Property II, L.P. - New York NY

G06F 17/27
G10L 15/00

704236, 704231, 704 9

A large amount of human labor is required to transcribe and annotate a training corpus that is needed to create and update models for automatic speech recognition (ASR) and spoken language understanding (SLU). Active learning enables a reduction in the amount of transcribed and annotated data required to train ASR and SLU models. In one aspect of the present invention, an active learning ASR process and active learning SLU process are coupled, thereby enabling further efficiencies to be gained relative to a process that maintains an isolation of data in both the ASR and SLU domains.

7562017, Jul 14, 2009

Sep 27, 2007

11/862656

Mazin G. Rahim - Warren NJ, US
Gokhan Tur - Parsippany NJ, US

AT&T Intellectual Property II, L.P. - New York NY

G06F 17/21
G06F 17/27
G10L 15/08

704257, 704 9, 704 10

An active labeling process is provided that aims to minimize the number of utterances to be checked again by automatically selecting the ones that are likely to be erroneous or inconsistent with the previously labeled examples. In one embodiment, the errors and inconsistencies are identified based on the confidences obtained from a previously trained classifier model. In a second embodiment, the errors and inconsistencies are identified based on an unsupervised learning process. In both embodiments, the active labeling process is not dependent upon the particular classifier model.

7571098, Aug 4, 2009

May 29, 2003

10/448149

Allen Louis Gorin - Berkeley Heights NJ, US
Giuseppe Riccardi - Hoboken NJ, US
Gokhan Tur - Parsippany NJ, US
Jeremy Huntley Wright - Berkeley Heights NJ, US

AT&T Intellectual Property II, L.P. - New York NY

G10L 15/18
G10L 15/00
G10L 15/06
G10L 15/28
G06F 17/27

704257, 704 9, 704236, 704243, 704255

Word lattices that are generated by an automatic speech recognition system are used to generate a modified word lattice that is usable by a spoken language understanding module. In one embodiment, the spoken language understanding module determines a set of salient phrases by calculating an intersection of the modified word lattice, which is optionally preprocessed, and a finite state machine that includes a plurality of salient grammar fragments.

7620550, Nov 17, 2009

Oct 3, 2007

11/866685

Narendra K. Gupta - Dayton NJ, US
Mazin G. Rahim - Warren NJ, US
Gokhan Tur - Morris Plains NJ, US
Antony Van der Mude - Hackettstown NJ, US

AT&T Intellectual Property II, L.P. - New York NY

G10L 15/18

704257, 704 9, 704270, 704246, 704254, 709228

A method of generating a natural language model for use in a spoken dialog system is disclosed. The method comprises using sample utterances and creating a number of hand crafted rules for each call-type defined in a labeling guide. A first NLU model is generated and tested using the hand crafted rules and sample utterances. A second NLU model is built using the sample utterances as new training data and using the hand crafted rules. The second NLU model is tested for performance using a first batch of labeled data. A series of NLU models are built by adding a previous batch of labeled data to training data and using a new batch of labeling data as test data to generate the series of NLU models with training data that increases constantly. If not all the labeling data is received, the method comprises repeating the step of building a series of NLU models until all labeling data is received. After all the training data is received, at least once, the method comprises building a third NLU model using all the labeling data, wherein the third NLU model is used in generating the spoken dialog service.

7664644, Feb 16, 2010

Jun 9, 2006

11/423212

Gokhan Tur - Castro Valley CA, US

AT&T Intellectual Property II, L.P. - New York NY

G10L 15/18

704257

A system, method and computer-readable medium provide a multitask learning method for intent or call-type classification in a spoken language understanding system. Multitask learning aims at training tasks in parallel while using a shared representation. A computing device automatically re-uses the existing labeled data from various applications, which are similar but may have different call-types, intents or intent distributions to improve the performance. An automated intent mapping algorithm operates across applications. In one aspect, active learning is employed to selectively sample the data to be re-used.

7835910, Nov 16, 2010

May 29, 2003

10/448415

Gokhan Tur - Parsippany NJ, US

AT&T Intellectual Property II, L.P. - New York NY

G10L 15/00

704243, 704 10, 704251

A system and method for exploiting unlabeled utterances in the augmentation of a classifier model is disclosed. In one embodiment, a classifier is initially trained using a labeled set of utterances. Another set of utterances is then selected from an available set of unlabeled utterances. In one embodiment, this selection process can be based on a confidence score threshold. The trained classifier is then augmented using the selected set of unlabeled utterances.