NLU:

« Natural Language Understanding or NLU is a subset of Natural Language Processing in Al which specifically deals with machine reading comprehension.

« NLU helps in facilitating direct human-machine interaction by allowing human languages to be understood by the computer without the use of
conditional statements.

NLP Vs NLU:

« In the image we can see the applications of NLP and NLU is a subset of NLP which deals with the understanding and comprehension part of NLP
« NLU interprets the meaning which the user communicates and classifies its proper intents.
« NLP deals with engaging in communication using human (natural language)

NLU Techniques:

« Summarization: Text Summarization refers to the process of using Deep Learning and Machine Learning models to synthesize large bodies of texts into their most important parts.

« Relation Extraction: Relationship extraction is the task of extracting semantic relationships from a text. Extracted relationships usually occur between two or more entities of a certain type (e.g. Person, Organisation, Location) and fall into a number of semantic categories (e.g. married to, employed by, lives in).

« Semantic parsing: Semantic parsing, the study of translating natural language utterances into machine-executable programs, is a well-established research area and has applications in question answering, instruction following, voice assistants, and code generation.

« Paraphrase: Paraphrasing or paraphrase in computational linguistics is the natural language understanding task of detecting and generating paraphrases. Applications of paraphrasing are varied including information retrieval, question answering, text summarization, and plagiarism detection.

« NLI: Natural Language Inference (NLI) is the task of determining whether the given “hypothesis” logically follows from the “premise”. In layman’s terms, you need to understand whether the hypothesis is true, while the premise is your only knowledge about the subject.

Problems faced by NLU systems:

« Ambiguities on the subject: textual data may not always be meaningful, which makes it difficult to derive the correct meaning or understanding or even get the correct context.

Example: “Old men and women were taken to safe locations”
In the above example, scope of the adjective is ambiguous, ie. whether it means ((old men) and women) or (old men and women) |

« Incomplete data: incomplete data in NLP refers to text with missing or inappropriate words which can affect the performance of a language model

Example: “l wanna travel”, “Pls send me the report”

In the above example, ‘want to’ is written as ‘wanna, ‘please’ is written as ‘Pls’

« Irony and sarcasm: Irony and sarcasm present problems for machine learning models because they generally use words and phrases that, strictly by definition, may be positive or negative, but actually connote the opposite.

« Domain-specific language: Different businesses and industries often use very different language. An NLP processing model needed for different domains may be different.

Example: The NLP model for healthcare domain would be very different than one used to process legal documents. which results in building or training their own models based on their domain.

NLU service providers:

1. IBM Watson Conversation
2. SNIPS API
3. Microsoft LUIS
4. Amazon Lex
5. API.ai
6. Recast.ai API
7. wit.ai API

Evolution over a time period has provided insight into these services. Facebook, Google, Microsoft, IBM and Amazon are important players in this field.

Next… Working with Text data -part4