AI with Python - Natural Language Processing

Natural Language Processing (NLP) refers to AI strategy of communicating with intelligent systems utilizing a natural language such as English.

Processing of Natural Language is required when you need an intelligent system like robot to proceed as per your instructions, when you want to hear decision from a dialogue based clinical expert system, etc.

The field of NLP involves making computers perform helpful tasks with the natural languages humans use. The input and output of an NLP system can be −

  • Speech
  • Written Text

Components of NLP

In this chapter, we will find out about the various components of NLP. There are two components of NLP. The components are described below −

Natural Language Understanding (NLU)

It includes the following tasks −

  • Planning the given input in natural language into useful representations.

  • Analyzing various aspects of the language.

Natural Language Generation (NLG)

It is the way of producing meaningful phrases and sentences in the form of natural language from some internal representation. It includes −

  • Text planning − This includes recovering the relevant content from the knowledge base.

  • Sentence planning − This includes picking the required words, forming meaningful phrases, setting tone of the sentence.

  • Text Realization − This is mapping sentence plan into sentence structure.

Difficulties in NLU

The NLU is very rich in form and structure; notwithstanding, it is ambiguous. There can be various levels of ambiguity −

Lexical ambiguity

It is at a very primitive level such as the word-level. For example, treating the word “board” as noun or verb?

Syntax level ambiguity

A sentence can be parsed in various ways. For example, “He lifted the beetle with red cap.” − Did he use cap to lift the beetle or he lifted a beetle that had red cap?

Referential ambiguity

Referring to something utilizing pronouns. For example, Seema went to Pooja. She said, “I am tired.” − Exactly who is tired?

NLP Terminology

Let us now observe a few important terms in the NLP terminology.

  • Phonology − It is investigation of organizing sound systematically.

  • Morphology − It is a study of construction of words from primitive meaningful units.

  • Morpheme − It is a primitive unit of meaning in a language.

  • Syntax − It refers to organizing words to make a sentence. It also includes determining the structural role of words in the sentence and in phrases.

  • Semantics − It is concerned with the meaning of words and how to join words into meaningful phrases and sentences.

  • Pragmatics − It manages using and understanding sentences in various situations and how the interpretation of the sentence is affected.

  • Discourse − It deals with how the immediately preceding sentence can affect the understanding of the next sentence.

  • World Knowledge − It includes the general knowledge about the world.

Steps in NLP

This section shows the various steps in NLP.

Lexical Analysis

It involves identifying and analyzing the structure of words. Lexicon of a language means the collection of words and phrases in a language. Lexical analysis is dividing the whole chunk of txt into sections, sentences, and words.

Syntactic Analysis (Parsing)

It involves analysis of words in the sentence for grammar and arranging words in a manner that shows the relationship among the words. The sentence such as “The school goes to boy” is rejected by English syntactic analyzer.

Semantic Analysis

It draws the exact meaning or the dictionary meaning from the content. The content is checked for meaningfulness. It is done by mapping syntactic structures and objects in the task domain. The semantic analyzer disregards sentence such as “hot ice-cream”.

Discourse Integration

The meaning of any sentence depends upon the meaning of the sentence just before it. In addition, it additionally brings about the meaning of immediately succeeding sentence.

Pragmatic Analysis

During this, what was said is re-interpreted on what it actually meant. It involves deriving those aspects of language which require real world information.

Input your Topic Name and press Enter.