How syntactic and semantic analyses are performed in natural language processing?

Syntactic Analysis

The syntactic analysis involves analyzing the grammatical syntax of a sentence to understand its meaning. 

For example, consider the following sentence: “The cow jumped over the moon“

Using Syntactic analysis, a computer would be able to understand the parts of speech of the different words in the sentence. Based on the understanding, it can then try and estimate the meaning of the sentence. In the case of the above example (however ridiculous it might be in real life), there is no conflict about the interpretation. Thus, the syntactic analysis does the job just fine. 

However, human language is nuanced and not always, is a sentence as simple as the one described above. Consider this: “Does this all sound like a joke to you?“

A human would easily understand the irateness locked in the sentence. However, a syntactic analysis may just be too naive for it. That leads us to the need for something better and more sophisticated, i.e., Semantic Analysis.

Syntactic analysis is much easier to implement than semantic analysis. The typical process includes:

Tokenization: Breaking a text into sentence tokens & then a sentence into word tokens

Lemmatization: Identifying the lemma of a word. Example: Cri will be the lemma for all cry, crying, cried.

POS Tagging: Identifying the part of speech (POS) of a particular word in a sentence.

Semantic Analysis

In the case of semantic analysis, a computer understands the meaning of a text by analyzing the text as a whole and not just looking at individual words. The context in which a word is used is very important when it comes to semantic analysis. Let’s revisit the same example: “Does it all sound like a joke to you?” While the word “joke” may be positive on its own, something sounding like a joke may not be so. Thus, the context here is derived by analyzing the whole sentence instead of isolated words.

The semantic analysis does throw better results, but it also requires substantially more training and computation. We will now understand why. 

A company receives an online review from a customer.  “Contrary to my request, the hairstylist dyed my hair orange! I thought it was a joke, but no. I’ll never go back!”

The company’s semantic analysis tool will proceed to the analysis of these statements. Here, artificial intelligence must understand the meaning of the words used. For example, “orange” should be analyzed as a homonym and a polysemantic word (i.e. one with multiple meanings). The machine must recognize that the single word corresponds to a color and not a fruit. The use of the word “joke” (with a generally positive connotation) must also be identifiable in a negative context. Finally, the tool must be able to identify the customer’s deep satisfaction that gives meaning to the message. These analyses allow for the classification of customer requests into different categories, by theme, feeling, intentions, or risks. For example, faced with a negative review like the above, the message could be categorized as one of “dissatisfaction,” with a high level of risk.