AI for Analyzing Sentences (Premium)
-
Introduction
When an AI program (such as a chatbot or personal assistant on a smartphone) tries to understand a user’s request, it often needs to analyze every word in the sentence. For example, is the user referring to any person or object? Is it a question or a command? You can find the following block in the AI category (premium only) to analyze a sentence:
Parameters
- Sentence: the first input is the sentence you want to analyze, such as “What’s your name?”. It can be one or multiple sentences, but should not be too long (one or two sentences would be ideal).
- Result Table: when the analysis is completed, the result will be stored in the table you have selected here. You can create a new table, or use the default table created for you. Note that the existing data in that table will be removed first before the new analysis result is written into the table.
Understanding the Result
The result table should contain many rows and many columns. For example, when we analyze “How are you?”, this is the result we get:
Each row in the table corresponds to one part of the sentence, as separated by spaces or punctuation marks. For example, if the input sentence is “how are you?”, then there will be 4 rows in the result table.
The meaning of each column is explained below:
Text
The first column is “text”, which is simply the part of the text from the input sentence. It can be a word like “How”, or a punctuation mark like “?”.
Lemma
Some words have a few different forms. For example, “is”, “am” and “are” have the same meaning. The “lemma” is the standard form for each word, just like what you would find in a dictionary. For example, the lemma for “is”, “am” and “are” are all “be”; the lemma for “eats” and “ate” are both “eat”.
Type
The third column is the “type” of each word, which is also called “part of speech”. It tells us what function each part serves.
For example, a “noun” is the name of some person or object, such as “John” or “table”, A “pronoun” is a noun that refers to an object, such as “you”, “it”.
Here is the full list of types:
- Noun: A person or a thing, such as “dog”, “Mike”, “student”.
- Verb: an action, such as “eat”, “is”, “sing”.
- Pronoun: a word that refers to people indirectly, such as “I”, “he”, “they”.
- Adjective: a word that describes a noun, such as “beautiful”, “funny”, “nice”.
- Adverb: a word that describes a verb, such as “quickly”, “very”, “often”.
- Determiner: a word that limits the reference to a noun, such as “a”, “the”, “every”.
- Adposition: a word that expresses relations, such as “at”, “for”, “before”.
- Conjunction: a word that joins different parts of a sentence, such as “and”, “but”, “so”.
- Number: a number
- Particle: a helper word that combines with other words to express a new meaning, such as “up” in “eat up”, “away” in “go away”, “out” in “look out,”
- Affix: a short word joined with another word to change its form, such as “ed” in “kicked”, “s” in “dogs”, “er” in “shorter”.
Person
The fourth column is “person”, which tells us if this part is referring to a person. For example, “I” is the “first” person, and “you” is the “second” person.
Offset
This is the index of the part in the entire input. For example, in “How are you?”, the second part “are” starts at letter 5, so its offset is 5.
Dependency Label and Index (Advanced)
The last 2 columns describe the dependency structure of the sentence. The basic idea is that different words in a sentence depend on each other to express the meaning of the sentence.
Usually, the “root” of a sentence is the main “verb” in it. For example, in “How are you”, the verb is “are”, so it is the root of the dependency structure. In other words, the entire sentence is trying to express “something are something”. Then the other words such as “How” and “you” depend on the root “are” to further refine the meaning of the sentence.
For another example, in “Tom’s brother ate 4 burgers”, the root is the verb “ate”, and the words “brother” and “burgers” refine the meaning further, so we know “brother ate burgers”. On top of that, we further refine the word “brother” with “Tom”, and refine the word “burgers” with “4”.
In the result table, the “label” column tells us the role of each part in the dependency structure, and the “depends” column tells us the row number of the word that this word depends on.
For example, for “How are you”, the label of “are” is “root”, and it depends on row 2, which is itself. The label for “you” is “nominal subject”, since it is the subject of the verb “are”, and it depends on row 2, which is “are”.
Example - Sentence Summary
The result of analyzing a sentence can be used in many ways. For example, suppose we want to create a short summary of a long sentence. This often makes it easier to understand the sentence. A simple method is to extract all the nouns in the sentence and ignore the other parts.
To do that, we can use a for-loop to go through every row in the result table, and if a word is marked as a noun, we add this word to the summary:
When you run this program, you should get a summary of 2 words:
