Imagine you are waking up from the bed and having conversation to digital assistant such as Alexa, Siri, Goggle.
you : Alexa! How is my schedule looks like today?
Alexa : “ You have 2 meeting scheduled today. one in 10:00 AM with john doe and another at 3:00 PM with Jane doe.
We use this smart assistance to do many of our daily tasks, and we are so much relaying on them. We talk to these assistants not in a programming language but in our natural language. Natural language is the communication between humans since the beginning. Teaching computers to analyze and understand human language is called Natural Language Processing.
Example Applications using NLP
Below are some real world applications using natural language processing.
- Analyze social media feed to understand the customer voice, trending.
- Voice based assistants — Interact with user, understand user commands and perform task based on it.
- Email platform such as Gmail, Microsoft uses various NLP features on their product such as spam classification, priority index, calendar event, auto complete etc.,
- Machine Translate : Translating the text from one language to another. (Google translate, Microsoft translate)
- E-Commerce : Understanding user review, extracting relevant information form the product.
- Automatic summary/report generation on domain specific areas like Law, medical etc.,
- Spelling and grammar correction tools.
- Plagiarism detection
Example of NLP Tasks
- Language Modelling : Task of predicting the next word based on the previous & historical words. Language model heavily use in the areas such as speech recognition, OCR, Handwriting recognition, Machine translation, Spelling correction.
- Text Classification : Grouping text into known set of categories based on its content.
- Information Extraction : Extracting relevant information form scientific/medical Journal and any text data on the internet.
- Information Retrieval : Finding documents relevant to user query from large collection of data. most of the search engine today use this method to retrieve the results.
- Conversational Agent : Dialogue systems that can make conversation in human language. e.g. Alexa, Siri, OK Google Etc.,
- Text Summarization : Create short summaries form longer document while maintain the core content and meaning of the document.
- Question Answering : System that can automatically answer question given in natural human language.
- Machine Translation : Converting text from one language to another while preserve it’s meaning.
- Topic Modelling: Uncovering topical structure of a large collection of docs.
Before jump into NLP algorithm and technique we need to understand the core concept of language.
Language is a structured system of communication that involves complex combination of its components such as characters, words, sentences etc.
At the core human language having four major building blocks:
- Phonemes : Any of the abstract units of the phonetic system of a language that correspond to a set of similar speech sounds. Phonemes may not have any meaning by themselves but can induce meanings when in combine with other phonemes. (speech and sounds) Speech to text, speaker identification, text to speech are some NLP tasks heavily relay on Phonemes.
- Morphemes & Lexemes : A morpheme is a meaningful unit of language that cannot be further divided. It formed by combination of phonemes. lexeme is roughly, the set of inflected forms taken by a single word. Lexeme is a foundation block for many NLP tasks such as Tokenization, word embedding, POS Tagging.
- Syntax : Set of rules to construct grammatically correct sentences out of words and phrases in a language.
- Context : Context is how different parts in a language convey a particular meaning. The meaning of a sentence can change based on the context. Generally context composed of sematic and pragmatics. Sematic is the direct meaning with out external context. Pragmatics adds external context by referring world knowledge into it. some tasks heavily rely on the context are sarcasm detection, text summarization, topic modelling.
Ambiguity and creativity of human language are just two of the characteristics that make NLP more challenging.
Ambiguity of Language : Is the presence of two or more meanings in a single passage. Uncertainty of meaning. Most human languages are inherently ambiguous.
“ He is good as John Doe” Figurative languages increase the ambiguity.
Creativity : Since the beginning language evolved over time. language is not only rule based there is also creativity to it. Various style, dialects, variation are used in all language. A very good example of this creativity is “poem”.
One of the key challenges in NLP is how to encode all the things that are common knowledge to humans in a computational model.
Diversity across language : There is no direct mapping between the vocabularies of any two language(most cases). This hurdle make NLP solution harder. So either NLP models needs to be language agonistic or models needs to be developed for each and every language. both are bit unrealistic.
Below are some of the common approaches to solve NLP problems.
- Rule based NLP
- Machine Learning
- Deep Learning
- Reinforced Learning
Rule based NLP
This approach likely to building rules for the task at hand. this requires some expertise in the domain to formulate rule, this also requires resources like dictionaries, thesaurus. Word net have been built to aid rule based NLP. WordNet® is a large lexical database of English. Nouns, verbs, adjectives and adverbs are grouped into sets of cognitive synonyms (synsets), each expressing a distinct concept. It’s a database of a word and the semantic relation between them. it captures synonyms, hyponyms, meronyms.
- Synonyms : Word with similar meanings.
- Hyponyms : Capture type of relationships. (Football, cricket, tennis — Sports)
- Meronyms : Capture part of relationship. (hands, legs — meronyms of body)
As trend continuous machine learning is one method used by many researchers/organization to perform NLP Tasks.
In order to apply machine learning algorithm on given text corpora the features from the text needs to be extracted. there are several methods are there to extract feature, I will cover those method deeply in next article.
Once the features are extracted, those features are then used to representation of text to learn a model. Then this model will be evaluate and improve over time.
Some of common machine learning algorithm in MLP tasks are Naive bayes, SVM, Logistic regression, Hidden markov model, Conditional random field.
- Long short term memory (LSTM)
- Convolutional Neural networks (CNN)
- Auto encoders
- Recurrent Neural networks (RNN)
Problem with deep learning NLP
- Deep learning algorithm tend to overfit on small datasets.
- Few-shot learning and synthetic data generation. (learning from very few training examples.)
- Domain adaption: when we develop large model on some common domains and apply the same model one newer domain, the model yield poor performance. Example the model trained on product review will not work well when applied to domains such as law, medical.
- Interpretable models : It’s hard to interpret a DL models because , most of the time DL models work like a black box. Many of the business demands explanation on why the model comes to the specific results.
- Common sense and world knowledge : We achieved benchmark NLP tasks using ML and DL models, language remains a bigger puzzle to researchers and scientist. mainly it’s because human language and common sense enormously evolved over time. so interprets exact context of the meaning is big challenge.
- Cost : Build NLP tasks can be pretty expansive, because of enormous parameters and GPU.
- On-device deployment : NLP solutions needs to be deployed on an embed device rather than cloud. because the model needs to be work without relaying on internet.
Voice based conversational agents like Alexa, Siri, OK Google are some of existing application of NLP.
Speech recognition and synthesis
Speech recognition converting speech signals to their phonemes, then this transcribed as words. Synthesis is reverse process where text results converted in spoken language.
Natural Language understanding
once words received the text analyzed using different technique based on the use case. sentiment analysis, named entity recognition, coreference resolution.
Once find the useful information from user speech, we need to find the intent of the words.
Send response back to user using speech synthesis.
- Practical natural language processing : Sowmya vajjala, Bodhisattwa Majumeder, Anuj gupta & harshit surana.
- Lipton, Zachary C. and Jacob Steinhardt. “Troubling Trends in Machine Learning Scholarship”, (2018).