Implement NLP use-cases using BERT

Explore the Implementation of NLP Tasks Using the Deep Learning Framework and Python (English Edition)

Amandeep

• Publisher: BPB Publications
• Year: 2021
• ISBN:
  9789390684625
• Topic: Informatik
• Subtopic: Natürliche Sprachverarbeitung

State-of-the-art BERT implementation for text classification

Description
This book provides a solid foundation for 'Natural Language Processing' with pragmatic explanation and implementation of a wide variety of industry wide scenarios. After reading this book, one can simply jump to solve real world problems and join the league of NLP developers.It starts with the introduction of Natural Language Processing and provides a good explanation of different practical situations which are currently implemented across the globe. Thereafter, it takes a deep dive into the text classification with different types of algorithms to implement the same. Then, it further introduces the second important NLP use case called Named Entity Recognition with its popular algorithm choices. Thereafter, it provides an introduction to a state of the art language model called BERT and its application.

What you will learn
? Learn to implement transfer learning on pre-trained BERT models.
? Learn to demonstrate a production ready Text Classification for domain specific data and networking using Python 3.x.
? Learn about the domain specific pre trained models with a library called `aiops` which has been specially designed for this book.

Who this book is for
This book is meant for Data Scientists and Machine Learning Engineers who are new to Natural Language Processing and want to quickly implement different NLP use-cases. Readers should have a basic knowledge of Python before reading the book.

Table of Contents
1. Introduction to NLP and Different Use-Cases
2. Deep Dive into Text Classification and Different Types of Algorithms in Industry
3. Named Entity Recognition
4. BERT and its Application
5. BERT: Text Classification
6. BERT: Text Classification Code

About the Authors
Amandeep has been working as a technical lead in the field of software development at the time of publishing this book. He has worked for almost eight years in a few of the top MNCs.

CHAPTER 1

Introduction to NLP and Different Use-Cases

Introduction

This chapter will introduce the Natural Language Processing concept in simple words that can easily be understood by any novice. Additionally, readers would get to know a brief introduction of 10+ NLP use-cases, which will build better exposure to the applicability of NLP, along with fundamental pre-processing steps.

Structure

• Understanding of NLP
• List of NLP use-cases
• Brief Introduction of each use-case
• Text pre-processing techniques
• Conclusion

Objective

To enable the reader for contributing to any discussion related to NLP Use-Cases by providing his views or understanding around the topic.

1.1 What is Natural Language Processing or NLP?

NLP is an acronym for multiple terms, but in this book, it will stand for Natural Language Processing. Natural language processing (NLP) is a subfield overlapping artificial intelligence, computer science, and linguistics that focuses on aiding computers to understand the natural language of humans.

It empowers machines to understand the text and the enormous amount of unstructured data, which is growing exponentially. Hence, we can say that NLP will play a pivotal role in next-generation computer systems.

The ultimate objective of NLP is to classify, extract, understand, translate, correct, paraphrase documents written in human languages and thus, utilizes the same to search for answers automatically for the questions or queries asked.

It has been published on various websites that Machine Learning Engineer and Data Scientists come under the top 10 most demanding jobs in the software industry that has the responsibility to inject intelligence in the applications. We have reached some level of saturation for learning from organized or structured data, and there is an ample amount of focus required to extract insights from unstructured data (approximately 80% of internet data is unstructured). It seems to me that text data should be at least 50% of unstructured data. Therefore, finding insights from the text is the need of the hour.

1.2 NLP use-cases

There are a plethora of NLP use-cases (a few of them are overlapping as well), and brief introductions of the same are provided below. We will take a deep dive into the first two of these in subsequent chapters.

• Text/Document/Sentence classification
  • Emotion classification or sentiment analysis
  • Subjectivity analysis
  • Sarcasm detection:
  • Intent classification
  • Hate speech detection
• Information extraction
  • Named entity recognition
• QA (questions and answers)
• Chat-bot
• Relation extraction
• Entity linking
• Text summarization
• Morphological analysis
• Semantic textual similarity
• Word sense disambiguation
• Spelling correction
• Grammatical error correction
• Language Modeling
• Slot filling
• Topic Modeling
• Paraphrase generation

1.3 Quick sneak on NLP use-cases

Let's take a quick overview of mentioned use-cases:

Text/Document/Sentence classification

It is a supervised machine learning problem where we classify text into one of the predefined categories such as whether the text is having polite language or offensive one.

Text or document may be interchanged, though it depends upon the individual. Additionally, these can have single or multiple sentences. So, some may use document classification for even sentence classification.

There are multiple applications of text classification, which are described below:

Emotion classification or sentiment analysis

Although some believe that sentiments [shown in square shapes] are after-effects of emotions [shown in oval shapes], we have considered it as one application.

Figure 1.1: Sentiments and Emotions

In this use-case, the task is to classify emails into any of 3 sentiments/emotions: positive, negative, or neutral.

Real-life use-cases:

• Proactive customer feedback:
  • The customer support team has the responsibility to keep NPS (Net Promoter Score), which is an evaluation metric for customer experience or satisfaction level.
  • Every organization wants to expand its business and feedback plays a pivotal role in doing so; however, it is quite difficult to enforce customers to fill the feedback form as it is a monotonous task.
  • In such a scenario, one can leverage the interactions, which happen with the customers, based on the daily grind.
  • In this, we try to extract the satisfaction level of different customersbased on different email communication happening with the clients.
  • Bonus Point: When...