Natural Language Processing with TensorFlow
eBook - ePub

Natural Language Processing with TensorFlow

Teach language to machines using Python's deep learning library

Thushan Ganegedara

Share book
  1. 472 pages
  2. English
  3. ePUB (mobile friendly)
  4. Available on iOS & Android
eBook - ePub

Natural Language Processing with TensorFlow

Teach language to machines using Python's deep learning library

Thushan Ganegedara

Book details
Book preview
Table of contents
Citations

About This Book

Write modern natural language processing applications using deep learning algorithms and TensorFlowAbout This Book• Focuses on more efficient natural language processing using TensorFlow• Covers NLP as a field in its own right to improve understanding for choosing TensorFlow tools and other deep learning approaches• Provides choices for how to process and evaluate large unstructured text datasets• Learn to apply the TensorFlow toolbox to specific tasks in the most interesting field in artificial intelligenceWho This Book Is ForThis book is for Python developers with a strong interest in deep learning, who want to learn how to leverage TensorFlow to simplify NLP tasks. Fundamental Python skills are assumed, as well as some knowledge of machine learning and undergraduate-level calculus and linear algebra. No previous natural language processing experience required, although some background in NLP or computational linguistics will be helpful.What You Will Learn• Core concepts of NLP and various approaches to natural language processing• How to solve NLP tasks by applying TensorFlow functions to create neural networks• Strategies to process large amounts of data into word representations that can be used by deep learning applications• Techniques for performing sentence classification and language generation using CNNs and RNNs• About employing state-of-the art advanced RNNs, like long short-term memory, to solve complex text generation tasks• How to write automatic translation programs and implement an actual neural machine translator from scratch• The trends and innovations that are paving the future in NLPIn DetailNatural language processing (NLP) supplies the majority of data available to deep learning applications, while TensorFlow is the most important deep learning framework currently available. Natural Language Processing with TensorFlow brings TensorFlow and NLP together to give you invaluable tools to work with the immense volume of unstructured data in today's data streams, and apply these tools to specific NLP tasks.Thushan Ganegedara starts by giving you a grounding in NLP and TensorFlow basics. You'll then learn how to use Word2vec, including advanced extensions, to create word embeddings that turn sequences of words into vectors accessible to deep learning algorithms. Chapters on classical deep learning algorithms, like convolutional neural networks (CNN) and recurrent neural networks (RNN), demonstrate important NLP tasks as sentence classification and language generation. You will learn how to apply high-performance RNN models, like long short-term memory (LSTM) cells, to NLP tasks. You will also explore neural machine translation and implement a neural machine translator.After reading this book, you will gain an understanding of NLP and you'll have the skills to apply TensorFlow in deep learning NLP applications, and how to perform specific NLP tasks.Style and approachThe book provides an emphasis on both the theory and practice of natural language processing. It introduces the reader to existing TensorFlow functions and explains how to apply them while writing NLP algorithms. The popular Word2vec method is used to teach the essential process of learning word representations. The book focuses on how to apply classical deep learning to NLP, as well as exploring cutting edge and emerging approaches. Specific examples are used to make the concepts and techniques concrete.

Frequently asked questions

How do I cancel my subscription?
Simply head over to the account section in settings and click on “Cancel Subscription” - it’s as simple as that. After you cancel, your membership will stay active for the remainder of the time you’ve paid for. Learn more here.
Can/how do I download books?
At the moment all of our mobile-responsive ePub books are available to download via the app. Most of our PDFs are also available to download and we're working on making the final remaining ones downloadable now. Learn more here.
What is the difference between the pricing plans?
Both plans give you full access to the library and all of Perlego’s features. The only differences are the price and subscription period: With the annual plan you’ll save around 30% compared to 12 months on the monthly plan.
What is Perlego?
We are an online textbook subscription service, where you can get access to an entire online library for less than the price of a single book per month. With over 1 million books across 1000+ topics, we’ve got you covered! Learn more here.
Do you support text-to-speech?
Look out for the read-aloud symbol on your next book to see if you can listen to it. The read-aloud tool reads text aloud for you, highlighting the text as it is being read. You can pause it, speed it up and slow it down. Learn more here.
Is Natural Language Processing with TensorFlow an online PDF/ePUB?
Yes, you can access Natural Language Processing with TensorFlow by Thushan Ganegedara in PDF and/or ePUB format, as well as other popular books in Computer Science & Artificial Intelligence (AI) & Semantics. We have over one million books available in our catalogue for you to explore.

Information

Natural Language Processing with TensorFlow


Table of Contents

Natural Language Processing with TensorFlow
Why subscribe?
PacktPub.com
Contributors
About the author
About the reviewers
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
1. Introduction to Natural Language Processing
What is Natural Language Processing?
Tasks of Natural Language Processing
The traditional approach to Natural Language Processing
Understanding the traditional approach
Example – generating football game summaries
Drawbacks of the traditional approach
The deep learning approach to Natural Language Processing
History of deep learning
The current state of deep learning and NLP
Understanding a simple deep model – a Fully-Connected Neural Network
The roadmap – beyond this chapter
Introduction to the technical tools
Description of the tools
Installing Python and scikit-learn
Installing Jupyter Notebook
Installing TensorFlow
Summary
2. Understanding TensorFlow
What is TensorFlow?
Getting started with TensorFlow
TensorFlow client in detail
TensorFlow architecture – what happens when you execute the client?
Cafe Le TensorFlow – understanding TensorFlow with an analogy
Inputs, variables, outputs, and operations
Defining inputs in TensorFlow
Feeding data with Python code
Preloading and storing data as tensors
Building an input pipeline
Defining variables in TensorFlow
Defining TensorFlow outputs
Defining TensorFlow operations
Comparison operations
Mathematical operations
Scatter and gather operations
Neural network-related operations
Nonlinear activations used by neural networks
The convolution operation
The pooling operation
Defining loss
Optimization of neural networks
The control flow operations
Reusing variables with scoping
Implementing our first neural network
Preparing the data
Defining the TensorFlow graph
Running the neural network
Summary
3. Word2vec – Learning Word Embeddings
What is a word representation or meaning?
Classical approaches to learning word representation
WordNet – using an external lexical knowledge base for learning word representations
Tour of WordNet
Problems with WordNet
One-hot encoded representation
The TF-IDF method
Co-occurrence matrix
Word2vec – a neural network-based approach to learning word representation
Exercise: is queen = king – he + she?
Designing a loss function for learning word embeddings
The skip-gram algorithm
From raw text to structured data
Learning the word embeddings with a neural network
Formulating a practical loss function
Efficiently approximating the loss function
Negative sampling of the softmax layer
Hierarchical softmax
Learning the hierarchy
Optimizing the learning model
Implementing skip-gram with TensorFlow
The Continuous Bag-of-Words algorithm
Implementing CBOW in TensorFlow
Summary
4. Advanced Word2vec
The original skip-gram algorithm
Implementing the original skip-gram algorithm
Comparing the original skip-gram with the improved skip-gram
Comparing skip-gram with CBOW
Performance comparison
Which is the winner, skip-gram or CBOW?
Extensions to the word embeddings algorithms
Using the unigram distribution for negative sampling
Implementing unigram-based negative sampling
Subsampling – probabilistically ignoring the common words
Implementing subsampling
Comparing the CBOW and its extensions
More recent algorithms extending skip-gram and CBOW
A limitation of the skip-gram algorithm
The structured skip-gram algorithm
The loss function
The continuous window model
GloVe – Global Vectors representation
Understanding GloVe
Implementing GloVe
Document classification with Word2vec
Dataset
Classifying documents with word embeddings
Implementation – learning word embeddings
Implementation – word embeddings to document embeddings
Document clustering and t-SNE visualization of embedded documents
Inspecting several outliers
Implementation – clustering/classification of documents with K-means
Summary
5. Sentence Classification with Convolutional Neural Networks
Introducing Convolution Neural Networks
CNN fundamentals
The power of Convolution Neural Networks
Understanding Convolution Neural Networks
Convolution operation
Standard convolution operation
Convolving with stride
Convolving with padding
Transposed convolution
Pooling operation
Max pooling
Max pooling with stride
Average pooling
Fully connected layers
Putting everything together
Exercise – image classification on MNIST with CNN
About the data
Implementing the CNN
Analyzing the predictions produced with a CNN
Using CNNs for sentence classification
CNN structure
Data transformation
The convolution operation
Pooling over time
Implementation – sentence classification with CNNs
Summary
6. Recurrent Neural Networks
Understanding Recurrent Neural Networks
The problem with feed-forward neural networks
Modeling with Recurrent Neural Networks
Technical description of a Recurrent Neural Network
Backpropagation Through Time
How backpropagation works
Why we cannot use BP directly for RNNs
Backpropagation Through Time – training RNNs
Truncated BPTT – training RNNs efficiently
Limitations of BPTT – vanishing and exploding gradients
Applications of RNNs
One-to-one RNNs
One-to-many RNNs
Many-to-one RNNs
Many-to-many RNNs
Generating text with RNNs
Defining hyperparameters
Unrolling the inputs over time for Truncated BPTT
Defining the validation dataset
Defining weights and biases
Defining state persisting variables
Calculating the hidden...

Table of contents