imdb movie review sentiment analysis github

Sentiment Analysis of IMDB Movie Reviews using Convolutional Neural Network (CNN) with Hyperparameters Tuning

Alireza bagheri, table of contents.

• Load IMDB movie reviews
• Decode reviews from index
• Truncate and pad the review sequences
• Build the model
• Create the model
• Tune hyperparameters
• Train the model
• Evaluate the model

Data ¶

In this project, I will use IMDB movie reviews. This dataset contains 50,000 movie's reviews from IMDB, labeled by sentiment (positive/negative). The dataset can be loaded and splitted into training and test sets as the following.

Load IMDB movie reviews ¶

Let us have a look at the first sample of training set.

As it clear, the text of reviews is integer-encoded, where each integer represents a specific word in the dictionary.

Decode reviews from index ¶

We can convert the integers back to words as the following.

In continue, I will only consider the top 5,000 most common words. I will also consider 20% of the training set for validation purpose.

Let us inspect how the first review looks like when we only consider the top 5,000 frequent words.

Truncate and pad the review sequences ¶

Movie reviews can be different lengths. We will use the pad_sequences function to standardize the lengths of the reviews.

Let us check the first padded review.

Build the model ¶

Create the model ¶.

In this project, I will consider a Convolutional Neural Network (CNN) for the text classification.

Tune hyperparameters ¶

Now, it is time to tweak hyperparameters to imporve accuracy over validation set.

Train the model ¶

Here, I train the model with the best obtained hyperparameters over train + validation sets.

Evaluate the model ¶

Finally, I evaluate performance of the trained model over unsean test set.

Reference ¶

https://keras.io/examples/imdb_cnn/

IMDb with Vanilla RNNs ¶

Imdb movie review dataset and preprocessing ¶.

The IMDb Movie Review Dataset contains 50,000 reviews, split 50/50 into positive and negative reviews. The goal is to create a model that can accurately predict the sentiment (positive or negative) of a given review.

In order for a review to become the input of a neural network, we need to preprocess the text. The first step is to tokenize the text, meaning to break up the text into individual units that are easily understood, such as words. The spacy tokenizer does this fairly well by splitting text by spaces and separating punctuation. The reviews also contain <br /> tags which need to be removed, which necessitates a custom tokenizer based on spacy.

Using the custom tokenizer, we can load and process the entire IMDb dataset (text and labels) using torchtext fields. To make it easier to access the dataset without having to spend time tokenizing every time, we can save the processed data into a json file.

Now we can easily load in the dataset. We also want to split the dataset into train/validation/test sets with sizes 60%/20%/20%.

Let’s take a look at one example review. For this dataset, a label of 0 is negative and 1 is positive.

We can build a vocabulary of tokens from the training set. The most common vocab tokens are show below. Each token in the vocab is assigned to a unique index, which allows for each token to be represented as a one-hot vector, which has the same length as the vocab. All elements are set to 0 except for the element at the token’s corresponding index, which is instead set to 1.

By setting a maximum of 25,000 tokens, we can prevent the model from learning the sentiment of every single token. All other words are simply replaced with <unk>, which will be ignored by the model.

We want to iterate through the data in batches for SGD. However, all sequences in a batch need to have the same length. This is accomplished by adding padding to the end of sequences, represented as <pad>. To minimize the amount of padding needed, we can use BucketIterators that sort the reviews by length and allow for batch iteration.

Recurrent Neural Network ¶

We’re going to use a simple, vanilla RNN for classification. RNNs have a hidden state that is determined by the previous hidden state, thus creating recurrence. This allows it to have a “memory”.

For classifying reviews, each time step inputs a single token formatted as a one-hot vector. The embedding layer converts this input to a dense embedding vector that no longer contains only 0 and 1. The RNN layer uses the embedding vector and previous hidden state to return an output and the current hidden state. The RNN output is passed to a fully connected linear output layer that allows for binary classification. The RNN model follows the tokens in sequential order, simulating how a human reads.

To improve accuracy, padded sequences are packed so that the RNN doesn’t learn from the padding tokens. The sequences are then repacked to continue passing data through the model. Dropout is also used in the embedding and hidden layers to randomly ignore/drop out nodes, which helps prevent overfitting.

Training and evaluation is nearly the same as for MLPs. The only difference is that each batch now needs to return sequence length along with the sequences so that the model can correctly pad and pack sequences.

We define methods for accuracy, training, and evaluation of single batches, then combine them into a method to train and validate a model over multiple epochs, returning loss and accuracy as it trains.

Let’s define

Now let’s see how the trained model performs on a test set it’s never seen before.

62% isn’t a great accuracy since 50% is chance performance. We can improve the model by using pretrained word embeddings that are used to initialize the weights to the embedding layer instead of using random initialization. This change increases the model’s rate of improvement and likelihood of finding a good local minimum or even global minimum for loss.

72% accuracy is much better, but there’s still room to improve. Better models such as LSTMs can be used to achieve higher accuracy, which we will explore in the next section.

Nested Cross Validation ¶

Using a test/val/train split is already quite effective. However, it only creates one split that can possibly create unbalanced datasets due to a random split. Consequently, a single test accuracy may not be fully representative of the model’s true accuracy.

Nested cross validation is a solution to this problem. Instead of splitting the dataset once, k-fold CV splits the dataset into k equal parts known as folds. In the outer CV, one of these folds are used as a test set, while the other k-1 folds are used as a trainval set. Then in the inner CV, the trainval set is similarly split into k folds, one of which is used as a validation set while the rest are used as a training set. In the outer and inner CVs, all k folds are used as a training and validation set, respectively.

In essence, the roles of each set remain the same, but apply to multiple folds instead of one. The validation set is used for hyperparameter optimization. In this case, optimization is done using a grid search, which finds the lowest mean loss across all k inner folds. Similarly, the test set is used to find the mean test loss and accuracy across all k outer folds.

Here we see that the mean test accuracy is much lower than without using nested CV. While the first outer fold performed fairly well, the second and third folds did not. Nested CV was able to give a better, less biased evaluation of our model than with a train/val/test split, though at the expense of a much longer computation time. 3 folds is also less than ideal, as 5 or 10 folds are generally preferred. The number of folds and epochs were only chosen to achieve a realistic computation time, as this project is primarily a proof of concept.

Now that we know the RNN model needs to be improved, we’ll explore using an LSTM model in the next section.

Sentiment Analysis on IMDB Movie Reviews

The goal of this project is to build a deep learning model that is able to understand if a comment that was made is positive or negative.

The model consists of an Embedding Layer, then the RNN and finally with a fully connected layer that is responsible for the final classification, 1/0 positive or negative review. For the training and evaluation of the model was used the IMDB Movie Reviews dataset.

Tokenizing Data

First we have to split the text into words in order to be able to construct the vocabulary.

(['i', 'don', 't', 'know', 'why', 'i', 'like', 'this', 'movie', 'so', 'well', 'but', 'i', 'never', 'get', 'tired', 'of', 'watching', 'it'], 1)

(['this', 'is', 'the', 'definitive', 'movie', 'version', 'of', 'hamlet', 'branagh', 'cuts', 'nothing', 'but', 'there', 'are', 'no', 'wasted', 'moments'], 1)

(['adrian', 'pasdar', 'is', 'excellent', 'is', 'this', 'film', 'he', 'makes', 'a', 'fascinating', 'woman'], 1)

(['ming', 'the', 'merciless', 'does', 'a', 'little', 'bardwork', 'and', 'a', 'movie', 'most', 'foul'], 0)

(['long', 'boring', 'blasphemous', 'never', 'have', 'i', 'been', 'so', 'glad', 'to', 'see', 'ending', 'credits', 'roll'], 0)

(['this', 'movie', 'is', 'terrible', 'but', 'it', 'has', 'some', 'good', 'effects'], 0)

The 0/1 in the end indicates if a comment is positive or negative.

Creating Vocabulary

First, we find the most frequent words that were used for the reviews:

with end of sequence symbol used for padding and are the unknown words in our vocabulary.

Then, we have to construct a vocabulary which maps each word to an integer. An exampe review with the mapped integers looks like this:

{'data': tensor([ 3, 5, 12, 1, 4, 1, 10, 1, 1, 1, 1, 6, 1, 1, 1, 1, 1]), 'label': tensor(1.)}

Constructing Embeddings

Now, we want to be able to understand the semantic relations between words and to understand if an order of the words has no meaning at all. For this purpose we are going to construct an Embedding Layer.

Then, we defined the Text classifier whiches components we already mentioned above, and trained the model with our dataset. After tunning the hyperparameters we got the following result.

81,23% Accuracy

Furthermore, a small live demo was created just for fun:

PyTorch was used for the development of the model.

Predicting the Sentiment of IMDB Movie Reviews using LSTM in PyTorch

November 9, 2022

This notebook takes inspiration and ideas from the following sources.

• “Machine learning with PyTorch and Scikit-Learn” by “Sebastian Raschka, Yuxi (Hayden) Liu, and Vahid Mirjalili”. You can get the book from its website: Machine learning with PyTorch and Scikit-Learn . In addition, the GitHub repository for this book has valuable notebooks: github.com/rasbt/machine-learning-book . Parts of the code you see in this notebook are taken from chapter 15 notebook of the same book.
• “Intro to Deep Learning and Generative Models Course” lecture series from “Sebastian Raschka”. Course website: stat453-ss2021 . YouTube Link: Intro to Deep Learning and Generative Models Course . Lectures that are related to this post are L15.5 Long Short-Term Memory and L15.7 An RNN Sentiment Classifier in PyTorch

Environment

This notebook is prepared with Google Colab.

• GitHub : 2022-11-09-pytorch-lstm-imdb-sentiment-prediction.ipynb

For “runtime type” choose hardware accelerator as “GPU”. It will take a long time to complete the training without any GPU.

This notebook also depends on the PyTorch library TorchText . We will use this library to fetch IMDB review data. While using the torchtext latest version, I found more dependencies on other libraries like torchdata . Even after resolving them, it threw strange encoding errors while fetching IMDB data. So I have downgraded this library till the version I found working without external dependencies. Consequently, torch is also downgraded to a compatible version, but I did not find any issue while working with a lower version of PyTorch for this notebook. It is preferred to restart the runtime after the library installation is complete.

Data Preparation

Download data.

Let’s download our movie review dataset. This dataset is also known as Large Movie Review Dataset , and can also be obtained in a compressed zip file from this link . Using the torchtext library makes downloading, extracting, and reading files a lot easier. ‘torchtext.datasets’ comes with many more NLP related datasets, and a full list can be found here .

Check the size of the downloaded data.

Split train data further into train and validation set

Both train and test datasets have 25000 reviews. Therefore, we can split the training set further into the train and validation sets.

How does this data look?

The data we have is in the form of tuples. The first index has the sentiment label, and the second contains the review text. Let’s check the first element in our training dataset.

Check the first index of the validation set.

Data preprocessing steps

From these two reviews, we can deduce that

• We have two labels. ‘pos’ for a positive and ‘neg’ for a negative review
• From the second review (from valid_dataset), we also get that text may contain HTML tags, special characters, and emoticons besides normal English words. It will require some preprocessing to remove them for proper word tokenization.
• Reviews can have varying text lengths. It will require some padding to make all review texts the same size.

Let’s take a simple text example and apply these steps to understand why these steps are essential in preprocessing. In the last step, we will create tokens from the preprocessed text.

Let’s put all the preprocessing steps in a nice function and give it a name.

Apply tokenizer on the example_text to verify the output.

Preparing data dictionary

We are successful in creating word tokens from our example_text . But there is one more problem. Some of the tokens are repeating. If we can convert these tokens into a dictionary along with their frequency count, we can significantly reduce the generated token size from these reviews. Let’s do that.

Let’s sort the output to have the most common words at the top.

It shows that in our example text, the top place is taken by pronouns (i and it) followed by the emoticon. Though our data is now correctly processed, it needs to be prepared to be fed to a model. Because [machine] models love math and work with numbers exclusively. To convert our dictionary of word tokens into integers, we can take help from torchtext.vocab . Its purpose in the official documentation is defined as link here

Factory method for creating a vocab object which maps tokens to indices.

Note that the ordering in which key value pairs were inserted in the ordered_dict will be respected when building the vocab. Therefore if sorting by token frequency is important to the user, the ordered_dict should be created in a way to reflect this.

It highlights three points:

• It maps tokens to indices
• It requires an ordered dictionary ( OrderedDict ) to work
• Tokens in vocab at the starting indices reflect higher frequency

This generated vocabulary shows that tokens with higher frequency ( i , it ) have been assigned lower indices (or integers). This vocabulary will act as a lookup table for us, and during training for each word token, we will find a corresponding index from this vocab and pass it to our model.

We have done many steps while processing our example_text . Let’s summarize them here before moving further

Summary of data dictionary preparation steps

• Generate tokens from text using the function tokenizer
• Find the frequency of tokens using Python collections.Counter
• Sort the tokens based on their frequency in descending order
• Put the sorted tokens in Python collections.OrderedDict
• Convert the tokens into integers using torchtext.vocab

Let’s apply all these steps on our IMDB reviews training dataset.

After tokenizing IMDB reviews, we find that there 69023 unique tokens.

We have added two extra tokens to our vocabulary.

• “pad” for padding. This token will come in handy when we pad our reviews to make them of the same length
• “unk” for unknown. This token will come in handy if we find any token in the validation or test set that was not part of the train set

Let’s also print the tokens present at the first ten indices of our vocab object.

It shows that articles, prepositions, and pronouns are the most common words in the training dataset. So let’s also check the least common words.

The least common words seem to be people or place names or misspelled words like ‘queueing’ and ‘seriousuly’.

Define data processing pipelines

At this point, we have our tokenizer function and vocabulary lookup ready. For each review item from the dataset, we are supposed to perform the following preprocessing steps:

For review text

• Create tokens from the review text
• Assign a unique integer to each token from the vocab lookup

For review label

• Assign 1 for pos and 0 for neg label

Let’s create two simple functions (inline lambda) for review text and label processing.

Instead of processing a single review at a time, we always prefer to work with a batch of them during model training. For each review item in the batch, we will be doing the same preprocessing steps i.e. review text processing and label processing. For handling preprocessing steps at a batch level, we can create another higher-level function that applies preprocessing steps at a batch level.

Sequence padding

In the above collate_batch function, I added one extra padding step.

We intend to make all review texts in a batch of the same length. For this, we take the maximum length of a text in a batch, all pad all the smaller text with extra dummy tokens (‘pad’) to make their sizes equal. Finally, with all the data in a batch of the same dimension, we convert it into a tensor matrix for faster processing.

To understand how PyTorch utility nn.utils.rnn.pad_sequence works, we can take a simple example of three tensors (a, b, c) of varying sizes (1, 3, 5).

Now let’s pad them to make sizes consistant.

Sequence packing

From the above output, we can see that after padding tensors of varying sizes, we can convert them into a single matrix for faster processing. But the drawback of this approach is that we can have many, many padded tokens in our matrix. They are not helping us in any way, instead of occupying a lot of machine memory. To avoid this, we can also squish these matrixes into a much condensed form called packed padded sequences using PyTorch utility nn.utils.rnn.pack_padded_sequence .

Here the tensor still holds all the original tensor values (1 to 9) but is very condensed and has no extra padded token. So how does this tensor know which tokens belong to which token? For this, it stores some additional information.

• batch sizes (or original tensor length)
• tensor indices

We can move back and forth between the padded pack and unpacked sequences using this information.

Run data preprocessing pipelines on an example batch

Let’s load our data in the PyTorch DataLoader class and create a small batch of 4 reviews. Preprocess the entire set with collate_batch function.

• text_batch.shape: torch.Size([4, 218]) tells us that in this batch, there are four reviews (or their tokens) and all have the same length of 218
• label_batch: tensor([1., 1., 1., 0.]) tells us that the first three reviews are positive and the last is negative
• length_batch: tensor([165, 86, 218, 145]) tells us that before padding the original length of review tokens

Let’s check what the first review in this batch looks like after preprocessing and padding.

To complete the picture, I have re-printed the original text of the first review and manually processed a part of it. You can verify that the tokens match.

Batching the training, validation, and test dataset

Let’s proceed on creating DataLoaders for train, valid, and test data with batch_size = 32

Define model training and evaluation pipelines

I have defined two simple functions to train and evaluate the model in this section.

RNN model configuration, loss function, and optimizer

We have seen the review text, which can be long sequences. We will use the LSTM layer for capturing the long-term dependencies. Our sentiment analysis model is composed of the following layers

• Start with an Embedding layer . Placing the embedding layer is similar to one-hot-encoding, where each word token is converted to a separate feature (or vector or column). But this can lead to too many features (curse of dimensionality or dimensional explosion). To avoid this, we try to map tokens to fixed-size vectors (or columns). In such a feature matrix, different elements denote different tokens. Tokens that are closed are also placed together. Further, during training, we also learn and update the positioning of tokens. Similar tokens are placed into closer and closer locations. Such a matrix layer is termed an embedding layer.
• After the embedding layer, there is the RNN layer (LSTM to be specific).
• Then we have a fully connected layer followed by activation and another fully connected layer.
• Finally, we have a logistic sigmoid layer for prediction

Define model loss function and optimizer

For loss function (or criterion), I have used Binary Cross Entropy , and for loss optimization, I have used Adam algorithm

Model training and evaluation

Let’s run the pipeline for ten epochs and compare the training and validation accuracy.

Evaluate sentiments on random texts

Let’s create another helper method to evaluate sentiments on random texts.

SENTIMENT ANALYSIS FOR IMDB MOVIE REVIEWS

This article shows the step-by-step process on performing sentiment analysis on IMDB movie reviews. The datasets used in this project are from kaggle .

Executive Summary

In this study, we conducted a sentiment analysis of movie reviews from IMDB to predict the overall sentiment of the audience towards a given movie. We used a combination of natural language processing techniques and machine learning algorithms to analyze the text of the reviews and classify them as either positive or negative. The models used are logistic regression, multinomial naive bayes, and neural network.

We found that our model was able to predict the sentiment of the reviews with a high degree of accuracy. Additionally, we also identified some common themes and words that were more prevalent in positive or negative reviews, serving as indicators of the audience’s sentiment.

Overall, our study provides valuable insights into the sentiment of movie audiences and can be used to inform the marketing and promotion of films.

Background and Research Questions

Conventionally, marketing and promotional campaigns for movies are based on historical data and tend to focus on movies that are similar to previously successful ones. This approach can exclude new or growing markets that might have different preferences. To better capture the evolving preferences of movie audiences, we made use of sentiment analysis on IMDB movie review data to build a predictive model. This can be helpful for the stakeholders by automating the extraction of audience sentiment on a movie efficiently, thereby informing the promotion and distribution of film.

The study aims to answer the following questions:

IMDB Movie Reviews Dataset

Source: IMDB Dataset of 50K Movie Reviews

Methodology

1. data preprocessing.

Normalization: The steps involved in cleaning vary according to data. For the movie reviews, we:

Tokenization and Lemmatization: Tokenization is the process of extracting words as separate tokens. Lemmatization is the process of returning words into their base form, e.g., “plays” -> “play.” Tokenization is necessary before lemmatization as it allows the lemmatization algorithm to operate on individual words rather than on the entire text. This makes the lemmatization process more efficient and accurate.

2. Vectorization

Encode the texts into numerical form before inputting the data into machine learning models. For this study, the TF-IDF algorithm is used due to its ability to take into account the frequency and context of words. Before vectorization, the normalized dataset is split into X_train, X_test, y_train, and y_test.

3. Model Training and Evaluation

Models trained are:

Predicting Sentiment

4. Results and Findings

All in all, we can make use of predictive models to predict audience sentiment based on their review comments. For this purpose, logistic regression is the best-performing model as it has the highest accuracy with fairly low training and testing time.

In this study, we conducted a sentiment analysis of movie reviews from IMDB to predict the sentiment of the audience towards a given movie based on unstructured review. It is important to note that thorough preprocessing involving text normalization and vectorization is needed before model training. Our findings showed that logistic regression was able to accurately predict the sentiment of the reviews with an accuracy of 0.8864 and fairly efficient training time. The common themes and words that were indicative of positive or negative sentiment are extracted from the model to allow for interpretation.

These findings have important implications for the film industry as they can be used to inform the marketing and promotion of movies, helping the stakeholders to capture the new market with ever-evolving preferences on movies.

The complete Python notebook for the sentiment analysis mentioned in this article can be found here .

Trending Tags

Deep Learning

IMDB Sentiment analysis

Imdb sentiment analysis #.

This tutorial is based on An Introduction to Keras Preprocessing Layers by Matthew Watson, Text classification with TensorFlow Hub: Movie reviews and Basic text classification by TensorFlow.

Main topics in this tutorial:

Build a binary sentiment classification model with keras

Use keras layers for data preprocessing

Use TensorBoard to view model results

Save and reload the model

Example for multiple feature engineering steps

Prerequisites #

To start this tutorial, you need the following setup:

Install TensorFlow (Note that we install TensorFlow Extended to obtain more deployment options. However, we don’t use the options in this tutorial)

We use the IMDB dataset with 50,000 polar movie reviews (positive or negative)

Training data and test data: each 25,000

Training and testing sets are balanced (they contain an equal number of positive and negative reviews)

The input data consists of sentences (strings)

The labels to predict are either 0 or 1.

Data import #

We use 3 data splits: training, validation and test data

Split the data into 60% training and 40% test

Split training into 60% training and 40% validation

Resulting data split:

15,000 examples for training

10,000 examples for validation

25,000 examples for testing

Explore data #

Each example is a sentence representing the movie review and a corresponding label.

The sentence is not preprocessed in any way.

The label is an integer value of either 0 or 1

0 is a negative review

1 is a positive review.

Let’s print first 2 examples.

Data preprocessing #

First, we need to decide how to represent the text data

TextVectorization #

We will be working with raw text (natural language inputs)

So we will use the TextVectorization layer.

It transforms a batch of strings (one example = one string) into either a

list of token indices (one example = 1D tensor of integer token indices) or

dense representation (one example = 1D tensor of float values representing data about the example’s tokens).

TextVectorization steps:

Standardize each example (usually lowercasing + punctuation stripping)

Split each example into substrings (usually words)

Recombine substrings into tokens (usually ngrams)

Index tokens (associate a unique int value with each token)

Transform each example using this index, either into a vector of ints or a dense float vector.

Multi-hot encoding #

Multi-hot encoding: only consider the presence or absence of terms in the review.

For example:

layer vocabulary is [‘movie’, ‘good’, ‘bad’]

a review read ‘This movie was bad.’

We would encode this as [1, 0, 1]

where movie (the first vocab term) and bad (the last vocab term) are present.

Create a TextVectorization layer with multi-hot output and a max of 2500 tokens

Map over our training dataset and discard the integer label indicating a positive or negative review (this gives us a dataset containing only the review text)

adapt() the layer over this dataset, which causes the layer to learn a vocabulary of the most frequent terms in all documents, capped at a max of 2500.

Adapt is a utility function on all stateful preprocessing layers, which allows layers to set their internal state from input data.

Calling adapt is always optional.

For TextVectorization, we could instead supply a precomputed vocabulary on layer construction, and skip the adapt step.

Next, we define a preprocessing function

This is especially useful if you combine multiple preprocessing steps

Here, we only use one step: preprocess converts raw input data to the representation we want for our model

Architecture #

Show model summary

Let’s visualize the topology of the model

We can now train a simple model on top of this multi-hot encoding.

First, we set up TensorBoard and an early stopping rule:

Define the directory where TensorBoard stores log files (we create folders with timestamps by using datetime )

We add keras.callbacks.TensorBoard callback which ensures that logs are created and stored.

To prevent overfitting, we use a callback wich will stop the training when there is no improvement in the validation accuracy for three consecutive epochs.

Model training:

Train the model for 10 epochs in mini-batches of 512 samples

We shuffle the data and use a buffer_size of 10000

We monitor the model’s loss and accuracy on the 10,000 samples from the validation set.

buffer_size is the number of items in the shuffle buffer. The function fills the buffer and then randomly samples from it. A big enough buffer is needed for proper shuffling, but it’s a balance with memory consumption. Reshuffling happens automatically at every epoch

Show number of epochs:

Evaluation #

Loss and accuracy #.

Show loss and accuracy for test data

Create a plot of accuracy and loss over time

model.fit() returns a history object that contains a dictionary with everything that happened during training.

There are four entries: one for each monitored metric during training and validation.

You can use these to plot the training and validation loss for comparison, as well as the training and validation accuracy:

Blue dots represent the training loss and accuracy

Solid red lines are the validation loss and accuracy.

Training loss decreases with each epoch

Training accuracy increases with each epoch.

This is expected when using a gradient descent optimization

It should minimize the desired quantity on every iteration.

TensorBoard #

We use the tensorboard.notebook API

Alternative option to view TensorBoard:

How to use TensorBoard in Visual Studio Code ( Stackoverflow ):

Open the command palette (Ctrl/Cmd + Shift + P)

Search for the command “Python: Launch TensorBoard” and press enter.

Select the folder where your TensorBoard log files are located:

Select folder logs/fit

Inference on new data #

Create new example data

Add a sigmoid activation layer to our model to obtain probabilities

Make predictions

Save model #

A Keras model consists of multiple components:

The architecture, or configuration, which specifies what layers the model contain, and how they’re connected.

A set of weights values (the “state of the model”).

An optimizer (defined by compiling the model).

A set of losses and metrics (defined by compiling the model or calling add_loss() or add_metric()).

The Keras model saving API makes it possible to save all of these pieces to disk at once, or to only selectively save some of them:

Saving everything into a single archive in the TensorFlow SavedModel format (or in the older Keras H5 format). This is the standard practice.

Saving the architecture / configuration only, typically as a JSON file.

Saving the weights values only. This is generally used when training the model.

We will save the complete model as Tensorflow SavedModel

Load model #

Multiple feature engineering steps #.

The following code is an add on to demonstrate how to perform further feature engineering

Let’s experiment with a new feature

Our multi-hot encoding does not contain any notion of review length

We can try adding a feature for normalized string length.

Create the normalization layer (which will scale the input to have 0 mean and 1 standard deviation)

Adapt it to our input

Within the preprocess function, we simply concatenate our multi-hot encoding and length features together

Use the new preprocess function in our model (we don’t use TensorBoard in this example):

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications