Loading saved keras model and continue training from last epoch
This article talks about loading saved keras model and continuing training from last epoch. To help you better understand, saving keras model requires h5py library. Model is serialized to JSON file whereas weights are serialized to HDF5. During loading of model, both json architecture and saved weights are being loaded. Along with this discussion, there would be some talks how to load model from last epoch or if the model is not saved, then start training from scratch. Code for this article is uploaded on github whose link I would mention in this article. Hence, there is no worry about code. After reading this article, users/readers would gain better insights about loading, restoring and training keras model. There would also be talks on why there is a need to restore the model from the last epoch.
Why there is a need to load the model from last epoch
This is main limelight of this article. Suppose you are training huge dataset for about 10000 epochs and by any reasons, your GPU gets struck/switched off/ hanged at 9999 epoch. Would you start the training from scratch. Is it worth and wise?. Obviously the answer is no and one will think that training must start from 9999 epoch. To help you restore the training from the last epoch (keras model), this article is written. I hope the vision of the article is clear in readers mind and now let us proceed towards github code. Before going further, if readers are interested, then they can read the below article also
Save and Restore Tensorflow Model
Discussion on Github code
Link for Github Code is
Loading-saved-keras-model-and-continue-training-from-last-epoch
Before Starting let us see the structure of the code
Images: This folder contains the images which helps one to understand the code better.
Weight: This folder is the checkpoint directory where weights are stored.
load_saved_keras_model.py: This is a python file which is the main file. This file is used to save keras model and load the model from either scratch or last epoch.
Pima-indians-diabetes.csv: This is the .csv file which is used to train the model.
It contains predictors (Data) as below
# 1. Number of times pregnant
# 2. Plasma glucose concentration a 2 hours in an oral glucose tolerance test
# 3. Diastolic blood pressure (mm Hg)
# 4. Triceps skin fold thickness (mm)
# 5. 2-Hour serum insulin (mu U/ml)
# 6. Body mass index (weight in kg/(height in m)^2)
# 7. Diabetes pedigree function
# 8. Age (years)Target is as below. Target is binary in nature.
# 9. Class variable (0 or 1)How code works
- First of all, data is loaded with the help of pandas library.
- Data (predictors) and Target (class) is splitted using slicing.
- Sequential model is created with single neuron in the last layer. One neuron is used as there is a single class to predict either 0 or 1. Activation function used here is softmax. Checkpoint directory is created. I have created manually. One can create it using mkdir in the code
- Condition is applied if model saves files (.hdf5) using training them training is resumed from last epoch else training starts from scratch. This condition is applied if training stops because of some reasons. Please see below screenshot.
Conclusion
As we have seen in this tutorial why resuming of training is important. Some light is also thrown on keras weights, model and hdf5 file. I have tried to create condition for both starting the training from scratch or starting from the last epoch. I hope readers have gained some better insights after reading this small article.
Originally published at http://ersanpreet.wordpress.com on December 22, 2019.
