Using Hyperas for Hyperparameter Tuning of Keras Model
Building a model is of no use if you cannot optimize it for a good performance and a higher accuracy. Generally, the model building requires less time than optimizing that model, because during optimization or tuning the model you need to look out for the best parameters which is a time-consuming process.
We can automate this process of finding out the best values for the hyperparameters and getting the highest accuracy of the Keras model. In this article, we will be discussing Hyperas which is an open-source python package used for automating the process of Keras Model Hyperparameter Tuning.
Let’s get started……
Like any other python library, we will use pip installation to install hyperas. Before installation, we need to run a command which is required to avoid errors at a later stage, make sure this command is the first code that you run while building this.
from __future__ import print_function
!pip install hyperas
In order to run hyperas, we need to change certain settings and also we will use a file named exactly the same as our collab notebook i.e if my collab notebook is named Hyperas.ipynb then in the below code, I will use Hyperas.ipynb as the file name.
!pip install -U -q PyDrive
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials# Authenticate and create the PyDrive client.
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)# Copy/download the file
fid = drive.ListFile({'q':"title='Hyperas.ipynb'"}).GetList()[0]['id']
f = drive.CreateFile({'id': fid})
f.GetContentFile('Hyperas.ipynb')
Next, we will import all the required dependencies for this article.
import numpy as npfrom hyperopt import Trials, STATUS_OK, tpe
from keras.datasets import mnist
from keras.layers.core import Dense, Dropout, Activation
from keras.models import Sequential
from keras.utils import np_utilsfrom hyperas import optim
from hyperas.distributions import choice, uniform
In this step, we will create two functions that will help us in loading the data and creating the model respectively.
Loading The Data
def data():
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(60000, 784)
x_test = x_test.reshape(10000, 784)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255
nb_classes = 10
y_train = np_utils.to_categorical(y_train, nb_classes)
y_test = np_utils.to_categorical(y_test, nb_classes)
return x_train, y_train, x_test, y_test
Creating The Model
def create_model(x_train, y_train, x_test, y_test):model = Sequential()
model.add(Dense(512, input_shape=(784,)))
model.add(Activation('relu'))
model.add(Dropout({{uniform(0, 1)}}))
model.add(Dense({{choice([256, 512, 1024])}}))
model.add(Activation({{choice(['relu', 'sigmoid'])}}))
model.add(Dropout({{uniform(0, 1)}}))
if {{choice(['three', 'four'])}} == 'four':
model.add(Dense(100))model.add({{choice([Dropout(0.5), Activation('linear')])}})
model.add(Activation('relu'))model.add(Dense(10))
model.add(Activation('softmax'))model.compile(loss='categorical_crossentropy', metrics=['accuracy'],
optimizer={{choice(['rmsprop', 'adam', 'sgd'])}})result = model.fit(x_train, y_train,
batch_size={{choice([64, 128])}},
epochs=2,
verbose=2,
validation_split=0.1)
#get the highest validation accuracy of the training epochs
validation_acc = np.amax(result.history['val_accuracy'])
print('Best validation acc of epoch:', validation_acc)
return {'loss': -validation_acc, 'status': STATUS_OK, 'model': model}
This is the final step where we will use hyperas and find out the best parameters and highest accuracy.
if __name__ == '__main__':
best_run, best_model = optim.minimize(model=create_model,
data=data,
algo=tpe.suggest,
max_evals=5,
trials=Trials(),
notebook_name='Hyperas')
X_train, Y_train, X_test, Y_test = data()
print("Evalutation of best performing model:")
print(best_model.evaluate(X_test, Y_test))
print("Best performing model chosen hyper-parameters:")
print(best_run)
Here you can clearly analyze how hyperas displayed the best performing hyperparameters along with the accuracy of the model. This is easy and can save time and effort.
Go ahead try this and let me know your experiences in the response section.
This article is in collaboration with
Thanks for reading! If you want to get in touch with me, feel free to reach me on hmix13@gmail.com or my LinkedIn Profile. You can view my Github profile for different data science projects and packages tutorials. Also, feel free to explore my profile and read different articles I have written related to Data Science.