Intro

Repeating my previous testing, this time using a GRU layer instead of an LSTM.

Below are results from three different image preprocessing methods, each trained five times with five different weight initializations and five different batch randomizations. Lines are the mean of the five runs, with error bars showing one standard deviation. ‘test’ in the plots below is actually the results on validation data generated using Keras’s validation_split argument to the fit method.

Methods and Results

First is my initial code with no normalization, just dividing by 255 to get values from 0.0 to 1.0.

# After loading images from multiple directories into the list 'images'
images = np.array(images)
images = images.astype(np.float) / 255.0

Second is centering the mean on zero, separately for each color band.

# After loading images from multiple directories into the list 'images'
images = np.array(images)
images = images.astype(np.float)
images[:,:,:,0] -= np.mean(images[:,:,:,0])
images[:,:,:,1] -= np.mean(images[:,:,:,1])
images[:,:,:,2] -= np.mean(images[:,:,:,2])

Third is the above plus dividing each color band by its standard deviation.

# After loading images from multiple directories into the list 'images'
images = np.array(images)
images = images.astype(np.float)
images[:,:,:,0] -= np.mean(images[:,:,:,0])
images[:,:,:,1] -= np.mean(images[:,:,:,1])
images[:,:,:,2] -= np.mean(images[:,:,:,2])
images[:,:,:,0] /= np.std(images[:,:,:,0])
images[:,:,:,1] /= np.std(images[:,:,:,1])
images[:,:,:,2] /= np.std(images[:,:,:,2])

The code used to run these experiments was this commit. I manually changed the lines listed above for the three separate runs and I changed one line in mode_keras.py to use a GRU layer instead of LSTM. Total number of training samples was approximately 7000.

Conclusions

Looks very similar to the LSTM tests. Given the GRU has fewer parameters than the LSTM, I will probably switch to using that.

Models

• Fully Connected
• LSTM
• GRU

Future

• Retrain the best of these experiments with more data.
• Run speed tests on MaLPi with all three versions.