Deep Neural Frameworks
Last updated
Was this helpful?
Last updated
Was this helpful?
Deep learning with pytorch -
, - yann lecun
Pytorch Official
|| || || || || || || ||
(good) -
, has several videos on the topic, going through many network types, creating custom activation functions, going through examples.
+ Two extra videos from the same author, and
Didn’t read:
Compares label with the rounded predicted float, i.e. bigger than 0.5 = 1, smaller than = 0
For categorical we take the argmax for the label and the prediction and compare their location.
In both cases, we average the results.
Note: probably doesn't reflect on adam, is there a reference?
KERAS FUNCTIONAL API
Keras metrics
The training loss is the average of the losses over each batch of training data. Because your model is changing over time, the loss over the first batches of an epoch is generally higher than over the last batches. On the other hand, the testing loss for an epoch is computed using the model as it is at the end of the epoch, resulting in a lower loss.
- Example script showing how to use stateful RNNs to model long sequences efficiently.
- this script demonstrate the use of a conv LSTM network, used to predict the next frame of an artificially generated move which contains moving squares.
and not teano (set the .bat file)
- with code example.
ns in NN Keras.
- classification regression and custom metrics
- accuracy, ROC, AUC, classification, regression r^2.
using MSE, comparing baseline vs wide vs deep networks.
? Formula and explanation
. Which are taken from , including entropy and f1
, in keras-tensorflow. When you do multi-GPU training, it is important to feed all the GPUs with data. It can happen that the very last batch of your epoch has less data than defined (because the size of your dataset can not be divided exactly by the size of your batch). This might cause some GPUs not to receive any data during the last step. Unfortunately some Keras Layers, most notably the Batch Normalization Layer, can’t cope with that leading to nan values appearing in the weights (the running mean and variance in the BN layer).
A flexible way to declare layers in parallel, i.e. parallel ways to deal with input, feature extraction, models and outputs as seen in the following images.
, and
.predict() generates output predictions based on the input you pass it (for example, the predicted characters in the )
.evaluate() computes the loss based on the input you pass it, along with any other metrics that you requested in the metrics param when you compiled your model (such as accuracy in the )
A Keras model has two modes: training and testing. Regularization mechanisms, such as Dropout and L1/L2 weight regularization, are turned off at testing time.