This is the Attention matrix result of evaluation data of the training model.

The color map of the attention matrix was saved as an image after every epoch during the training process.

The following video is a collection of those attention image sequences.

100 epochs were done while training the model.

For example, after epoch 1 of train function trainIters() was done, then insert evaluation data into the trained model and save the attention matrix result. Then restart the trainIters() and after epoch 2 of train function trainIters() was done, then inset evaluation data into the trained model and save the attention matrix result, so on.

The hyperparameters for the following results are

• Encoder learning rate = 0.0004
• Five decoders learning rate = 0.0005 (all same values for five decoders)
• Batch size = 1024
• Teachering force ratio = 0.5
• Time length = 1

1) Thumb finger attention matrix result

Thumb.mp4

2) Index finger attention matrix result

index.mp4

3) Middle finger attention matrix result

Middle.mp4

4) Ring finger attention matrix result

Ring.mp4

5) Pinky finger attention matrix result

Pinky.mp4