recurrent neural network - Simple RNN in Torch -

i trying implement rnn in torch. used start simple task of predicting next item in sequences. sequences subsequences of {1,2,3,4,5,6,7,8,9,10} offset chosen randomly.

i want implement network architecture 1 hidden layer lstm cells. why use nn.seqlstm(inputsize, outputsize)

rho = 5 -- number of steps bptt hiddensize = 15 inputsize = 1 outputsize = 1 seqlen = 5 nindex = 10 batchsize = 4  seqlstm = nn.seqlstm(inputsize, outputsize) criterion = nn.sequencercriterion(nn.classnllcriterion())   outputs = seqlstm:forward(inputs) -- inputs seqlen x batchsize x inputsize err = criterion:forward(outputs, targets) -- targets seqlen x batchsize x 1 

• do need nn.lookuptable?
• this code seems bit simple , missing glue guess. parts missing make complete?

unless not whole code, there quite bit missing, not glue.

1- having input , output size of 1 lstm layer not make sense

2- wouldn't mess around rho unless want network backpropagate 5 timesteps only

3- classnllcriterion expects log probabilities input

4- need optimization algorithm in order train network such sgd (see optim library torch)

5- atleast need input layer , output layer in network. input layer transforms input embeddings forwarded through hidden layer(s) lstm layer. lookuptable can used in input layer. output layer potentially form probability distribution on possible outputs using softmax layer.

6- indeed little simple

nn.lookuptable module stores embeddings input. example, if you're trying predict next character in sequence, have lookuptable of size 26xindim have embedding of size indim each character in alphabet. indeed use one.

looking @ this, suspect might not entirely familiar how neural networks built. suggest familiarize little bit more before jumping in. here nice book michael nielsen deep learning. explains intuitively.

looking @ examples element-research/rnn or torch/nn helpful. karpathy's char-rnn interesting.

i hope of help.