Hello Long

First all thanks for writing such an article. From the article as well as some other sources, I had learnt the following concepts. I am writing to recheck whether I had learn right or wrong.

1.) Seq2Seq model has encoder as well as decoder. Encoder contains different number of LSTM layers and each layer contains different number of neurons. Last hidden layer of the encoder is called thought vectors which serves as input to the decoder.

2.) Decoder contains LSTM layers and each LSTM layer contains memory units. To the first layer of decoder, <sos> and thought vector is input to produce the output and hidden state. Hidden state and output will serve as input to next LSTM cell of the decoder to produce the new output. In this way the new sequence is formed based on the input sequence.

Based on above understanding, I was able to generate the pseudo code also

input = Input(length=layers LSTM at decoder)
model = Model(input, dense(lstm(emb(input))))
thought_vector = encoder output
x = <sos>
for each_value in range(length):
x, h = model.predict(thought_vector, x)

I hope if this needs some correction, please do correct me so that my learning can be improved.