Exploring the Depths of Recurrent Neural Networks with Stochastic Residual Learning

Basically, residual connections can be better than stacked RNNs in the presented case of sentiment analysis.