> For the complete documentation index, see [llms.txt](https://blog.sunilgudivada.dev/notebook/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://blog.sunilgudivada.dev/notebook/transformers-and-llms/rnns.md).

# RNNs

Recurrent neural network

Long Short Term memory 

| Method   | Pros                                                                       | Cons                                                                                 |
| -------- | -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| Word2Vec | <ul><li>Very simple, yet powerful</li><li>Intitutive embdeddings</li></ul> | <ul><li>Word order does not count</li><li>Embeddings are not context aware</li></ul> |
| RNNs     | <ul><li>Word Order matters</li><li>State of art results</li></ul>          | <ul><li>Vanishing gradient problem</li><li>Slow computations</li></ul>               |
|          |                                                                            |                                                                                      |