LLMs are built on top of the Transformer architecture, but before Transformers, the leading architecture for building NLP apps was Recurrent Neural Networks (RNN), such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) networks.