Its a great time to follow the trends and research in natural language processing (NLP). Every other day researchers publish new model architectures and techniques which try to improve the current state-of-the-art (SOTA) models. However, even though there is an abundance of different architectures and methods, almost all of these are based on 2 main ideas - Attention Models and the Transformer Architecture.
