Fine-Tuning Tutorial: Falcon-7b LLM to a General Purpose Chatbot

In the past, most models underwent training using the supervised method, where input features and corresponding labels were fed. In contrast, LLMs take a different route by undergoing unsupervised learning.

Falcon, a decoder-only autoregressive model, boasts 40 billion parameters and was trained using a substantial dataset of 1 trillion tokens. This intricate training process spanned two months and involved the use of 384 GPUs hosted on AWS.