Building LLMs from the Ground Up: A 3-hour Coding Workshop

The author has prepared a 3-hour coding workshop presentation on Large Language Models (LLMs), covering their implementation, training, and usage. The workshop includes an introduction to LLMs, understanding LLM input data, coding an LLM architecture, pretraining, loading pretrained weights, instruction finetuning, benchmark evaluation, and evaluating conversational performance. The video presentation includes clickable chapter marks for easy navigation.

In addition to the video, the author provides several resources for further learning. These include a book and a GitHub repository on building an LLM from scratch, a separate GitHub repository with the workshop code, a Lightning Studio for the workshop, and a LitGPT GitHub repository. The author encourages viewers to explore these resources and enjoy the learning process.

Key takeaways:

The author has prepared a 3-hour coding workshop presentation on implementing, training, and using Large Language Models (LLMs).
The workshop covers various topics including introduction to LLMs, understanding LLM input data, coding an LLM architecture, pretraining, loading pretrained weights, instruction finetuning, and evaluating performance.
The video of the workshop has clickable chapter marks, allowing viewers to jump directly to topics of interest.
Additional resources such as a book on building an LLM from scratch, GitHub repositories with workshop code and other related resources are provided.

Building LLMs from the Ground Up: A 3-hour Coding Workshop

Key takeaways:

Comments (0)

Newsletter