GitHub - zemlyansky/gpt-tfjs: GPT in TensorFlow.js
Jul 30, 2023 - github.com
The article provides information about a minimal implementation of the GPT architecture in TensorFlow.js, which is more or less similar to the minGPT implementation in PyTorch. It includes an example of how to use the GPTLMHeadModel from the 'gpt-tfjs' package. The example demonstrates how to configure the model, train it using a TensorFlow dataset, and generate new tokens. It also mentions that the 'trainDataset' used in the example is a TensorFlow dataset, and provides a link to a sample dataset.
The article also mentions that the testing script relies on minGPT to generate test data and requires Pytorch to be installed. Additionally, it provides a link to a demo that showcases how to load GPT-2 weights from Hugginface and run inference in the browser.
Key takeaways:
The article provides a minimal implementation of GPT architecture in TensorFlow.js, following the minGPT implementation in PyTorch.
It includes an example of how to use the GPTLMHeadModel, including how to train the model and generate new tokens.
The training data for the model should be a TensorFlow dataset, with a link provided to an example dataset.
There is a demo available that shows GPT-2 weights from Hugginface being loaded and run in the browser.
