Ask HN: If we train an LLM with “data” instead of “language” tokens

The article discusses the concept of traditional machine learning (ML) and how it involves extensive feature extraction and engineering from a specific problem space before training computation is applied. The author suggests that this approach can yield effective pattern detection, prediction, and anomaly detection models.

The author then poses a hypothetical scenario where all types of data (ranging from weather metrics to kindergarten grades) are scraped and used to build a model with enough weights for this diverse data. The author questions what kind of use cases such a large data model might open up, implying the potential for new, unexplored applications of ML.

Key takeaways:

The article discusses the limitations of traditional ML, which requires extensive feature extraction and engineering before training.
It suggests the possibility of scraping all types of data, from weather metrics to web page clicks, to build a comprehensive model.
The author questions what kind of use cases such a large data model might open up.
There is an implication that this approach could lead to more effective pattern detection, prediction, and anomaly detection models.

Ask HN: If we train an LLM with “data” instead of “language” tokens

Key takeaways:

Comments (0)

Newsletter