Sign up to save tools and stay up to date with the latest in AI
bg
bg
1

GitHub - distantmagic/structured: Extracts structured data from unstructured input. Programming language agnostic. Uses llama.cpp

May 12, 2024 - github.com
The article discusses a project that began as a Go conversion of an existing library but has evolved into a more general-purpose library. The library features a language-agnostic HTTP server that can be set up in front of llama.cpp, offering the same features with a Go-like API. It is model agnostic, meaning it can map data from any JSON schema to any Go struct or plain JSON. The library is primarily focused on llama.cpp, but support for other vendor APIs may be added in the future.

The article also provides a detailed guide on how to use the library, including how to start a server, issue requests, and extract structured data from strings. It also explains how to map the extracted result onto an arbitrary struct. The library's API may change over time as new features are implemented. The article concludes with a mention of a related project, a llama.cpp load balancer, supervisor, and request queue, and an invitation to join the community on Discord.

Key takeaways:

  • The project is a Go conversion of a general-purpose library, featuring a language-agnostic HTTP server that can be set up in front of llama.cpp.
  • It is model agnostic and can map data from arbitrary JSON schema to arbitrary Go struct or plain JSON.
  • The HTTP API allows you to start a server and point it to your local llama.cpp instance, issue requests, and receive responses in JSON matching your schema.
  • Programmatic usage of the API includes initializing the mapper, extracting structured data from string, and mapping extracted result onto an arbitrary struct.
View Full Article

Comments (0)

Be the first to comment!