Sign up to save tools and stay up to date with the latest in AI
bg
bg
1

NVIDIA Chat With RTX

Feb 14, 2024 - nvidia.com
Chat with RTX is a demo application that allows users to personalize a GPT large language model (LLM) connected to their own content such as documents, notes, videos, or other data. The app leverages retrieval-augmented generation (RAG), TensorRT-LLM, and RTX acceleration to provide contextually relevant answers from a custom chatbot. The system runs locally on a Windows RTX PC or workstation, ensuring fast and secure results.

The application supports various file formats and can load content from a specified folder or a YouTube playlist. It requires a Windows 11 platform with a NVIDIA GeForce™ RTX 30 or 40 Series GPU or NVIDIA RTX™ Ampere or Ada Generation GPU with at least 8GB of VRAM, 16GB or greater RAM, and driver 535.11 or later. Developers can use the TensorRT-LLM RAG developer reference project from GitHub to develop and deploy their own RAG-based applications for RTX, accelerated by TensorRT-LLM.

Key takeaways:

  • Chat With RTX is a demo app that allows users to personalize a GPT large language model connected to their own content for quick, contextually relevant answers.
  • The app supports various file formats and can even load transcriptions from a YouTube playlist.
  • Developers can use the TensorRT-LLM RAG developer reference project from GitHub to develop and deploy their own RAG-based applications for RTX.
  • The app requires a Windows 11 platform with a NVIDIA GeForce™ RTX 30 or 40 Series GPU or NVIDIA RTX™ Ampere or Ada Generation GPU with at least 8GB of VRAM, 16GB or greater RAM, and a driver of 535.11 or later.
View Full Article

Comments (0)

Be the first to comment!