Sign up to save tools and stay up to date with the latest in AI
bg
bg
1

Large Language Models up to 4x Faster on RTX With TensorRT-LLM for Windows | NVIDIA Blog

Oct 18, 2023 - blogs.nvidia.com
NVIDIA has announced significant advancements in generative AI, with its GeForce RTX and NVIDIA RTX GPUs, which are equipped with dedicated AI processors called Tensor Cores, bringing generative AI to over 100 million Windows PCs and workstations. The company has released tools to help developers accelerate their large language models (LLMs), including scripts that optimize custom models with TensorRT-LLM, TensorRT-optimized open-source models, and a developer reference project. Furthermore, TensorRT acceleration is now available for Stable Diffusion in the popular Web UI by Automatic1111 distribution, speeding up the generative AI diffusion model by up to 2x over the previous fastest implementation.

In addition, NVIDIA has launched RTX Video Super Resolution (VSR) version 1.5, which improves the quality of streamed video content by reducing or eliminating artifacts caused by video compression. It also sharpens edges and details. The new version further improves visual quality with updated models, de-artifacts content played in its native resolution, and adds support for RTX GPUs based on the NVIDIA Turing architecture. RTX VSR 1.5 is available today for all RTX users in the latest Game Ready Driver and will be available in the upcoming NVIDIA Studio Driver, scheduled for early next month.

Key takeaways:

  • Generative AI on PC is getting up to 4x faster via TensorRT-LLM for Windows, an open-source library that accelerates inference performance for AI large language models.
  • NVIDIA has released tools to help developers accelerate their LLMs, including scripts that optimize custom models with TensorRT-LLM, TensorRT-optimized open-source models and a developer reference project.
  • TensorRT acceleration is now available for Stable Diffusion in the popular Web UI by Automatic1111 distribution, speeding up the generative AI diffusion model by up to 2x over the previous fastest implementation.
  • RTX Video Super Resolution (VSR) version 1.5 is available as part of today’s Game Ready Driver release, improving the quality of streamed video content by reducing or eliminating artifacts caused by video compression and sharpening edges and details.
View Full Article

Comments (0)

Be the first to comment!