Ask HN: Cheapest hardware to run Llama 2 70B

The user "gorbypark" suggests that a 192gb Mac Studio could potentially run an unquantized 70B more cost-effectively than a multi-GPU setup comprised of Nvidia cards. However, they note that they haven't done the calculations to confirm this. They also propose that if electricity costs over a certain period are considered, the Mac could turn out to be even cheaper.

Key takeaways

A 192gb Mac Studio is suggested to be capable of running an unquantized 70B.
The Mac Studio might be a cheaper alternative to a multi gpu setup made up of nvidia cards.
The cost-effectiveness of the Mac Studio may increase when considering electricity costs over a certain time period.
The user, however, has not done the actual math to confirm these assumptions.

Ask HN: Cheapest hardware to run Llama 2 70B

Key takeaways

Discussion (0)