Run Deepseek locally on a 24g GPU: Quantizing on our Giga Computing 6980P Xeon
With the help of some forum friends, Wendell now has the smallest model of Deepseek so far! Thank you Giga Computing, Kioxia, and Intel, for lending us all the goods to use for our testing!
Hugging Face Link https://huggingface.co/ubergarm/DeepSeek-R1-0528-GGUF
Forum Link https://forum.level1techs.com/t/deepseek-deep-dive-r1-at-home/225826/13…
The Chassis https://www.gigabyte.com/Enterprise/Rack-Server/R284-A91-AAL1
Kioxia https://americas.kioxia.com/en-us/business/ssd/data-center-ssd/cd8-r.ht…
0:00 Intro
2:17 Online vs Local
3:40 Hardware
9:36 Using the Machine and Results
18:00 Conclusion
https://www.youtube.com/@Level1LinksWithFriends
Go Subscribe to our New Links with Friends Channel!
Starting in July, we're going to post links with friends on its own channel. This is still Level1Techs, we’re just separating the content types so they can take their own algorithm journeys. We also plan on posting more videos with the trio on that page!