{"id":2121,"date":"2025-02-04T22:56:00","date_gmt":"2025-02-04T14:56:00","guid":{"rendered":"https:\/\/tongwing.woon.sg\/blog\/?p=2121"},"modified":"2025-02-05T17:05:34","modified_gmt":"2025-02-05T09:05:34","slug":"how-to-run-deepseek-r1-locally","status":"publish","type":"post","link":"https:\/\/tongwing.woon.sg\/blog\/how-to-run-deepseek-r1-locally\/","title":{"rendered":"How to run DeepSeek-R1 locally"},"content":{"rendered":"\n<p>This is about running the <a href=\"https:\/\/github.com\/deepseek-ai\/DeepSeek-R1\">full model<\/a> that has <a href=\"https:\/\/github.com\/deepseek-ai\/DeepSeek-R1\/blob\/main\/figures\/benchmark.jpg\">OpenAI o1-level performance<\/a>, not those distilled models <a href=\"https:\/\/www.youtube.com\/watch?v=o1sN1lB76EA\">running on Raspberry Pi<\/a>. A distilled model is like a student learning from a master. It may be able to do some things that the master do, but it is not at the same level.<\/p>\n\n\n\n<p>Someone wrote a post on X\/Twitter on how to achieve this in ~USD6000. This is the <a href=\"https:\/\/x.com\/carrigmat\/status\/1884244369907278106\">original post<\/a>, but if you don&#8217;t have an X\/Twitter account, you can also use <a href=\"https:\/\/xcancel.com\/carrigmat\/status\/1884244369907278106\">this link<\/a> to view the whole thread. There are also others who wrote about the same setup <a href=\"https:\/\/www.pcgamer.com\/hardware\/graphics-cards\/today-i-learned-i-can-run-my-very-own-deepseek-r1-chatbot-on-just-usd6-000-of-pc-hardware-and-no-megabucks-nvidia-gpus-required\/\">here<\/a> and <a href=\"https:\/\/damiandabrowski.medium.com\/day-30-of-100-days-agentic-engineer-challenge-6k-hardware-to-run-deepseek-r1-locally-7f96c0149e79\">here<\/a> so I won&#8217;t repeat it here.<\/p>\n\n\n\n<p>This is a video capture of the model&#8217;s output in realtime:<\/p>\n\n\n\n<figure class=\"wp-block-video\"><video controls src=\"https:\/\/tongwing.woon.sg\/blog\/wp-content\/uploads\/2025\/02\/rh83qoEcShpxtljL.mp4\"><\/video><\/figure>\n\n\n\n<p>This setup is impressive for a few reasons:<\/p>\n\n\n\n<ul>\n<li>CPU-Only Processing \u2013 No GPUs are involved<\/li>\n\n\n\n<li>Decent Token Generation Speed \u2013 6-8 tokens per second<\/li>\n\n\n\n<li>Energy Efficient \u2013 Operates on &lt;400W of power<\/li>\n\n\n\n<li>Cost-Effective  \u2013 Cost ~USD6,000, a fraction of the estimated $100,000+ required for a GPU-based setup<\/li>\n<\/ul>\n\n\n\n<p>The setup is not exactly cheap, but it is within research or hobbyist-level budget. It will be very interesting to see how much more optimization can be done to make it even more affordable without compromising on quality.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This is about running the full model that has OpenAI o1-level performance, not those distilled models running on Raspberry Pi. A distilled model is like a student learning from a master. It may be able to do some things that the master do, but it is not at the same level. Someone wrote a post [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[34],"tags":[],"_links":{"self":[{"href":"https:\/\/tongwing.woon.sg\/blog\/wp-json\/wp\/v2\/posts\/2121"}],"collection":[{"href":"https:\/\/tongwing.woon.sg\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tongwing.woon.sg\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tongwing.woon.sg\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tongwing.woon.sg\/blog\/wp-json\/wp\/v2\/comments?post=2121"}],"version-history":[{"count":8,"href":"https:\/\/tongwing.woon.sg\/blog\/wp-json\/wp\/v2\/posts\/2121\/revisions"}],"predecessor-version":[{"id":2134,"href":"https:\/\/tongwing.woon.sg\/blog\/wp-json\/wp\/v2\/posts\/2121\/revisions\/2134"}],"wp:attachment":[{"href":"https:\/\/tongwing.woon.sg\/blog\/wp-json\/wp\/v2\/media?parent=2121"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tongwing.woon.sg\/blog\/wp-json\/wp\/v2\/categories?post=2121"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tongwing.woon.sg\/blog\/wp-json\/wp\/v2\/tags?post=2121"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}