{"id":17145,"date":"2026-08-22T22:43:38","date_gmt":"2026-08-23T05:43:38","guid":{"rendered":"https:\/\/mattfife.com\/?p=17145"},"modified":"2026-08-15T00:25:29","modified_gmt":"2026-08-15T07:25:29","slug":"hosting-local-models-with-unsloth-studio","status":"publish","type":"post","link":"https:\/\/mattfife.com\/?p=17145","title":{"rendered":"Hosting local models with unsloth studio"},"content":{"rendered":"\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<span class=\"embed-youtube\" style=\"text-align:center; display: block;\"><iframe loading=\"lazy\" class=\"youtube-player\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/XujmGCiiviM?version=3&#038;rel=1&#038;showsearch=0&#038;showinfo=1&#038;iv_load_policy=1&#038;fs=1&#038;hl=en-US&#038;autohide=2&#038;wmode=transparent\" allowfullscreen=\"true\" style=\"border:0;\" sandbox=\"allow-scripts allow-same-origin allow-popups allow-presentation allow-popups-to-escape-sandbox\"><\/iframe><\/span>\n<\/div><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">This video shows you how to load models in <a href=\"https:\/\/unsloth.ai\/docs\/new\/studio\" data-type=\"link\" data-id=\"https:\/\/unsloth.ai\/docs\/new\/studio\">Unsloth Studio<\/a> &#8211; and how to share them locally and across the network. Using Unsloth Studio, you can download and use models directly (much like ollama &#8211; but it appears to be much faster since it seems to use llama.cpp under the hood. You can also host these models either locally on the same machine, or set up an external port and connect Claude Code or other cli from another machine. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/unsloth.ai\/docs\/basics\/claude-code#windows\" data-type=\"link\" data-id=\"https:\/\/unsloth.ai\/docs\/basics\/claude-code#windows\">Hosting a model in Unsloth Studio so another machine can connect to it involves a few easy steps<\/a>. First, install Unsloth Studio on your host machine (with the big GPU and lots of memory). Then download and start the desired model in Unsloth Studio. After that, go to Settings -> API. Create a new API key. On your client system set up a batch file or environment variables then run claude from the command line:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>> set ANTHROPIC_BASE_URL=http:\/\/&lt;ip address of your server>:&lt;port number>\n> set ANTHROPIC_AUTH_TOKEN=&lt;your unsloth api key - usually: sk-unsloth-d5134......>\n> set ANTHROPIC_MODEL=Qwen3.8\n\n> claude --model Qwen3.8 %*<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>This video shows you how to load models in Unsloth Studio &#8211; and how to share them locally and across the network. Using Unsloth Studio, you can download and use models directly (much like ollama &#8211; but it appears to be much faster since it seems to use llama.cpp under the hood. You can also host these models either locally on the same machine, or set up an external port and connect Claude Code or other cli from another machine&#8230;.<\/p>\n<p class=\"read-more\"><a class=\"btn btn-default\" href=\"https:\/\/mattfife.com\/?p=17145\"> Read More<span class=\"screen-reader-text\">  Read More<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[28,9],"tags":[],"class_list":["post-17145","post","type-post","status-publish","format-standard","hentry","category-ai","category-cool"],"jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p4WECr-4sx","jetpack-related-posts":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/mattfife.com\/index.php?rest_route=\/wp\/v2\/posts\/17145","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mattfife.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mattfife.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mattfife.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/mattfife.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=17145"}],"version-history":[{"count":1,"href":"https:\/\/mattfife.com\/index.php?rest_route=\/wp\/v2\/posts\/17145\/revisions"}],"predecessor-version":[{"id":17146,"href":"https:\/\/mattfife.com\/index.php?rest_route=\/wp\/v2\/posts\/17145\/revisions\/17146"}],"wp:attachment":[{"href":"https:\/\/mattfife.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=17145"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mattfife.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=17145"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mattfife.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=17145"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}