/bant/ - International/Random » Thread #20774795

[33 / 9 / 1]

8KiB, 225x225, 1708014784319144.jpg

View Same Google ImgOps iqdb SauceNAO

Anonyomus ID:ti8MqvLe Mon 15 Jul 2024 16:01:02 No.20774795 View View Reply Original Report

Quoted By: >>20774823 >>20774977 >>20776220

I wish I knew how to run AI chat bots locally on my PC

Anonymous

Anonymous ID:MH0112a6 Mon 15 Jul 2024 16:06:10 No.20774823 Report

Quoted By: >>20774977

>>20774795
yeah im too retarded to figure it out.

Normie !GzTNQPcpf6

Normie !GzTNQPcpf6 ID:jIuSO8VN Mon 15 Jul 2024 16:23:25 No.20774977 Report

Quoted By: >>20775007 >>20775054

>>20774795
>>20774823
https://github.com/Jaseunda/local-ai

Download, install via the executable for your OS and have fun. Don't ever say I didn't do anything for you retards.

Anonymous

Anonymous ID:cnnPeune Mon 15 Jul 2024 16:29:13 No.20775007 Report

Quoted By: >>20775015 >>20776268

>>20774977
you're ugly and a terrible person kill yourself lol

Normie !GzTNQPcpf6

View Same Google ImgOps iqdb SauceNAO 1715618364436523.jpg, 61KiB, 675x832

Normie !GzTNQPcpf6 ID:jIuSO8VN Mon 15 Jul 2024 16:30:43 No.20775015 Report

Quoted By:

>>20775007
Feel better now that you've said that?

Anonyomus

View Same Google ImgOps iqdb SauceNAO 1690690746251748.png, 269KiB, 493x293

Anonyomus ID:ti8MqvLe Mon 15 Jul 2024 16:41:39 No.20775054 Report

Quoted By: >>20775495 >>20776170

>>20774977
Thank you but I don't know if I trust this. It doesn't seem like whatever /g/ uses..

Anonymous

Anonymous ID:rY51U/2q Mon 15 Jul 2024 17:56:04 No.20775495 Report

Quoted By: >>20775564 >>20776220

>>20775054
Download koboldcpp and download a gguf of your choice. Start with LLaMA 3 8B instruct tunes.

Anonyomus

Anonyomus ID:ti8MqvLe Mon 15 Jul 2024 18:05:52 No.20775564 Report

Quoted By:

>>20775495
I don't know what any of that is..

Anonymous

Anonymous ID:3b5RkP9k Mon 15 Jul 2024 18:50:46 No.20775823 Report

Quoted By: >>20776201

nigga got the entire world of knowledge at his fingertips and cant do this
have you even searched google or youtube or whatever

Normie !GzTNQPcpf6

Normie !GzTNQPcpf6 ID:jIuSO8VN Mon 15 Jul 2024 19:16:57 No.20776170 Report

Quoted By: >>20776201

>>20775054
Yeah because /g/ actually knows what they're doing whereas you admitted you're too retarded so I provided the retard solution.
It's Github and you can look through the file structure but you're too retarded to know what you're looking at.
Retard.

Anonyomus

Anonyomus ID:ti8MqvLe Mon 15 Jul 2024 19:22:50 No.20776201 Report

Quoted By: >>20776206

>>20775823
Yeah bro. I did a few months ago but the video tutorials were all like 30 minutes long and the rentry guides where like 3000 paragraphs for something that should ideally be a click and run. Anway, I did get sillytavern to run since I found a quick easy tutorial, but still gotta find out how to run it locally since it doesn't seem to come with that feature.
>>20776170
U_U everyone is so mean to me.

Normie !GzTNQPcpf6

Normie !GzTNQPcpf6 ID:jIuSO8VN Mon 15 Jul 2024 19:23:57 No.20776206 Report

Quoted By: >>20776217

>>20776201
Retard.

Anonyomus

View Same Google ImgOps iqdb SauceNAO 1715744840710238.png, 1MiB, 1920x1080

Anonyomus ID:ti8MqvLe Mon 15 Jul 2024 19:25:48 No.20776217 Report

Quoted By:

>>20776206
Said the guy namefagging on an anonymous website.

Anonymous

Anonymous ID:CXPGJ02x Mon 15 Jul 2024 19:26:48 No.20776220 Report

Quoted By: >>20776304

>>20774795
for waifu/larpshit, use silly tavern. otherwise, download a model of your choice like what >>20775495 suggested and use oogabooga (really) https://github.com/oobabooga/text-generation-webui

Anonymous

View Same Google ImgOps iqdb SauceNAO 1720995219407918.gif, 715KiB, 498x451

Anonymous ID:dAVjp5FP Mon 15 Jul 2024 19:37:22 No.20776268 Report

Quoted By:

>>20775007

He's ugly and probably a terrible person

But at least he helped your lazy ass so in my eyes he's worth much more than you ever will be Amerimutt

Anonyomus

Anonyomus ID:ti8MqvLe Mon 15 Jul 2024 19:46:20 No.20776304 Report

Quoted By: >>20776423

>>20776220
I got silly tavern working, but it's only "kobold hoard" or whatever which doesn't seem to run locally. I used to use character.ai but I don't wanna give my data to those companies anymore which is why I wanna use something local.

Anonyomus

Anonyomus ID:ti8MqvLe Mon 15 Jul 2024 19:56:07 No.20776343 Report

Quoted By: >>20776443 >>20776463

Man. According to this video I gotta install like 5 different things just to get local kobald. Why do they make things like this?

Anonymous

Anonymous ID:yfK3z8vZ Mon 15 Jul 2024 20:14:15 No.20776423 Report

Quoted By:

>>20776304
https://github.com/LostRuins/koboldcpp
Once you download that, point Silly tavern to 5001 or whatever the port is.

Anonymous

Anonymous ID:rY51U/2q Mon 15 Jul 2024 20:19:20 No.20776443 Report

Quoted By: >>20776502

>>20776343
It's four steps, anon-kun.
1) Grab latest kobold: https://github.com/LostRuins/koboldcpp/releases
2) Grab checkpoint from huggingface: https://huggingface.co/mradermacher/L3-8B-Lunaris-v1-i1-GGUF/tree/main (Start with q4_k_m)
3) Run kcpp and load model.
4) Paste http://localhost:5001/ into server url inside SillyTavern.

Anonymous

Anonymous ID:CXPGJ02x Mon 15 Jul 2024 20:25:31 No.20776463 Report

Quoted By:

>>20776343
i will remain in the thread to provide emotional support until you figure it out

Anonyomus

View Same Google ImgOps iqdb SauceNAO 1696505622709333.png, 312KiB, 1205x767

Anonyomus ID:ti8MqvLe Mon 15 Jul 2024 20:34:30 No.20776502 Report

Quoted By: >>20776541

>>20776443
Why they telling me to install this shitto then?

Anonymous

Anonymous ID:CXPGJ02x Mon 15 Jul 2024 20:42:52 No.20776541 Report

Quoted By: >>20776585

>>20776502
doesn't hurt. me personally, i have a 4070-Super NVIDIA card and never had to bother with that to run huge models. can always remove/add it later.

Anonyomus

Anonyomus ID:ti8MqvLe Mon 15 Jul 2024 20:51:19 No.20776585 Report

Quoted By: >>20776594

>>20776541
What about this "git" thing?

Anonymous

Anonymous ID:CXPGJ02x Mon 15 Jul 2024 20:53:19 No.20776594 Report

Quoted By: >>20776597 >>20776679

>>20776585
for cloning models from hugging face in this case. so like, when you download it, tick the option to add it to right-click/context menu.. then navigate to the directory you want to drop your models. right click, (show more options?), "open git bash here", click,
type in console git lfs install , then git clone https://huggingface.co/mradermacher/L3-8B-Lunaris-v1-i1-GGUF and once it's down you'll have the model itself

Anonymous

Anonymous ID:CXPGJ02x Mon 15 Jul 2024 20:55:09 No.20776597 Report

Quoted By:

>>20776594
i haven't slept in a very long time but hopefully you get the idea. install git, open the git console to directory, git lfs install, git clone $(url)

Anonymous

View Same Google ImgOps iqdb SauceNAO 1714189444520731.png, 7KiB, 722x68

Anonymous ID:CXPGJ02x Mon 15 Jul 2024 20:58:07 No.20776602 Report

Quoted By:

alternatively, go to https://huggingface.co/mradermacher/L3-8B-Lunaris-v1-i1-GGUF/tree/main and click the download button on one of them . git clone is useful for when there's json and other shit that needs to go with it and isn't all in one neat package. git cloning from the one provided will just download all models in that folder but that's what git is for

Anonyomus

Anonyomus ID:ti8MqvLe Mon 15 Jul 2024 21:14:40 No.20776679 Report

Quoted By: >>20776687

>>20776594
I'm trying to follow this video ( https://www.youtube.com/watch?v=Y3RCir5GiIY&list=WL&index=7 ) but I don't know if it's up to date. Anyway thanks for this. I'll see if it works when everything finishes installing.

Anonymous

Anonymous ID:CXPGJ02x Mon 15 Jul 2024 21:17:24 No.20776687 Report

Quoted By: >>20776690 >>20777110

>>20776679
sure, just don't git clone that particular repository anon linked above. i didn't notice it had like 20 different models in it so you should just download one from huggingface's website through conventional means (pressing the download button on one of the cptk files) and forgetting about git until you encounter a repo that would benefit from it

Anonymous

Anonymous ID:CXPGJ02x Mon 15 Jul 2024 21:18:13 No.20776690 Report

Quoted By:

>>20776687
gguf files*

Anonyomus

View Same Google ImgOps iqdb SauceNAO 1694071688627324.png, 750KiB, 1456x1508

Anonyomus ID:ti8MqvLe Mon 15 Jul 2024 22:52:28 No.20777110 Report

Quoted By: >>20777134

>>20776687
welp I'm fucked I guess since that video I'm following wants me to install this 4bit patch thing but the link to it is dead. I tried skipping the step, but then I ran into this other issue where I don't have the file it wants me to rename.

Anonymous

View Same Google ImgOps iqdb SauceNAO 1699504510519724.png, 14KiB, 1577x47

Anonymous ID:CXPGJ02x Mon 15 Jul 2024 22:58:06 No.20777134 Report

Quoted By:

>>20777110
https://huggingface.co/mayaeary/pygmalion-6b-4bit-128g/tree/main
if that's the particular one you meant

Anonymous

Anonymous ID:CXPGJ02x Mon 15 Jul 2024 23:06:40 No.20777174 Report

Quoted By: >>20777670

don't use kobold patch; just rename file and kobold should support and infer

Anonyomus

Anonyomus ID:ti8MqvLe Tue 16 Jul 2024 00:17:13 No.20777670 Report

Quoted By:

>>20777174
Thanks for your help, but I have to go to bed now. I'll try attempting it again tomorrow.

Capcode	All Only User Posts Only Verified Posts Only Moderator Posts Only Manager Posts Only Admin Posts Only Developer Posts Only Founder Posts
Show Posts	All Only With Images Only Without Images Only Spoiler Images Only Non-Spoiler Images
Deleted Posts	All Only Deleted Posts Only Non-Deleted Posts
Ghost Posts	All Only Ghost Posts Only Non-Ghost Posts
Post Type	All Only Sticky Threads Only Opening Posts Only Reply Posts
Results	All Grouped By Threads
Order	Latest Posts First Oldest Posts First

On these archives

Your latest searches