What is the best setup for using local LLM ?
I have tried setting LM-Studio up to run the server and provide a local model.
I can connect fine from Luxlab and simple prompts work. Slow but it works.
But when the prompt gets a bit more complicated for example building something then after some time the chat display the error "Fetch failed" and it stops. But I can see LM Studio is still processing the prompt and generating tokens. The response just never reach Laxlb.
I have tried 2 different setup.
First setup is my day to day workstation a Mac Studio M1 Ultra with 64 GB ram. I run both Luxlab and LM Studio on this.
Second setup I tried was my Macbook Air M5 with 24 GB ram and using a smaller model. And connecting over network. I stopped all apps I was not using except for LM Studio.
Same happened on both configurations. After a while a "Fetch failed" error occurs.
My plan is to get a Mac Mini next year with max available memory. So some information / tips on how to setup LM Studio for best results with Luxlab would be great.
I use LM Studio to have the GUI. At least for now, with a dedicated Mac mini I could probably love without the gui since it would be running without display connected anyway.
