Using local LLM

Tommy Jensen
Tera Guru

What is the best setup for using local LLM ?

I have tried setting LM-Studio up to run the server and provide a local model.

I can connect fine from Luxlab and simple prompts work. Slow but it works.

But when the prompt gets a bit more complicated for example  building something then after some time the chat display the error "Fetch failed" and it stops. But I can see  LM Studio is still processing the prompt and generating tokens. The response just never reach Laxlb.

I have tried 2 different setup.

First setup is my day to day workstation a Mac Studio M1 Ultra with 64 GB ram. I run both Luxlab and LM Studio on this.

Second setup I tried was my Macbook Air M5 with 24 GB ram and using a smaller model. And connecting over network. I stopped all apps I was not using except for LM Studio.

Same happened on both configurations. After a while a "Fetch failed" error occurs.

My plan is to get a Mac Mini next year with max available memory. So some information / tips on how to setup LM Studio for best results with Luxlab would be great.
I use LM Studio to have the GUI. At least for now, with a dedicated Mac mini I could probably love without the gui since it would be running without display connected anyway.


2 REPLIES 2

PranjulM
ServiceNow Employee

Hi Tommy,

Thanks for the detailed write-up. From what you describe, LM Studio keeps generating while
Lux Lab stops waiting, so I'd like to understand what the app sees at that moment. A few
questions:

1. Timing: How long after you send the prompt does "Fetch failed" appear? Is it roughly the
same duration every time, or does it vary?
2. Output before failure: Do you see any text or tool activity in the chat before the error,
or is it silent until it fails?
3. Setup: Which model and context length are you using in LM Studio, and how is Lux Lab
pointed at it (base URL, localhost or IP)?

Answers to even the first two will help us tell whether the app is hitting a timeout or the
connection is being closed some other way.

One more thing: local models can give different results from run to run, so it's hard to
compare setups with different prompts. Could you run the exact same prompt on both machines
(the Mac Studio and the MacBook Air) and note for each:

- whether it completes or fails
- how long it ran before failing

Please use a prompt that triggered the error for you before, and keep it identical on both.
That will show whether the behavior is consistent or depends on the machine and model.

Thanks,
Pranjul Mishra

Thanks for rersponsding. I did the tests again.

I used same model on both setups: gemma 4 12b 256k context

And I used the same prompt "add a simple todo app to this project"

On the mac studio (127.0.0.1) the fetch failure came after about 10 minutes and there where some output before but it was silent some time before failing. LM Studio continued to generate tokens after the failure. The large screenshot show both LM Studio and Luxlab.

On the Macbook (192...) the first attempt failed after 5 minutes before giving the first response. I then tried to tell it to continue but nothing happened. And LM Studio was not generating tokens when the failure occurred. Then I tried again in a new chat. This time I got an in initial response and then after approving it failed after about 8 minutes. After second time LM Studio in this case also still generated tokens. The small screenshot is from this attempt.

So failure after 10 and 8 minutes from the initial prompt. And for both some activity then some silence before failing.

luxlab_local_studio_1.pngluxlab_local_macbook_1.png