Tried a local LLM for coding and it hallucinated an entire fake API library - is that just how it is right now?

So I spent last weekend messing with Llama 3.1 8B through Ollama, trying to get it to help me write some Python for a side project at home in Chicago. I asked it to use a specific library for PDF parsing, and it gave me this whole block of code with functions that sounded legit. None of it worked. Took me an hour to realize the library doesn't even have half those methods. My buddy swears by using cloud models like Claude for code, but I'm trying to keep my data local. Is this just the tradeoff with smaller models right now, or am I missing something in how I'm prompting it? Anyone else run into this with local LLMs and coding?

2 comments

2 Comments

wade76728d ago

Welcome to the club, the fake library is a rite of passage at this point.

brian_jackson28d ago

Man it's frustrating when that happens. You spend all that time debugging something that never existed in the first place. I've been there with the smaller models too, they're just not reliable enough yet for anything specific like that.