Show HN: Personal Replit Ghostwriter

#1

Post by MLH6ft1 » Sat, Jul 01, 2023, 3:11 PM UTC

What it is: A website to run a Ghostwriter like code generation assistant for free!

Backstory: Hi All, I recently stumbled upon GGML 4 bit quantized LLMs and the fact that small version of these GGML models (i.e., upto 7b) can run smoothly on a CPU! The GGML 4 bit quantized version of replit’s codeInstruct-3b model, only requires 2GB of RAM! So I quickly tested it and hosted the model on a free HuggingFace Space and it works!

Let me know your thoughts on it!

Show HN: Personal Replit Ghostwriter
personal-replit-ghostwriter.sixftone-mlh.repl.co

Re: Show HN: Personal Replit Ghostwriter

#2

Post by schemescape » Sun, Jul 02, 2023, 3:10 AM UTC

I’m not familiar with code generating models in general, so I don’t know how much to expect, but I tried one moderately simple task and one very simple task, and both times it produced something that looked like code but didn’t make any sense and used the wrong keywords.

It’s neat, and probably beyond what I would have expected a few years ago, but it doesn’t seem useful in practice.

It’s very cool that it works on CPU quickly and doesn’t take too much RAM, though! Thanks for letting me try it!

Re: Show HN: Personal Replit Ghostwriter

#3

Post by MLH6ft1 » Sun, Jul 02, 2023, 10:29 AM UTC

schemescape wrote: post #2

I’m not familiar with code generating models in general, so I don’t know how much to expect, but I tried one moderately simple task and one very simple task, and both times it produced something that looked like code but didn’t make any sense and used the wrong keywords. It’s neat, and probably beyond what I would have expected a few years ago, but it doesn’t seem useful in practice. It’s very cool that it works on C…

I think that's because Replit's model was trained to generate small snippets of code or parts of your code, which you can glue together later on, rather generating the entire code in the first shot itself.

But I was more surprised by the fact we can run such a model on a CPU!