Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

github.com

Post reply on HN
Print view View original

61–70 of 74 posts

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#61

Post by JackHopkins »

lamroger wrote: post #59

The guardrails are cool! I think more details of where the data goes and when it goes from few-shot to fine-tune will be helpful.

Good to know, we'll make it more clear in the docs! To answer regarding these 2 areas,

1) The data for finetuning currently is saved on disk for low latency reading and writing. Both test statements and datapoints from the function execution are saved to the dataset. We also are aware that saving to disk is not the best option and limits many use-cases so we're currently working on creating persistence layers to allow communication with S3 / Redis / Cloudflare as the external data storage.

2) Currently starting the fine-tuning job happens after the dataset has at least 200 datapoints from GPT-4 executions and align statements. Once the finetuning is completed, the execution model for the function is automatically switched to the finetuned GPT 3.5 turbo model. Whenever the finetuned model breaks the constraints, the teacher (GPT4) is called upon to fix the datapoint and this datapoint will be saved back to the dataset for future iterative finetuning and improvements. We are also working on adding in ways for the user to include a "test-set" which could be used to evaluate if the finetuned model achieves the required performance before switching it as the primary executor of the function

Hope this makes it more clear, if you have any additional questions, let me know!

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#62

Post by OJFord »

JackHopkins wrote: post #25
OJFord wrote: post #23

Why 'Monkeypatch', when it's for Python, where that has an established and as far as I can tell (?) completely irrelevant meaning?

(As I understand it) monkey-patch means to modify code in runtime. I thought the naming was relevant because by adding ‘@monkey.patch’ to an unimplemented function, this library gives it an implementation at runtime.

I guess. I think of it more like overriding an existing thing with another for a given scope/time/test, whereas this is providing an implementation for a thing which exists only as a stub.

Maybe it 'counts', I'm not meaning to be picky about the term, my point is more like even if this works by monkeypatching, I wouldn't personally use the term for the product which does something else by those means, if that makes sense? MonkeyLLM or MonkAIpatch or something, sure.

The other comment with a 'list comprehension' example puts it well I think.

(Seeing as you asked for name suggestions elsewhere, I think I prefer the 'stub' theme: Stubby, StubAI, Stubmonkey if you like monkeys/wanted an easier logo, something like that.)

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#63

Post by pbronez »

JackHopkins wrote: post #47
vqbd wrote: post #43

Earlier quoted context omitted.

The problem isn't accidentally downloading the wrong package. How am I supposed to talk about this package? I regularly use monkeypatching (the existing meaning) to prototype out concepts. "I monkey patch dot pied this function"? Cause that doesn't seem to be the name used everywhere. Why not name it something like Gorillapatch? Gorillas are stronger than monkeys as a slogan or whatever. The core issue is how am I su…

This is a solid point, at the end of the day creating confusion with the name wasn't the goal in any way as there already are 100s of (often overlapping) terms floating around in this scene. I appreciate the critique and we'll have a think over this. If you have any other naming ideas, would love to hear them!

Maybe keep the “Monkey” and work around that?

MonkeySeeDo —> it sees what you’re doing and does it better

CutMonkey —> it’s a monkey patch that’s cut weight to lean fighting trim

TypeMonkey —> it uses types to intelligently monkey patch your code

MonkeyZipper —> monkey patches that compresses your code

MonkeyModels —> monkey patches your models

Learning Monkey —> learns how to improve your Large Models

Branching out from there…

SimianStudy —> it’s a monkey patch that learns

PyStill —> it distills your Python functions

AutoSqueeze —> automatically squeezes your AI code into efficient implications

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#64

Post by CyberDildonics »

JackHopkins wrote: post #32
JackHopkins wrote: post #30

Earlier quoted context omitted.

Monkeypatch (as I understand it) means to modify code at runtime. This library modifies functions at runtime to use an LLM as an execution target - I thought it was an apt (if admittedly cheeky) name! Appreciate the critique regardless.

The package is called ‘monkeypatch.py’ on GitHub and PyPi.

Monkey patching isn't what your library is supposed to do, it's just a mechanic that it uses to get there. This would be like me making a scripting language and calling it "virtual machine". Then when people ask "why is a scripting language called virtual machine" I would say "it uses a virtual machine and the file is called virtualmachine.py".

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#65

Post by jstarfish »

JackHopkins wrote: post #57
zo1 wrote: post #54

Earlier quoted context omitted.

Many people here have said something about the naming, and you keep repeating that it's "monkeypatch.py" in response as if it fixes it. Maybe take the advice and just rename it to something while it's still early. You'll have a tough-enough time convincing people to use this novel/odd/unique concept without having the name confusion and bad-will from the community stemming from you appropriating a common term.

Thoughts on something like PyMonkeyPatch? GorillaPatch?

Honestly dude, just drop the simian theme and pick a different, possibly-endangered animal instead.

Monkeys have nothing to do with your project and were a meme over a decade ago, which makes your brand-new project look dated.

Apes are associated with being heavyweight and freakishly strong and have been long-associated with racial slurs in America. You're only ever one degree removed from coming out with "chimpout.py" or "statutoryape.py" or something else that'll get you cancelled for unintended racism.

Your tool seems like it's meant to be reliable, used for work, and possibly elegant in its code. Consider the name of a work animal for their efficiency or birds for lightweight, graceful maneuverability.

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#66

Post by JackHopkins »

OJFord wrote: post #62
JackHopkins wrote: post #25

Earlier quoted context omitted.

(As I understand it) monkey-patch means to modify code in runtime. I thought the naming was relevant because by adding ‘@monkey.patch’ to an unimplemented function, this library gives it an implementation at runtime.

I guess. I think of it more like overriding an existing thing with another for a given scope/time/test, whereas this is providing an implementation for a thing which exists only as a stub. Maybe it 'counts', I'm not meaning to be picky about the term, my point is more like even if this works by monkeypatching, I wouldn't personally use the term for the product which does something else by those means, if that makes s…

I do get the point and the difference from the classical monkey-patching. I like the stub ideas though!

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#67

Post by ASalazarMX »

mitthrowaway2 wrote: post #46

There seems to be a lot of (justified) concern about the name. Maybe call it LLMonkeyPatch?

LLMonkeyPatch is one of the best suggestions here, only adds two letters that fence nicely the scope of the monkey patching, and looks playful.

Other suggestions, like PyMonkeyPatch, leave the reader to guess what is being monkey patched.

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#68

Post by ipsum2 »

JackHopkins wrote: post #52
ipsum2 wrote: post #44

Earlier quoted context omitted.

Calling my library "listcomprehension.py" doesn't really avoid confusion. In fact, `pip install monkey-patch.py` looks downright odd.

Yeah I definitely agree on the latter point, it does look odd. PyMonkeyPatch?

I don't think you grasp the point.

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#69

Post by JackHopkins »

jstarfish wrote: post #65
JackHopkins wrote: post #57

Earlier quoted context omitted.

Thoughts on something like PyMonkeyPatch? GorillaPatch?

Honestly dude, just drop the simian theme and pick a different, possibly-endangered animal instead. Monkeys have nothing to do with your project and were a meme over a decade ago, which makes your brand-new project look dated. Apes are associated with being heavyweight and freakishly strong and have been long-associated with racial slurs in America. You're only ever one degree removed from coming out with "chimpout.p…

Yeah this is fair. I’m not attached to a simian theme if we’re ditching specific association to monkey-patching something. Or indeed, a ‘patching’ theme for that matter.

A new name is definitely in order. I will think about it over the weekend.

Thanks for the feedback, I appreciate it.

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#70

Post by JackHopkins »

ipsum2 wrote: post #68
JackHopkins wrote: post #52

Earlier quoted context omitted.

Yeah I definitely agree on the latter point, it does look odd. PyMonkeyPatch?

I don't think you grasp the point.

I understand the point. I would ideally like an association with monkey-patching something as that is relevant to the behaviour of the package. However, not so similar that it shadows the technique of monkey-patching!
Post reply on HN