Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

github.com

Post reply on HN
Print view View original

51–60 of 74 posts

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#51

Post by JackHopkins »

babyshake wrote: post #50

Not including "pass" in a function definition in Python makes the code not compilable, and if we're using VSCode, PyCharm, etc. our IDEs will complain about this whenever the code is viewed. Is this an intentional design decision?

The IDEs shouldn't complain if the function has a docstring (which all the MP functions should have as that's the instruction that is executed) and the @patch decorator, atleast the ones we have tried it with have liked the syntax in that sense so far. But adding a "pass" is also permissible if the IDE does complain

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#52

Post by JackHopkins »

ipsum2 wrote: post #44
JackHopkins wrote: post #39

Earlier quoted context omitted.

The python package (and repo) is called ‘monkeypatch.py’ for the avoidance of confusion.

Calling my library "listcomprehension.py" doesn't really avoid confusion. In fact, `pip install monkey-patch.py` looks downright odd.

Yeah I definitely agree on the latter point, it does look odd. PyMonkeyPatch?

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#53

Post by mnky9800n »

JackHopkins wrote: post #13
javidlakha wrote: post #12

Earlier quoted context omitted.

How does that work?

Currently we distill the general GPT-4 down to function specific GPT3.5 turbo model using pseudo-labelling. The input-output pairs from the aligned few-shot GPT-4 are saved and this dataset is used to finetune a function-specific GPT3.5 model. Then that finetuned GPT3.5 is switched as the primary model used to carry out the function, which results in multiple times lower costs as the need for few-shot examples is rem…

Does it ever just use the code that works and no longer makes calls to any LLM?

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#54

Post by zo1 »

JackHopkins wrote: post #32
JackHopkins wrote: post #30

Earlier quoted context omitted.

Monkeypatch (as I understand it) means to modify code at runtime. This library modifies functions at runtime to use an LLM as an execution target - I thought it was an apt (if admittedly cheeky) name! Appreciate the critique regardless.

The package is called ‘monkeypatch.py’ on GitHub and PyPi.

Many people here have said something about the naming, and you keep repeating that it's "monkeypatch.py" in response as if it fixes it. Maybe take the advice and just rename it to something while it's still early. You'll have a tough-enough time convincing people to use this novel/odd/unique concept without having the name confusion and bad-will from the community stemming from you appropriating a common term.

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#55

Post by JackHopkins »

mnky9800n wrote: post #53
JackHopkins wrote: post #13

Earlier quoted context omitted.

Currently we distill the general GPT-4 down to function specific GPT3.5 turbo model using pseudo-labelling. The input-output pairs from the aligned few-shot GPT-4 are saved and this dataset is used to finetune a function-specific GPT3.5 model. Then that finetuned GPT3.5 is switched as the primary model used to carry out the function, which results in multiple times lower costs as the need for few-shot examples is rem…

Does it ever just use the code that works and no longer makes calls to any LLM?

Great question! That is one of the ideas that we have on the roadmap and seems quite exciting to us. The general feasibility of switching the function execution over from a LLM to synthesised code depends on the specific use-case and if a deterministic program can solve the use-case well enough (or atleast as well as the SOTA LLMs can). But for all those cases where this could be done, the cost and latency of executing the program would become essentially 0

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#56

Post by JackHopkins »

zo1 wrote: post #54
JackHopkins wrote: post #32

Earlier quoted context omitted.

The package is called ‘monkeypatch.py’ on GitHub and PyPi.

Many people here have said something about the naming, and you keep repeating that it's "monkeypatch.py" in response as if it fixes it. Maybe take the advice and just rename it to something while it's still early. You'll have a tough-enough time convincing people to use this novel/odd/unique concept without having the name confusion and bad-will from the community stemming from you appropriating a common term.

Don't get me wrong, I do appreciate the criticism of the current naming! It does seem to create some unwanted friction of using or talking about the library, I was just trying to explain the thought process and ideate on top of it but we will have a second look regarding the name and how to make using and talking about the library as unconfusing as possible

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#57

Post by JackHopkins »

zo1 wrote: post #54
JackHopkins wrote: post #32

Earlier quoted context omitted.

The package is called ‘monkeypatch.py’ on GitHub and PyPi.

Many people here have said something about the naming, and you keep repeating that it's "monkeypatch.py" in response as if it fixes it. Maybe take the advice and just rename it to something while it's still early. You'll have a tough-enough time convincing people to use this novel/odd/unique concept without having the name confusion and bad-will from the community stemming from you appropriating a common term.

Thoughts on something like PyMonkeyPatch? GorillaPatch?

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#58

Post by sweetgiorni »

Slightly tangential: is it unfair/unreasonable to judge a project by its name? It's hard not to interpret this project's name as the result of poor judgement. Is that sufficient cause to write off the project entirely? That may seem a tad dramatic but I feel that it's a fairly strong signal for how little effort I need to put into evaluating it.

Re: Show HN: MonkeyPatch – Cheap, fast and predictable LLM functions in Python

#60

Post by JackHopkins »

sweetgiorni wrote: post #58

Slightly tangential: is it unfair/unreasonable to judge a project by its name? It's hard not to interpret this project's name as the result of poor judgement. Is that sufficient cause to write off the project entirely? That may seem a tad dramatic but I feel that it's a fairly strong signal for how little effort I need to put into evaluating it.

Do any other names jump out at you as preferable?
Post reply on HN