161–170 of 255 posts
Suggestion: publish a live demo of the "needle playground". It's small enough that it should be pretty cheap to run this on a little VPS somewhere!
thanks, yeah, the problem is just handling scale, we don't have the infra ready to go, but anyone can do that. Its easy for people to run on their laptops straight up. Will try the VPS route.
Can it summarize text it fetches? Come to think of it, this could be a nice model to have as the first pass in a more complex agent system where Needle hands of the results of a tool call to a larger model. I will defiantly play around with this!
> I will defiantly play around with this! Are you Calvin or Hobbes?
Are you worried about Google's response to this? Google reportedly reacts to distillation attempts " with real-time proactive defenses that can degrade student model performance ". So if they detected you, they could have intentionally fed you a dumber but plausible variant of Gemini: https://cloud.google.com/blog/topics/threat-intelligence/dis... But also, this model is small and just focusing on the tool use. In te…
Are you worried about Google's response to this? Google reportedly reacts to distillation attempts " with real-time proactive defenses that can degrade student model performance ". So if they detected you, they could have intentionally fed you a dumber but plausible variant of Gemini: https://cloud.google.com/blog/topics/threat-intelligence/dis... But also, this model is small and just focusing on the tool use. In te…
You could run Gemma models locally to distill them. Or any other model with tool use.
Earlier quoted context omitted.
Many people tease that they will, and start... but then kinda stop. But mostly just been building my own bespoke thing on my own bespoke platform, and kinda running out of steam because I need to make $$ instead.
Ah, sad, but not surprising. The hard part of getting a game going is assembling and sustaining a community.
How could you use this for composability? I.e. chaining together multiple tools. For example web_search → summarize_url → send_email
Query: get the weather for san francisco and email the result to test@test.com
Result: [{"name":"get_weather","arguments":{"location":"san francisco"}},{"name":"send_email","arguments":{"to":"test@test.com","subject":"San Francisco","body":"Please find the weather attached."}}]
Earlier quoted context omitted.
works for me: input: i need to contact my boss i will be late. output: [{"name":"send_email","arguments":{"to":"boss@company.com","subject":"Running late","body":"I will be late for the meeting."}}] it did have the send_email tool on the left hand side though
Boss: what meeting are you talking about..? In the ideal scenario, the boss also uses Needle, which checks emails and schedule a late meeting with whoever sent that email. Needle on the other side receives the invite for a late meeting, and notify OP he's got a 67% chance of getting fired today.
Do you have any examples or data on the discriminatory power of the model for tool use? The examples are things like "What is the weather in San Francisco", where you are only passed a tool like tools='[{"name":"get_weather","parameters":{"location":"string"}}]', I had a thing[1] over 10 years ago that could handle this kind of problem using SPARQL and knowledge graphs. My question is how effective is it at handling…
Thanks to a Huggingface linked below, I tested it and im not impressed. prmopt: i need to contact my boss i will be late. Result: 20mins [{"name":"set_timer","arguments":{"time_human":"20 minutes"}}]. It didnt use the email tool and i tried 2-3 different ways of asking it.
Output: [{"name":"send_email","arguments":{"to":"bigboss69420@corporatepersonhood.net","subject":"upcoming_meetings","body":"I'll be 15 minutes late"}},{"name":"send_email","arguments":{"to":"bigboss69420@corporatepersonhood.net","subject":"time","body":"I'll be 15 minutes late"}},{"name":"send_email","arguments":{"to":"bigboss69420@corporatepersonhood.net","subject":"time","body":"I'll be 15 minutes late"}}]
Context definitely helps. But yeah the quality of it doesn't seem to be too high. To be fair it makes you realise that not only is parameter extraction required, but also content generation (email body). Also debouncing the 3 tool calls.
Maybe under very specific circumstances/very tight harness this sort of model would be useful?