91–100 of 264 posts
> Please implement for me. Please write a high quality, general purpose solution. If the task is unreasonable or infeasible, or if any of the tests are incorrect, please tell me. Do not hard code any test cases. Please tell me if the problem is unreasonable instead of hard coding test cases! I have pretty good success with just telling agents "don't cheat"
[flagged]
[flagged]
Obviously this should not be taken as a representative case and I will caveat that the problem was not trivial ... basically dealing with a race condition I was stuck with for the past 2 days. The TLDR is that all models failed to pinpoint and solve the problem including Claude 4. The file that I was working with was not even that big (433 lines of code). I managed to solve the problem myself. This should be taken as…
Earlier quoted context omitted.
They gave a bullet point in that intro which I disagree with: "The only way to make GenAI applications secure is through vulnerability scanning and guardrail protections." I still don't see guardrails and scanning as effective ways to prevent malicious attackers. They can't get to 100% effective, at which point a sufficiently motivated attacker is going to find a way through. I'm hoping someone implements a version o…
I only half understand CaMeL. Couldn't the prompt injection just happen at the stage where the P-LLM devises the plan for the other LLM such that it creates a different, malicious plan? Or is it more about the user then having to confirm/verify certain actions and what is essentially a "permission system" for what the LLM can do? My immediate thought is that that may be circumvented in a way where the user unknowingl…
After Google io they had to come up with something even if it is underwhelming
Exactly. It's getting to the point where the quality of the top AI labs are either not ground-breaking (except Google Gemini Diffusion) and labs are rushing to announce their underwhelming models. Llama as an example. Now in the next 6 months, you'll see all the AI labs moving to diffusion models and keep boasting around their speed. People seem to forget that Google Deepmind can do more than just "LLMs".
Given the cited stats here and elsewhere as well as in everyday experience, does anyone else feel that this model isn’t significantly different, at least to justify the full version increment? The one statistic mentioned in this overview where they observed a 67% drop seems like it could easily be reduced simply by editing 3.7’s system prompt. What are folks’ theories on the version increment? Is the architecture sig…
Right now I'm swapping between Gemini and Opus depending on the task. Gemini's 1M token context window is really unbeatable.
But the quality of what Opus 4 produces is really good.
edit: forgot to mention that this is all for Rust based work on InfluxDB 3, a fairly large and complex codebase. YMMV
Earlier quoted context omitted.
I'm noticing much more flattery ("Wow! That's so smart!") and I don't like it
Agreed. It was immediately obvious comparing answers to a few prompts between 3.7 and 4, and it sabotages any of its output. If you're being answered "You absolutely nailed it!" and the likes to everything, regardless of their merit and after telling it not to do that , you simply cannot rely on its "judgement" for anything of value. It may pass the "literal shit on a stick" test, but it's closer to the average ChatG…
Earlier quoted context omitted.
Perhaps they want to include online discussions/commentaries about their paper in the training data without including the paper itself
Most online discussion doesn't contain the entire text. You can pick almost any sentence from such a document and it'll be completely unique on the internet. I was thinking it might be related to the difficulty of building a search engine over the huge training sets, but if you don't care about scaling or query performance it shouldn't be too hard to set one up internally that's good enough for the job. Even sharded…