161–170 of 336 posts
The buildings weren't minecraft style in either case. They weren't formed on a voxel grid and the textures weren't 16x16, but rather a rectangle or at least stretched to one. Also buildings typically are not just built as a cuboid.
I was using gemini 2.5 pro yesterday and it does seem decent. I still think claude 3.5 is better at following instruction then the new 3.7 model which just goes ham messing stuff up. Really disappointed by Cursor and the Claude CLI tool, for me they create more problems then fix. I cant figure out how to use them on any of my projects with out them ruining the project and creating terrible tech debt. I really like th…
Whenever I read about LLMs or try to use them, I feel like I am asleep in a dream where two contradicting things can be true at the same time. On one hand, you have people claiming "AI" can now do SWE tasks which take humans 30 minutes or 2 hours and the time doubles every X months so by Y year, SW development will be completely automated. On the other hand, you have people saying exactly what you are saying. Usually…
From my use case, the Gemini 2.5 is terrible. I have a complex Cython code in a single file (1500 lines) for a Sequence Labeling. Claude and o3 are very good in improving this code and following the commands. The Gemini always try to do unrelated changes. For example, I asked, separately, for small changes such as remove this unused function, or cache the arrays indexes. Every time it completely refactored the code a…
set temperature to 0.4 or lower.
Earlier quoted context omitted.
Google has had almost perfect recall in the needle in the haystack test since 1.5[1], achieving close to 100% over the entire context window. I can't provide a link benchmarking 2.5 Pro in particular, but this has been a solved problem with Google models so I assume the same is true with their new model. [1] https://cloud.google.com/blog/products/ai-machine-learning/t...
Has those results been reproduced elsewhere with other benchmarks than what Google seems to use? Hard to trust their own benchmarks at this point, and Im not home at the moment so cant try it myself either.
There are some more advanced tests where it's far less impressive. Just a couple of days ago Adobe released one such test- https://github.com/adobe-research/NoLiMa
Here is a real coding problem that I might be willing to make a cash-prize contest for. We'd need to nail down some rules. I'd be shocked if any LLM can do this: https://github.com/solvespace/solvespace/issues/1414 Make a GTK 4 version of Solvespace. We have a single C++ file for each platform - Windows, Mac, and Linux-GTK3. There is also a QT version on an unmerged branch for reference. The GTK3 file is under 2KLOC.…
The ridiculous amount of data required to get here hints that there is something wrong in my opinion.
I'm not sure if we're totally on the same page, but I understand where you're coming from here. Everyone keeps talking about how transformational these models are, but when push comes to shove, the cynicism isn't out of fear or panic, its disappointment over and over and over. Like, if we had an army of virtual programmers fixing serious problems for open source projects, I'd be more excited about the possibilities than worried about the fact that I just lost my job. Honest to God. But the thing is, if that really were happening, we'd see it. And it wouldn't have to be forced and exaggerated all the time, it would be plainly obvious, like the way AI art has absolutely flooded the Internet... except I don't give a damn if code is soulless as long as it's good, so it would possibly be more welcome. (The only issue is that it most likely actually suck when that happens, and rather just be functional enough to get away with, but I like to try to be optimistic once in a while.)
You really make me want to try this, though. Imagine if it worked!
Someone will probably beat me to it if it can be done, though.
Earlier quoted context omitted.
Convert the GTK 3 and GTK 4 API documentation into a single `.txt` file each. Upload one of your platform-specific C++ file's source, along with the doc `.txt` into your LLM of choice. Either ask it for a conversion function-by-function, or separate it some other way logically such that the output doesn't get truncated. Would be surprised if this didn't work, to be honest.
Do you really need to provide the docs? I would have imagined that those docs are included in their training sets. There is even a guide on how to migrate from GTK3 to GTK4, so this seems to be a low-hanging fruit job for an LLM iff they are okay for coding.
Earlier quoted context omitted.
This is the smoothest tom sawyer move I've ever seen IRL, I wonder how many people are now grinding out your GTK4 port with our favorite LLM/system to see if it can. It'll be interesting to see if anyone gets something working with current-gen LLMs. UPDATE: naive (just fed it your description verbatim) cline + claude 3.7 was a total wipeout. It looked like it was making progress, then freaked out, deleted 3/4 of its…
>> This is the smoothest tom sawyer move I've ever seen IRL That made me laugh. True, but not really the motivation. I honestly don't think LLMs can code significant real-world things yet and I'm not sure how else to prove that since they can code some interesting things. All the talk about putting programmers out of work has me calling BS but also thinking "show me". This task seems like a good combination of simple…
In my experience it seems like it depends on what they’ve been trained on
They can do some pretty amazing stuff in python, but fail even at the most basic things in arm64 assembly
These models have probably not seen a lot of GTK3/4 code and maybe not even a single example of porting between the two versions
I wonder if finetuning could help with that
I'd like to see an honest attempt by someone to use one of these SOTA models to code an entire non-trivial app. Not a "vibe coding" flappy bird clone or minimal ioS app (call API to count calories in photo), but something real - say 10K LOC type of complexity, using best practices to give the AI all the context and guidance necessary. I'm not expecting the AI to replace the programmer - just to be a useful productivi…
Tech stack is nothing fancy/rare but not the usual ReactJS slop either - it's C# with OpenGL.
I can't comment about the best practices though because my codebase follows none of them.
Yes, the user has to know enough to guide the AI when it's failing. So it can't exactly replace the programmer as it is now.
It really can't do niche stuff however - like SIMD. Maybe it would be better if I compiled a cheatsheet of .NET SIMD snippets and howtos because this stuff isn't really on the internet in a coherent form at all. So it's highly unlikely that it was trained on that.
Here is a real coding problem that I might be willing to make a cash-prize contest for. We'd need to nail down some rules. I'd be shocked if any LLM can do this: https://github.com/solvespace/solvespace/issues/1414 Make a GTK 4 version of Solvespace. We have a single C++ file for each platform - Windows, Mac, and Linux-GTK3. There is also a QT version on an unmerged branch for reference. The GTK3 file is under 2KLOC.…
Gemini takes parts of code and just writes (same as before) even when i ask it to provide full code. which for me is deal breaker
Yeah - I tried Gemini 2.0 Flash a few week ago, and while the model itself is decent this was very annoying. It'd generate full source if I complained, but then next change would go back to "same as before" ... over and over ...