If I recall correctly, there were some papers which suggested that LLMs favor LLM-generated passages over human written ones. I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful. It always picks its own :) I've also experienced that both Claude and Codex routinely include generated we…
Three sites made 215,128 “best software” pages for AI. Perplexity cites them
191–200 of 265 posts
Re: Three sites made 215,128 “best software” pages for AI. Perplexity cites them
#192Earlier quoted context omitted.
A bit different, but one thing I’ve seen is models repackaging Reddit slop. Like, it will do a search, find a Reddit thread somewhat related where someone in a comment casually mentioned incorrect information that any human would have dismissed. The model takes that as granted, but expands on it and present it as a well established fact, presented in a very plausible fashion. In general I don’t find models to be good…
Newspapers have been doing this for a long time, notably the Metro in London.
Re: Three sites made 215,128 “best software” pages for AI. Perplexity cites them
#193Earlier quoted context omitted.
A bit different, but one thing I’ve seen is models repackaging Reddit slop. Like, it will do a search, find a Reddit thread somewhat related where someone in a comment casually mentioned incorrect information that any human would have dismissed. The model takes that as granted, but expands on it and present it as a well established fact, presented in a very plausible fashion. In general I don’t find models to be good…
Newspapers have been doing this for a long time, notably the Metro in London.
Re: Three sites made 215,128 “best software” pages for AI. Perplexity cites them
#194If I recall correctly, there were some papers which suggested that LLMs favor LLM-generated passages over human written ones. I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful. It always picks its own :) I've also experienced that both Claude and Codex routinely include generated we…
I noticed this when I tried to get LLMs to play text adventures. Early on in my experiments, I wanted to give them hints when they got stuck at a puzzle. I did this by stopping the loop, injecting thoughts into the LLMs own persistent scratchpad (as if it had thought of that itself), and then starting the loop up again.[1] That way, the LLM would read what it had intended to remember from the previous turn including…
Re: Three sites made 215,128 “best software” pages for AI. Perplexity cites them
#195I do think models currently don't have enough source skepticism. If you look at agent traces when asked to compare two options to help inform a decision, many of the comparison pages cited in research are often hosted by one of the companies being compared; nearly all are AI-generated AEO plays. Not deeply considering the motive of published information is currently a glitch that can be exploited, but the window will…
Re: Three sites made 215,128 “best software” pages for AI. Perplexity cites them
#196Re: Three sites made 215,128 “best software” pages for AI. Perplexity cites them
#197Re: Three sites made 215,128 “best software” pages for AI. Perplexity cites them
#198Earlier quoted context omitted.
> If I recall correctly, there were some papers which suggested that LLMs favor LLM-generated passages over human written ones That makes sense. What an LLM does is output what the model thinks is the best set of tokens in response to a given input, so when you ask it to judge the best response to that input it is going to conclude that the best one is the one that must closely matches what it would output, which is…
Does an LLM have any idea of what "best" is? I think it doesn't, and just predicts the range of most statistically likely next tokens based on its training data, and picks one of those.
"Best" here isn't being used to imply a conscious decision, but at each stage which token has the best score coming out of the model, so the overall best is the sequence of those best tokens. When judging another output it is essentially running the numbers the same way.
It is a bit more complicated than that as the output tokens become part of the context for the next choice, but I think that simplified way of thinking about it holds water.
Re: Three sites made 215,128 “best software” pages for AI. Perplexity cites them
#199Earlier quoted context omitted.
> You're asking basically to ignore bugs and correctness. Not ignore correctness, just bugs that will be caught by tooling. > Can it be a useful comparison? IME, yes. LLMs in an agent-loop are trivially able to write spaghetti code that will never do an off-by-one error or something else that is easily caught by tooling, which is not something humans can do. Judging code on whether it has bugs easily caught by toolin…
> just bugs that will be caught by tooling Bugs are bugs, if the instruction is "ignore bugs except for those that can be caught by you" then the instruction is basically "ignore bugs". And it implies "ignore correctness" because when program is incorrect we usually refer to it as a... you guessed it, "bug".
No, the instruction is "ignore bugs that can be caught by tooling", unless you are seriously complaining that missing a semi-colon should register the developer as a junior?
> And it implies "ignore correctness" because when program is incorrect we usually refer to it as a... you guessed it, "bug".
This ("Bugs are bugs" sentiment) is digressing from my original point, but I have some time to engage, so...
Now, this is a take (one that I used to hold, once upon a time), but it is incorrect.
There is no definite "correct" and "incorrect" states in non-trivial applications, because every non-trivial application has unspecified requirements that are understood by most parties involved (customer and developer) whilst not being written down anywhere.
For example, the "save file" specification for a cross-platform application does not specify the allowed/disallowed characters in a filename. The understanding by both the client and the dev is that the filename can be whatever the underlying OS and filesystem allows it to be but this is not written in the spec!
Is this a bug?
If the user saves a file to a filename with some odd characters in the name, then moves it to portable storage that truncates the filename/removes emojis/whatever, then attempts to upload it back to the system, the system can refuse because the metadata inside the file does not match the filename.
User is going to report it as a bug! The developer is going to reject it as a bug (there is no error in the code).
Sure, contrived example, but Line of Business applications have thousands of these unspecified but common-sense requirements baked in.
I'm looking at my employers triaging system right now, and even though this is a high-level business app (written mostly in SQL and C#), there is one category for bug (e.g. specific field not saved on form submission - defect in code), and another for deficiency (e.g. form field 'total' does not subtract non-tax costs - ambiguity in spec). The reason this is important is because clients aren't billed for bug fixes, but they are billed for disambiguating a spec + writing code.
Both those things were reported by the client as a "bug".
The reality is that we aren't dealing with what is "implied", only with what is there. There are defects in code and defects in specs. The code ones are the easy ones.
Re: Three sites made 215,128 “best software” pages for AI. Perplexity cites them
#200Earlier quoted context omitted.
Ah nice, the site uses nextjs/vercel and is hosted by AWS ...
Why is this significant? Many sites are setup like that.