91–100 of 170 posts
The article says there aren't too many useless comments but the code has: // Get the Origin header from the request const origin = request.headers.get('Origin');
Earlier quoted context omitted.
If you trust everything the LLM tells you, and you learn from code, then yes the same exact risks apply. But this is not how you use (or should use) LLMs when you’re learning a topic. Instead you should use high quality sources, then ask the LLM to summarize them for you to start with (NotebookLM does this very well for instance, but so can others). Then you ask it to build you a study plan, with quizzes and exercise…
> Instead you should use high quality sources, then ask the LLM to summarize them for you to start with (NotebookLM does this very well for instance, but so can others). How do you determine if the LLM accurately reflects what the high-quality source contains, if you haven't read the source? When learning from humans, we put trust on them to teach us based on a web-of-trust. How do you determine the level of trust wi…
Bonus: the high quality source is going to be mostly AI written anyway
I just finished writing a Kafka consumer to migrate data with heavy AI help. This was basically best case a scenario for AI. It’s throw away greenfield code in a language I know pretty well (go) but haven’t used daily in a decade. For complicated reasons the whole database is coming through on 1 topic, so I’m doing some fairly complicated parallelization to squeeze out enough performance. I’d say overall the AI was c…
Earlier quoted context omitted.
LLMs make learning new material easier than ever. I use them a lot and I am learning new things at an insane pace in different domains. The maximalists and skeptics both are confusing the debate by setting up this straw man that people will be delegating to LLMs blindly. The idea that someone clueless about OAuth should develop an OAuth lib with LLM support without learning a lot about the topic is... Just wrong. Don…
On the flip side, I wanted to see what common 8 layer PCB stackups were yesterday. ChatGPT wasn't giving me an answer that really made sense. After googling a bit, I realized almost all of the top results were AI generated, and also had very little in the way of real experience or advice. It was extremely frustrating.
Before AI generated results, the first page of Google was SEO-optimised crap blogs. The internet has been hard to search for a while.
Earlier quoted context omitted.
LLMs make learning new material easier than ever. I use them a lot and I am learning new things at an insane pace in different domains. The maximalists and skeptics both are confusing the debate by setting up this straw man that people will be delegating to LLMs blindly. The idea that someone clueless about OAuth should develop an OAuth lib with LLM support without learning a lot about the topic is... Just wrong. Don…
On the flip side, I wanted to see what common 8 layer PCB stackups were yesterday. ChatGPT wasn't giving me an answer that really made sense. After googling a bit, I realized almost all of the top results were AI generated, and also had very little in the way of real experience or advice. It was extremely frustrating.
The article says there aren't too many useless comments but the code has: // Get the Origin header from the request const origin = request.headers.get('Origin');
The article says there aren't too many useless comments but the code has: // Get the Origin header from the request const origin = request.headers.get('Origin');
Those kinds of comments are a big LLM giveaway, I always remove them, not to hide that an LLM was used, but because they add nothing.
> I’m also an expert in OAuth
I'll admin I think Neil is significantly more of an expert than me, so I'm delighted he took a pass at reviewing the code! :)
I'd like to respond to a couple of the points though.
> The first thing that stuck out for me was what I like to call “YOLO CORS”, and is not that unusual to see: setting CORS headers that effectively disable the same origin policy almost entirely for all origins:
I am aware that "YOLO CORS" is a common novice mistake, but that is not what is happening here. These CORS settings were carefully considered.
We disable the CORS headers specifically for the OAuth API (token exchange, client registration) endpoints and for the API endpoints that are protected by OAuth bearer tokens.
This is valid because none of these endpoints are authorized by browser credentials (e.g. cookies). The purpose of CORS is to make sure that a malicious website cannot exercise your credentials against some other website by sending a request to it and expecting the browser to add your cookies to that request. These endpoints, however, do not use browser credentials for authentication.
Or to put in another way, the endpoints which have open CORS headers are either control endpoints which are intentionally open to the world, or they are API endpoints which are protected by an OAuth bearer token. Bearer tokens must be added explicitly by the client; the browser never adds one automatically. So, in order to receive a bearer token, the client must have been explicitly authorized by the user to access the service. CORS isn't protecting anything in this case; it's just getting in the way.
(Another purpose of CORS is to protect confidentiality of resources which are not available on the public internet. For example, you might have web servers on your local network which lack any authorization, or you might unwisely use a server which authorizes you based on IP address. Again, this is not a concern here since the endpoints in question don't provide anything interesting unless the user has explicitly authorized the client.)
Aside: Long ago I was actually involved in an argument with the CORS spec authors, arguing that the whole spec should be thrown away and replaced with something that explicitly recognizes bearer tokens as the right way to do any cross-origin communications. It is almost never safe to open CORS on endpoints that use browser credentials for auth, but it is almost always safe to open it on endpoints that use bearer tokens. If we'd just recognized and embraced that all along I think it would have saved a lot of confusion and frustration. Oh well.
> A more serious bug is that the code that generates token IDs is not sound: it generates biased output.
I disagree that this is a "serious" bug. The tokens clearly have enough entropy in them to be secure (and the author admits this). Yes, they could pack more entry per byte. I noticed this when reviewing the code, but at the time decided:
1. It's secure as-is, just not maximally efficient. 2. We can change the algorithm freely in the future. There is not backwards-compatibility concern.
So, I punted.
Though if I'd known this code was going to get 100x more review than anything I've ever written before, I probably would have fixed it... :)
> according to the commit history, there were 21 commits directly to main on the first day from one developer, no sign of any code review at all
Please note that the timestamps at the beginning of the commit history as shown on GitHub are misleading because of a history rewrite that I performed later on to remove some files that didn't really belong in the repo. GitHub appears to show the date of the rebase whereas `git log` shows the date of actual authorship (where these commits are spread over several days starting Feb 27).
> I had a brief look at the encryption implementation for the token store. I mostly like the design! It’s quite smart.
Thank you! I'm quite proud of this design. (Of course, the AI would never have come up with it itself, but it was pretty decent and filling in the details based on my explicit instructions.)
Part of me this "written by LLM" has been a way to get attention on the codebase and plenty of free reviews by domain expert skeptics, among the other goals (pushing AI efficiency to investors, experimenting, etc).
I didn't think of that, though. I didn't have an agenda here, I just put the note in the readme about it being LLM-generated only because I thought it was interesting.
Really interesting breakdown. What jumped out to me wasn’t just the bugs (CORS wide open, incorrect Basic auth, weak token randomness), but how much the human devs seemed to lean on Claude’s output even when it was clearly offbase. That “implicit grant for public clients” bit is wild; it’s deprecated in OAuth 2.1, and Claude just tossed it in like it was fine, and then it stuck.