Live data from Hacker News

You can read my blog posts using curl

mahdi.blog

1–10 of 73 posts

Re: You can read my blog posts using curl

#5
Why not use user agent detection (and maybe have a header/footer that says "This page has been formatted for readability based on your User Agent: curl").

I'm guessing the blog is made by a static site generator, so the above is harder than it seems. I suppose one could add a reverse proxy that redirects to /raw/$PAGE when it sees "curl".

Re: You can read my blog posts using curl

#6
Fun idea.

If the site is not a static one, you could check the request's User Agent server-side, and return the raw version directly (or redirect to /foo/raw) if the UA contains 'curl' or 'wget'.

If the site is static and you are able & willing to change your vhost config, you could detect the UA too, and redirect to /foo/raw.

Just a few ideas. This is a fun little project you've got here. Well done.

Re: You can read my blog posts using curl

#9
post #5

Why not use user agent detection (and maybe have a header/footer that says "This page has been formatted for readability based on your User Agent: curl"). I'm guessing the blog is made by a static site generator, so the above is harder than it seems. I suppose one could add a reverse proxy that redirects to /raw/$PAGE when it sees "curl".

User-agent specific stuff is pretty hacky in general. I at least would avoid it unless I've tried everything else first.

Re: You can read my blog posts using curl

#10
In these comments people have suggested to check for the User-Agent, but wouldn't it make more sense to check if the Accept header mentions text/html? I realize that curl sends "Accept: /", but if you wanted to see a page in plain text, you would have to pass -H "Accept: text/plain". I think that uses HTTP much more like it was intended.
Post reply on HN