Althttpd: Simple webserver in a single C file • Hacker News Board

Re: Althttpd: Simple webserver in a single C file

#251

Post by networked » Tue, Jun 08, 2021, 6:24 PM UTC

sgbeal wrote: post #213

networked wrote: post #166

Earlier quoted context omitted.

Fossil has enough wiki and theming features to power a customized website and to let you edit its contents in the browser. I've joked that Fossil SCM is secretly "Fossil CMS". My personal website, https://dbohdan.com/ , is powered by Fossil. A year ago I was shopping for a wiki engine, didn't love any I looked at, and realized I could try something I was already familiar with: Fossil. It did take a few hacks to make…

> The wiki lacks category ... Just FYI: we recently improved the internals to be able to add propagating tags to wiki pages[1], so it will eventually be possible to use those to categorize/group your wiki pages. What's missing now is UIs which can make use of that feature. The CLI tag command can make use of them, but that doesn't help your UI much. > ... and transclusion features For the wiki it seems unlikely to me…

This is excellent news! I hope Fossil can eventually obsolete both my tag script and the JavaScript TOC.

As for transclusions, I don't expect Fossil to implement them. While something like https://www.pmwiki.org/wiki/PmWiki/IncludeOtherPages would be cool, it seems probably out of scope for Fossil.

Re: Althttpd: Simple webserver in a single C file

#252

Post by emn13 » Tue, Jun 08, 2021, 6:39 PM UTC

bastardoperator wrote: post #207

emn13 wrote: post #103

Earlier quoted context omitted.

I never understood the point of squashing if you just want the short version, only read the merge commits; if you want the full details to figure out some bug or whatever, then yes, I want those fix typo commits most definitely, because as often as not, those are at fault. Squashing buys you next to nothing, and costs you the ability to dive into the history in greater detail. I suppose if your project is truly huge,…

Who squashes like this? I rebase all of my PR's not because I want to trash history, but because I want history to be meaningful. If I include all of my "whoops typo fix" and "pay respect to the linter gods" commits, I have made my default branch history much less readable. I would say what you're describing is a break down in CI/CD and code review. How is code that is that broken getting into your default branch in…

I certainly don't, but it's a standard option in git merge (and IIRC github supports it), so I'm pretty sure some teams do.

As to rebases to clean up history (and not just the PR itself)... personally, I don't think that's worth it. My experience with history like this is that it's relevant during review, and then around 95% of it is irrelevant - you may not know which 5 % are relevant beforehand, but it's always some small minority. It's worth cleaning up a PR for review, but not for posterity. And when it comes to review, I like commits like "pay respect to the linter gods" and the like, because they're easy to ignore, whereas if you touch code and reformat even slightly in one commit, it's often harder to skip the boring bits; to the point that I'll even intentionally commit poorly formatted code such that the diff is easy to read and then do the reformat later. Removing clear noise (as in code that changes back and forth and for no good reason) is of course nice, but it's easy to overdo; a few typo commits barely impact review-ability (imho), and rebases can and do introduce bugs - you must have encountered semantic merge conflicts before, and those are 10 times as bad with rebases, because they're generally silent (assuming you don't test each commit post-rebase), but leave the code in a really confusing situation, especially when people fix the final commit in the PR, but not the one were the semantic merge conflict was introduced, and laziness certainly encourages that.

It also depends on how proficient you are with merge conflicts and git chicanery. If you are; then the history is yours to reshape; but not everybody is, and then I'd rather review an honest history with some cruft, rather than a frankenstein history with odd seams and mismatched stuff in a commit that basically exists because "I kept on prodding git till it worked".

Re: Althttpd: Simple webserver in a single C file

#253

Post by rkeene2 » Tue, Jun 08, 2021, 6:42 PM UTC

e12e wrote: post #243

rkeene2 wrote: post #117

Earlier quoted context omitted.

filed [0] is written to be readable, and stateless, and runs from a chroot, and has no configuration file. It doesn't run from xinetd and it's multi-threaded, though. I wrote it because no other web server could serve files fast enough on my system (not lighttpd, not nginx, not Apache httpd, not thttpd) to keep movies from buffering. [0] https://filed.rkeene.org/

> I wrote it because no other web server could serve files fast enough on my system (not lighttpd, not nginx, not Apache httpd, not thttpd) to keep movies from buffering. Could you expand on that? What type of files, how many clients? I seem to recall plain apache2 from spinning rust streaming fine to vlc over lan - but last time I did that was before HD was much of a thing... Now I seem to stream 4k hdr over ZeroTie…

You might want to try filed !

This was for serving MPEG4-TS files with, IIRC, H.264 video and MPEG-III audio streams -- nothing fancy -- from a server running a container living on a disk attached via USB/1.1.

While USB/1.1 has enough bandwidth to stream the video, the other HTTP servers were too slow with Range requests, because they would do things like wait for logs to complete and open the file to serve (which is synchronous and requires walking the slow disk tree).

Re: Althttpd: Simple webserver in a single C file

#254

Post by emn13 » Tue, Jun 08, 2021, 6:43 PM UTC

coldtea wrote: post #195

plorkyeran wrote: post #175

Earlier quoted context omitted.

Storing every single version of the file which ever hit disk locally on my machine in the history would be the most accurate, yet no one seems to advocate for that. Even with immutable history, which versions go into the history is a choice the developer makes.

> Storing every single version of the file which ever hit disk locally on my machine in the history would be the most accurate, yet no one seems to advocate for that. You'd be surprised. It's only because we understand (and are used to) tool limitations (regarding storage, load, etc) that we don't advocate for that, not because some other way is philosophically better. I'd absolutely like to have "every single versio…

I mean, this isn't even really all that far-fetched, other systems do work like that, such as e.g. word's track changes or gdoc's history - or even a database's transaction log.

And while those histories are typically unreadable, it is possible to label (even retroactively) relevant moments in "history"; and in any case just because a consumer-level wordprocessor doesn't export the history in a practical way doesn't mean a technical VCS couldn't do better - it just means we can't take git-as-is or google-docs-as-is and add millions of tiny meaningless "commits" and hope it does anything useful.

Re: Althttpd: Simple webserver in a single C file

#255

Post by jiofih » Tue, Jun 08, 2021, 7:00 PM UTC

chousuke wrote: post #214

user-the-name wrote: post #174

Earlier quoted context omitted.

Why not? I do not see any reason to have a history at all for anything except to be able to go back to a specific version to track down a problem. Inaccurate history makes that less useful.

Typo commits and the usual iteration during development isn't "accurate history". Noise in commit logs provides negative value. Ideally, each commit should be something that you could submit as a stand-alone patch to a mailing list; whether it's a single commit that was perfect from the get-go or fifty that you had to re-order and rewrite twenty times does not matter at all; the final commit message should contain an…

Patch-perfect commits are an idealistic goal. The truth is that, as already mentioned, many of those typos and “dirty” commits can be the source of bugs that you’re looking for. Hiding them hides the history.

Re: Althttpd: Simple webserver in a single C file

#256

Post by xyzzy_plugh » Tue, Jun 08, 2021, 7:00 PM UTC

pdimitar wrote: post #216

xyzzy_plugh wrote: post #206

Earlier quoted context omitted.

We probably use the tools differently. I squash minor commits in my local branch, so I never want the review system squashing branches. If I put a branch for review, it is a series of patches which should be applied as-is. See the kernel mailing list and associated patch sets for the sort of thing I mean. I frequently use git rebase in interactive mode to rearrange and curate my commits to form whatever narrative I'm…

We definitely do use tools differently. Not looking to pick a fight here, mind you, but the Linux kernel is hardly a representative demonstration oh how to consume Git out there in the wild. The way you describe your usage it already seems you kinda sorta do your own squashed commits, only you want several of them get merged into the main branch, not just one. So you're still rewriting history a bit, no?

Yes! Absolutely I am rewriting history. I don't care about my messy history, and nor should anyone else. I routinely force-push to my branches because, again, it's my mess. Tools like mailing lists obviously handle this fine because every revision of my patches is published there, so who cares. Tools like Gerrit handle this well because it stores patch sets regardless of branch state -- as they are unrelated -- which feels like the right model. Tools like Github just suck at this in general and fall apart when you start force-pushing PR branches, but whatever.

The problem with squashing at review time is that it is incompatible with my model. I'd rather teach engineers to not push shitty "fix the thing" or "oops" commits that are ultimately useless for me, the reviewer or reader. Just use git commit --fixup and git rebase --autosquash as $DEITY intended and force push away. It's your world, you'll be deleting the branch after you land it anyways.

The tool works so well when used as intended: a distributed version control system. The centralized model adds so much pain and suffering.

Re: Althttpd: Simple webserver in a single C file

#257

Post by jmercouris » Tue, Jun 08, 2021, 7:04 PM UTC

SQLite wrote: post #159

ankurpatel wrote: post #150

Performance is really bad. This is good for running a small HTTP server on an embedded device but if plan is to use it for HTTP server to serve production web traffic performance is really bad. Below is report of running the server and hitting a minimal index.html page and hitting it with artillery. All virtual users finished Summary report @ 09:39:57(-0400) 2021-06-08 Scenarios launched: 33645 Scenarios completed: 2…

Indeed. And a Citation biz-jet is way faster, flies higher, goes further, and carries more passengers than a Carbon Cub. On the other hand, the Citation costs more, burns more gas, takes more maintenance, and is more complex to fly, and you should not try to land a Citation on a sandbar in a remote Alaskan river. Choose the right tool for the job. Changing the https://sqlite.org/ website to run off of Nginx or Apache…

It is not clear whether you would spend more time on administration with another webserver. I don't have experience with your webserver, but mine are 'set it' and 'forget it' affairs.

Re: Althttpd: Simple webserver in a single C file

#258

Post by geoffdunbar » Tue, Jun 08, 2021, 7:18 PM UTC

ktpsns wrote: post #2

Here is the actual single C-code file: https://sqlite.org/althttpd/file?name=althttpd.c Something I absolutely love about text based protocols such as HTTP/1 is how easy you can implement it in any virtually programming language. Sure, the implementation is not top-of-the-notch, but it just damned works, it is portable, it is understandable by humans. That's something what's got lost with HTTP/2 and HTTP/3, respectiv…

I love C, but it's pretty scary sometime. 5 minutes ago, "I wonder if I can find a potential memory overwrite in 5 minutes?"

Sure enough, the function StrAppend potentially overflows a size_t size (without checking), and then writes into memory could be past the end of the allocated buffer. Given 5 minutes, I didn't look thoroughly if this is actually exploitable, but it's definitely a red-flag for the code. Be careful out there! Hopefully I am missing something, or this is just a simple oversight, but I would carefully audit this code before using it.

Submitted a ticket through the Althttpd website.

static char StrAppend(char zPrior, const char zSep, const char zSrc){ char zDest; size_t size; size_t n0, n1, n2;

  if( zSrc==0 ) return 0;
  if( zPrior==0 ) return StrDup(zSrc);
  n0 = strlen(zPrior);
  n1 = strlen(zSep);
  n2 = strlen(zSrc);
  size = n0+n1+n2+1;
  zDest = (char*)SafeMalloc( size );
  memcpy(zDest, zPrior, n0);
  free(zPrior);
  memcpy(&zDest[n0],zSep,n1);
  memcpy(&zDest[n0+n1],zSrc,n2+1);
  return zDest;
}

Re: Althttpd: Simple webserver in a single C file

#259

Post by 40four » Tue, Jun 08, 2021, 7:29 PM UTC

ankurpatel wrote: post #150

Performance is really bad. This is good for running a small HTTP server on an embedded device but if plan is to use it for HTTP server to serve production web traffic performance is really bad. Below is report of running the server and hitting a minimal index.html page and hitting it with artillery. All virtual users finished Summary report @ 09:39:57(-0400) 2021-06-08 Scenarios launched: 33645 Scenarios completed: 2…

Fair enough, but when considering the reasons and decisions behind using this server from the developers, isn’t your point kind of moot?

It’s not optimized for high ‘performance’. It’s optimized for low resource usage, and the ability to reliably serve large amounts of requests on a small budget, right?

They state that the website is currently serving 500K requests & 50GB of bandwidth per day. Respectfully, this is quite the opposite of your ‘only good for small embedded devices’ claim.

I think this is very interesting, and I’m glad I know this exists now! Worth considering if you have the right type of use case.

Re: Althttpd: Simple webserver in a single C file

#260

Post by spacechild1 » Tue, Jun 08, 2021, 7:35 PM UTC

geoffdunbar wrote: post #258

ktpsns wrote: post #2

Here is the actual single C-code file: https://sqlite.org/althttpd/file?name=althttpd.c Something I absolutely love about text based protocols such as HTTP/1 is how easy you can implement it in any virtually programming language. Sure, the implementation is not top-of-the-notch, but it just damned works, it is portable, it is understandable by humans. That's something what's got lost with HTTP/2 and HTTP/3, respectiv…

I love C, but it's pretty scary sometime. 5 minutes ago, "I wonder if I can find a potential memory overwrite in 5 minutes?" Sure enough, the function StrAppend potentially overflows a size_t size (without checking), and then writes into memory could be past the end of the allocated buffer. Given 5 minutes, I didn't look thoroughly if this is actually exploitable, but it's definitely a red-flag for the code. Be caref…

> Sure enough, the function StrAppend potentially overflows a size_t size

How should this happen in practice? The three strings would have to be larger than the available address space...