Live data from Hacker News

“I Could Rewrite Curl”

daniel.haxx.se

151–160 of 193 posts

Re: “I Could Rewrite Curl”

#151
This happens everywhere. The devil is in details. If we use an application and the surface we see looks simple, we assume it must be simple to build :) Then we start to see all the complexity behind.

Shameless plug: This kind of posts was the reason I wrote about it recently too [1]

1 - https://world.hey.com/joaoqalves/i-could-build-this-during-t...

Re: “I Could Rewrite Curl”

#152
post #30

Let's be honest here. Curl to 98% of people is http/1 requests with some post params and maybe json body, with some custom headers. Most language standard libs (which might be using libcurl) can facilitate writing that relatively quickly. And probably with a more 'modern' CLI. Curl though, is a very wide breadth project. It currently supports the following protocols: DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP,…

Thanks for sharing this insight, I hadn't realized it was capable of all those protocols.

I fall firmly into "those 98%" you mention.

Dlang's stdlib (only) implementation for making network requests is quite literally a direct "curl" wrapper:

https://dlang.org/phobos/std_net_curl.html

You've changed on my mind on this being an odd design decision.

Baking in libcurl for the user (or optionally allowing them to dynamically link) if they want networking was maybe the most sane/pragmatic choice you could have made for a new language.

Re: “I Could Rewrite Curl”

#153
post #106

Earlier quoted context omitted.

A question that arises out of this is, should the 90% use cases be handled by a small, simple tool, or by the tip of the iceberg of a large, complex tool such as curl? I can see the advantages of standardizing on a complex, powerful tool for simple use cases. For one thing, it may be the only way to standardize: simple versions are too easy to write, and therefore you get dozens of competitors, none of whom are popul…

Sometimes yes, sometimes no. Counter-example: Jenkins. It does what you ask of it, its base install is "naked" and only contains the minimum functionality in the core. Everything then becomes a plugin. Git. GitHub. Branch for multi-branch pipelines. Credentials management. And on and on and on. Now you have stay on top of maintaining the plugins in addition to the core. Also, many plugins require other plugins so jus…

I can definitely see your point, having experienced the same thing with plugins for SBT, the Scala build tool. I didn't really consider the case of a small core with a multitude of plugins as a twist on the small, simple tool. I think you're right that a plugin architecture lets a thousand flowers bloom, but you don't get long-term stability, because people move on to other tools and stop maintaining the plugins they wrote.

For example, VSCode plugins are great because VSCode is thriving, and Emacs packages are a crapshoot because many of the programmers who wrote them have moved on. Eventually VSCode plugins will be like Emacs packages.

Re: “I Could Rewrite Curl”

#154
post #49

Earlier quoted context omitted.

Plenty of applications use TCP and UDP, or even custom protocols on top of raw IP. Not everything needs to come through the browser.

Micro-services, for example, are pretty much all HTTP + JSON. The only place for HTTP is the browser and purely for historical reasons. Using it for anything server-server is a waste of network and CPU.

Thankfully it is going away with gRPC and friends.

Re: “I Could Rewrite Curl”

#155
post #68

Earlier quoted context omitted.

True. I'm kind of surprised (again) by Daniel after 20 years of maintaining curl being so salty about "people not appreciating the achievement". Yes, for overwhelming majority of users curl in 2021 is just a simple http/https request client. And, yes, in 2021 that could be written over a weekend with a better CLI, mainly because most of the languages already have all important stuff implemented in a [standard] librar…

That was honestly my first thought. Of course it's easy to rewrite curl nowadays. It can't be anything more complicated than slapping a command line interface on top of libcurl...

At first I laughed... then I thought about the sheer number of command line options that you can find in the man pages. After thinking about that, even writing the command line interface would be a non-trivial undertaking.

Re: “I Could Rewrite Curl”

#156
post #8
post #5

I can write cURL in 1 line of bash: alias mycurl='curl' /s

most people underestimate huge effort under simple to use products but integrated to our lives heavily. * redis - I can do it, listen socket, serve from hashmap * uber - how come this app is so huge, I can make it in 2 weekends (google map + api + database) * cURL - easy, use my fancy library (it already implements HTTP stack) and rewrite it ....

What's interesting is that the next Uber may be created by somebody naive about the effort it will take. Being naive can be an advantage in this case. Sometimes people who KNOW the effort are discouraged and never try. And it's the guy who is clueless who ends up embarking on the quest.

Re: “I Could Rewrite Curl”

#157
post #30

Let's be honest here. Curl to 98% of people is http/1 requests with some post params and maybe json body, with some custom headers. Most language standard libs (which might be using libcurl) can facilitate writing that relatively quickly. And probably with a more 'modern' CLI. Curl though, is a very wide breadth project. It currently supports the following protocols: DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP,…

I'm one of the 98%, and I don't like using cURL for this reason -- the CLI feels clunky and its gigantic number of options are distracting for my simple use case. Can anyone recommend a good alternative for the base case?

I recognize the huge amount of time and effort that went into make cURL, but I agree that the gigantic number of options are distracting. When I read the cURL man page, I have a hard time finding what I want because there are literally dozens of screens of options.

That said, I still will reach for cURL even when simpler options exist because it's ubiquitous. Same thing with Bash, grep, and sed.

Re: “I Could Rewrite Curl”

#158
post #144
post #72

Earlier quoted context omitted.

More and more APIs use HTTP as the transport for various reasons, but one of them is that HTTP has considered large number of edge cases and has been tested "in the fire" - rolling your own custom API with security and encryption is much more fraught with dangers.

Except HTTP 2.0/3.0 has very little to do with HTTP 1.0, and is more akin to use TCP/UDP directly anyway. It just happens to get a free pass over port 80.

And that's the other major reason - if your traffic LOOKS like web traffic, it will slip through firewalls and other middle-ware devices.

Re: “I Could Rewrite Curl”

#159
post #107

Earlier quoted context omitted.

Today I would probably reach for an RRB tree and define some kind of simple heuristic for transients, giving me free undo/redo, and reasonably simple support for multiple cursors. Back then a gap buffer seems like a great idea. Two linked lists is a simple enough idea.

Kudos to you for supporting multiple cursors from the start, which a buffer with a gap can't support efficiently. Douglas Englebart would have approved! https://en.wikipedia.org/wiki/Collaborative_real-time_editor >History of key products: The first instance of a collaborative real-time editor was demonstrated by Douglas Engelbart in 1968, in The Mother of All Demos. Widely available implementations of the concept to…

The multiple-cursors problem goes to show how much of computing is STILL strongly single-user, even for all our multiuser underpinnings.

Arguably the only "single user" devices should be things like the mouse itself, as multiple people can see the same screen (and maybe even use the same keyboard).

Some games implemented this to allow two-player on the same machine - each would get a joystick or "their half" of the keyboard.

Re: “I Could Rewrite Curl”

#160

Earlier quoted context omitted.

All those people don't realise tools like curl did start with few lines back then. And after the first two minutes of testing the devs started adding additional code to handle failures, invalid inputs, different platforms, different versions of a standard etc. One can write a lot of things in 100 lines if it doesn't have to meet higher standards than a basic chat app tutorial in the "getting started" section of a pro…

> One can write a lot of things in 100 lines if it doesn't have to meet higher standards than a basic chat app tutorial in the "getting started" section of a programming language. I'd argue that even a basic chat app is massively complex, only that all the really complex parts have already been solved. I once tried to write one on the LOWEST level I could. Realized after a month that there were not enough hours left…

really, we're standing on the shoulders of giants
Post reply on HN