Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“I Could Rewrite Curl”

daniel.haxx.se

Post reply on HN
Print view View original

161–170 of 193 posts

Re: “I Could Rewrite Curl”

#161

Post by jolmg »

jeffrwells wrote: post #115
parhamn wrote: post #30

Let's be honest here. Curl to 98% of people is http/1 requests with some post params and maybe json body, with some custom headers. Most language standard libs (which might be using libcurl) can facilitate writing that relatively quickly. And probably with a more 'modern' CLI. Curl though, is a very wide breadth project. It currently supports the following protocols: DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP,…

This begs the question of why all of these protocols should be in one library, instead of one each. If a project is only going to ever use http, why would I need to bundle in all of the others? Genuinely curious what the advantages are

What better thing to do with a universal resource locator syntax than to have a universal client?

> If a project is only going to ever use http, why would I need to bundle in all of the others?

Your project might only ever think to use http, but my project might appreciate having one client for any URL, instead of having to parse URLs myself and decide on separate clients based on a part of it.

Re: “I Could Rewrite Curl”

#162

Post by jethro_tell »

throwaway894345 wrote: post #124
rad_gruchalski wrote: post #120

Earlier quoted context omitted.

> A question that arises out of this is, should the 90% use cases be handled by a small, simple tool, or by the tip of the iceberg of a large, complex tool such as curl? But nobody is forced to use curl? People use it because it’s convenient, shoot them selves in the foot and then lash out at the author the tool for their own choices. Where’s the fault?

> But nobody is forced to use curl? I like curl, but this isn't true. When you're SSH-ing onto a box, you often don't have permissions to install your favorite CLI tool, and even if you do have said permissions it's inconvenient to have to install it each time you SSH onto the box (not a major inconvenience, mind you). Moreover, in many cases you need to run a script or some other software that depends directly on cu…

Well, your distro choose to include one lib that can be used with most protocols out there. They include the multi tool and many of the apps also included on the box require the tool.

If you want to use something else. You have to install it on the host. Every box comes with bash, but we still install other languages and frameworks on the host so we can run out applications with tools that make sense.

If you want a different tool, make it part of your default install.

Re: “I Could Rewrite Curl”

#163

Post by Gunax »

dcminter wrote: post #64
FriedrichN wrote: post #45

Earlier quoted context omitted.

I am always highly annoyed when someone has documentation for their web API and all that they provide is some postman collection of crap (or a crappy code example in an arbitrary language) and some vague abstract description of their API. Then I'll have to spend X amount of time on reverse engineering it to curl so I can see what's actually going on. Often I'll discover that they're doing really weird stuff (double b…

Doesn't Postman have a feature to allow exporting the request as curl and others though?

Sure, then they copy that nonsense command and send it to you, without really understanding what all of the options are, and just treating it like a magic blob.

I could totally see myself doing this

Re: “I Could Rewrite Curl”

#164

Post by jethro_tell »

Y_Y wrote: post #142
dkarl wrote: post #106

Earlier quoted context omitted.

A question that arises out of this is, should the 90% use cases be handled by a small, simple tool, or by the tip of the iceberg of a large, complex tool such as curl? I can see the advantages of standardizing on a complex, powerful tool for simple use cases. For one thing, it may be the only way to standardize: simple versions are too easy to write, and therefore you get dozens of competitors, none of whom are popul…

I would be surprised if nobody had tried to make a mini-curl that could go in busybox. The idea of having a tiny version of the program which handles the 90% cases by itself but can call out to the real-deal bigger brother when necessary is a nice one. This sort-of happens already with lots of common tools which are shadowed by shell builtins, why not curl?

For embedded systems we'll usually make our own curl with the exact protocols features we need.

No surprise but curl can do that too!

Re: “I Could Rewrite Curl”

#165

Post by swiley »

bombcar wrote: post #63
Denvercoder9 wrote: post #17

I wouldn't be surprised if the first comment is right that you could build a HTTP client sufficient for 90% of the usecases in ~100 lines or so. It's that other 10% that takes 90% of the time, effort and complexity, and for which having libraries like curl is invaluable.

I think there are two classes of cURL users - those who use it as a "let me download this file to the current folder" users who always forget the -O; and that use case is quite simple and seems it could be easily replaced. And then there are those who use all the API features and realize it's a complex program. But the missing link is that even the first use case has many complex edge-cases that aren't obvious at fir…

I always use wget for the "let me download this file to the current folder" because I know the flags better.

curl is for weird stuff like "POST" requests or other protocols.

Re: “I Could Rewrite Curl”

#167

Post by throwaway894345 »

jethro_tell wrote: post #162
throwaway894345 wrote: post #124

Earlier quoted context omitted.

> But nobody is forced to use curl? I like curl, but this isn't true. When you're SSH-ing onto a box, you often don't have permissions to install your favorite CLI tool, and even if you do have said permissions it's inconvenient to have to install it each time you SSH onto the box (not a major inconvenience, mind you). Moreover, in many cases you need to run a script or some other software that depends directly on cu…

Well, your distro choose to include one lib that can be used with most protocols out there. They include the multi tool and many of the apps also included on the box require the tool. If you want to use something else. You have to install it on the host. Every box comes with bash, but we still install other languages and frameworks on the host so we can run out applications with tools that make sense. If you want a d…

The point is that you’re not always the person who gets to decide which distro, which tools to install atop the distro, or which dependencies your scripts will use. If you own those choices, of course you can add in your own tool, but you frequently don’t own those choices.

Re: “I Could Rewrite Curl”

#168

Post by res0nat0r »

parhamn wrote: post #30

Let's be honest here. Curl to 98% of people is http/1 requests with some post params and maybe json body, with some custom headers. Most language standard libs (which might be using libcurl) can facilitate writing that relatively quickly. And probably with a more 'modern' CLI. Curl though, is a very wide breadth project. It currently supports the following protocols: DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP,…

I suspect curl and rsync power more things on the internet than people realize. They are both completely amazing tools that I use almost daily.

Re: “I Could Rewrite Curl”

#169

Post by benibela »

As Delphi programmer, I have so far been avoiding libcurl successfully, and spent years rewriting curl because I need to make HTTP requests

At first it was simple. I was making Windows apps and Windows has an API for HTTP requests. Just call wininet with the url, and it handles everything. No need to use someone's library if there is a standard OS API. And it is fully integrated in Windows, you get the default proxy settings and user-defined HTTPS certificates. It even shares cookies with the internet explorer (which I did not want, so I wrote my own cookie parser)

Then I made a linux port. Suddenly there is no API. But I also avoid C, since it is unsafe, while Delphi/Pascal has memory safe strings and arrays. So I needed a HTTP Pascal library. There are several libraries, which also support other protocols I have never heard of. Then I used the synapse library. It is really causing a lot of problems. Incomplete cookie handling, no gzip encoding, OpenSSL messing everything up...

Then I made an Android port. The standard stable system library was Apache HttpComponents, so I use that. Shortly after, Google makes a new Android version, and removes Apache HttpComponents. Then I had to find a new library.

In hindsight all the porting were useless projects. I should have told people to use WINE and be done with it. A stable API is much better than a library. Or write my own HTTP client from the start, which would have been much easier than switching libraries all the time. Especially if I have to implement half of it (header parsing, OpenSSL interfacing) anyways when the libraries do not work properly

Re: “I Could Rewrite Curl”

#170

Post by craftinator »

baud147258 wrote: post #160
craftinator wrote: post #133

Earlier quoted context omitted.

> One can write a lot of things in 100 lines if it doesn't have to meet higher standards than a basic chat app tutorial in the "getting started" section of a programming language. I'd argue that even a basic chat app is massively complex, only that all the really complex parts have already been solved. I once tried to write one on the LOWEST level I could. Realized after a month that there were not enough hours left…

really, we're standing on the shoulders of giants

Or possibly just standing on the shoulders of people standing on the shoulders of people standing on...
Post reply on HN