“I Could Rewrite Curl”

This is a problem I fight against nail and tooth almost daily, and it's closely related to the not invented here syndrome [0]. I would even go so far as to call it the Dunning–Kruger effect of technical knowledge. For context, I'm a frontend developer slash consultant who usually gets called in when development slowly grinds to a halt due to an overbearing number of bugs and regressions. Almost exclusively, at their core these problems can be reduced to underestimating complexity and trying to build something from scratch. It's a very common theme — "I can build form validation from scratch, that's easy." But then there's async validators, fields that have circular references and timezone issues. "I can build a table from scratch," but then there's endless scrolling, custom filters and performance issues. I think the common denominator is that all of these issues look simple when you address the smallest possible usable case, but quickly blow up in your face when you start building without thinking what you need to add on later. Unless you want to hire me, don't be that guy. Forms, tables, network layers and most other things are already solved problems — please don't try to build them from scratch unless you really need to and have a good plan.

[0] https://en.wikipedia.org/wiki/Not_invented_here

>Just a little tangent but I think everyone should know cURL… not postman, not httpie or any other more “friendly” tool.

5 years ago I was partner in a startup, we got some money from government right about the time I said I have to take a few months off to do consulting and make some money, my partner said I want to use money to make app I said no for various reasons but he went ahead and did app.

So I was helping them to do the app a couple hours every night - while consulting 8+ hours a day. Anyway the guy making app was using postman, I'd never used postman. He had a problem with the a part of the API not working.

I sent him a curl request to post a two line json document to the api and showed it got the correct response back.

His response - I'm not a curl expert or anything, it would really help me out if you downloaded and installed postman and tried to make it work!

My response - I'm not a curl expert or anything, I showed you it worked in curl because that is basically the default and should establish it is possible to do. You figure out how to do it in your tool.

This is quite acceptable, IMNSHO. If you're reinventing a wheel, of course you need to know why your predecessors have not used the Obvious Simple Shortcut That Intuitively Makes Sense. Was it not available back when this was written? Was it to keep compatibility with Windows XP? Or perhaps your idea really did not occur to the developers.

Most time - when reimplementing stuff - is not spent coding, or even debugging, but on research.

"I could rewrite hn's url parsing regex!" -not me ;)

Curl is magic.

Every so often (less so in recent years) I come across a bad homespun implementation of an HTTP client embedded in a larger project. Usually, a few lines of code to open a socket and POST some data to a hard coded URL. I always replace these with curl, usually over the objections of the original programmer.

The problem with HTTP (v1) is that it seems very simple to write a cut down client. But your simple client better handle proxies, various encodings, chunked bodies, redirects, HTTPs (with all the machinery that implies), and a whole host of annoying little features that the client is not in control of.

"But I don't need to implement talking through a proxy, we don't use one"

You don't use one now, but in 6 months you might. Or your client's might install one without your knowledge.

It is actually pretty simple to write an HTTP server (I wouldn't advise it) because the server controls a lot of the conversation. But clients are best left to either the OS or curl.

Self-promotion time: https://sheep.horse/2019/3/using_libcurl_effectively_and_saf...