API Practices If You Hate Your Customers • Hacker News Board

Re: API Practices If You Hate Your Customers

#231

Post by MadWombat » Thu, Dec 12, 2019, 4:52 PM UTC

AgentME wrote: post #129

MadWombat wrote: post #46

> An operation is idempotent if performing it multiple times yields the same result as performing it exactly once. And then he casually offers an API that does different things on first and second call as the "good" example. If you have a "create a virtual machine" API it better create a fucking virtual machine. If I call the damn thing twice, I expect to have two VMs. If there is some sort of unique argument like cr…

With an idempotent API, starting a VM and doing something with it can look like this: let new_id = generate_id(); retry_with_backoff(() => api.make_vm(new_id)); retry_with_backoff(() => api.do_thing_with_vm(new_id)); This works even if any individual API calls fail, or if the API call makes it to the API server but the response fails to make it to the client. If the APIs aren't idempotent, then you would have to do t…

Lets start simple, your example assumes that you generate the id yourself. In my experience a common API usage pattern would look more like

  try:
      vm_id = api.make_vm()
  except SomeError as e:
      log.error(e)
  else:
      res = api.do_thing_with_vm(vm_id)

and in your example, if we are generating ids ourselves, we still have to verify that we got the right VM. If your ids are provably unique, there is no reason to generate them, the API can take care of that, but if you want something like a named entity, you have a problem. What if the name is already taken? So your code would look more like

    new_id = generate_id()
    try:
        vm = api.get_vm(new_id)
    except VM_DoesNotExist:
        vm = api.make_vm(new_id)
    except SomeError as e:
        log.error(e)
    else:
        api.do_thing_with_vm(new_id)

because if the make_vm API simply returns a VM whether it was created or not, it is entirely possible that you are getting a VM that is busy doing something else for some other process.

Re: API Practices If You Hate Your Customers

#232

Post by DCoder » Thu, Dec 12, 2019, 5:36 PM UTC

bouke wrote: post #203

gtsteve wrote: post #2

I was expecting to see two of my pet hates - returning a null array to represent no items, and returning a single object without an array to represent one item. I also once worked with an API where you had to send the data in POST format - abc=123&def=456. After much pressure from their customers, they finally relented and added an XML version of their API... where your request could look like this: abc=123&def=456 .…

I had the pleasure of working with the API of a customer that wanted to expose a JSON/REST API to their existing XML/SOAP backend. Instead of going the sane rout and re-use the XSDs to serve as the structure for the JSON, they just made the JSON structure up on the go. One child node? That would be one JSON object / value for you sir. Multiple child nodes? That would be on JSON array for you sir. No child node? No JS…

That reminds me of a vendor who wraps SAP Business One in their own webservice. This webservice has two business methods.

The first one, ExecuteXML, takes an representing a regular SAP B1 XML request and passes it on to one of the real SAP services. We have to find our own XSDs for the inner part, because they sure as hell don't have those.

The second one is ExecuteSQL. It lets us run raw SQL against the SAP database. It doesn't have any support for prepared parameters. What it does have is a blacklist to prevent DDL and other funny business, such as semicolons. This blacklist runs on the raw string you send, and doesn't understand any escape characters. To send a string containing a literal semicolon, I had to turn it into CONVERT(VARCHAR(MAX), 0x...).

Re: API Practices If You Hate Your Customers

#233

Post by wwweston » Thu, Dec 12, 2019, 6:38 PM UTC

celticmusic wrote: post #191

wwweston wrote: post #130

Earlier quoted context omitted.

While we're asking questions: what stack out there makes reading/writing HTTP headers hard? Because that's all it takes to work with response codes. And yes, of course parsing JSON isn't difficult. Note that I said parsing messages -- the message field not the response body. And parsing that message field and checking conditionals to determine your client's behavior is something you'll have to write code for unless y…

> And parsing that message field and checking conditionals to determine your client's behavior is something you'll have to write code for unless you're just relaying the error message back to the end user. Oh noooo, you have to.... handle errors coming out of API's you consume. What an imposition... It must be terrible to be you, always having to write code to handle when things don't go exactly down the happy path.…

> Oh noooo, you have to.... handle errors coming out of API's you consume. What an imposition... It must be terrible to be you, always having to write code to handle when things don't go exactly down the happy path.

Do you really think this is a conversation about never wanting to handle errors, or is that sarcasm as a convenient way of getting out of actually thinking during the discussion?

Good specific HTTP status codes from the application layer help the client sort errors by type before they have to parse specifics. Or in cases where the client may not even have been prepared for the specifics.

Have you really never found a software situation where it's useful to know what the type of error is before you get into the details?

If that's the case, you definitely shouldn't be anywhere near API design.

> Imagine seeing a 500 and knowing for sure that there's a server misconfiguration.

Imagine knowing there's a whole range of 5xx status codes that allow for both that possibility and others.

> Being able to trust http status codes: it's what's for dinner.

Well, I'm glad we've gotten here, given that you seemed to start with "200 all the things, sort it out in the response body."

Re: API Practices If You Hate Your Customers

#234

Post by lilyball » Thu, Dec 12, 2019, 7:29 PM UTC

simiones wrote: post #208

lilyball wrote: post #161

Earlier quoted context omitted.

Why would you want to marshal a nil slice to json as non-null? That just hides the fact that you had a nil slice. In fact, that's the exact opposite problem; representing nulls as empty arrays.

As explained in the thread, that is because Go best practice is to treat `nil` and `[]T{}` as the same thing in every API. If you want an explicit `null` in your JSON you can use `*[]T` as your field type, and then it makes more sense. The very idea of allowing the nil value for slices seems to be very strange and inconsistent, as slices are struct types, not pointer types in Go (they contain a pointer and other memb…

It is admittedly very weird that Go supports nil slices/maps instead of just having the nil value be an empty slice/map that points to constant storage. But as long as Go has a semantic difference internally, representing that as null externally makes sense. Though I suppose as long as the conversion from null to empty array/object is opt-in it's fine.

For context, we recently had a bug where backend forgot to initialize their map, so they were sending us a null where we expected an object, and it would have gone undetected for much longer if the JSON didn't contain a literal null there.

Re: API Practices If You Hate Your Customers

#235

Post by ergothus » Thu, Dec 12, 2019, 9:41 PM UTC

lowercased wrote: post #216

ergothus wrote: post #74

Earlier quoted context omitted.

That's the DESIRE for what the junior learned. But is that what they actually learned? One could just as easily (or more easily) decide instead that this is the way business is done. It's literally all they've ever seen.

There's no way to know in advance what someone will learn from an experience. You also don't know what they learn 'now', and what they might reevaluate and relearn years from now about that same situation. Basing your response decision primarily around what someone might learn isn't a great way to decide how to respond. Couple folks I'm working with right now, and I had thought a couple of times "well, this wasn't a…

> There's no way to know in advance what someone will learn from an experience.

Correct. You can, however, improve the odds. Being explicit about what you want them to learn as opposed to expecting a certain degree of interpretation cannot hurt your odds, though it can't guarantee them.

> Basing your response decision primarily around what someone might learn isn't a great way to decide how to respond.

In this context, what else was the point of the response? I was comparing to the STFU tell-off of the above post and the defense of it that they would learn from the experience.

Re: API Practices If You Hate Your Customers

#236

Post by sangnoir » Thu, Dec 12, 2019, 9:51 PM UTC

donkeyd wrote: post #177

gtsteve wrote: post #2

I was expecting to see two of my pet hates - returning a null array to represent no items, and returning a single object without an array to represent one item. I also once worked with an API where you had to send the data in POST format - abc=123&def=456. After much pressure from their customers, they finally relented and added an XML version of their API... where your request could look like this: abc=123&def=456 .…

I worked with a product for natural language processing that wanted the text in a query string. This led to 100+ page documents begin sent as a string in the request. My usual REST testing app would freeze up if I wanted to test some of the largest documents in the data set.

> This led to 100+ page documents begin sent as a string in the request

How were you able to do this when the standard maximum length of a query string is 1024 bytes? I guess you could flaunt the standard as you were responsible for the backend

Re: API Practices If You Hate Your Customers

#237

Post by perl4ever » Fri, Dec 13, 2019, 1:37 AM UTC

Ididntdothis wrote: post #62

gtsteve wrote: post #2

I was expecting to see two of my pet hates - returning a null array to represent no items, and returning a single object without an array to represent one item. I also once worked with an API where you had to send the data in POST format - abc=123&def=456. After much pressure from their customers, they finally relented and added an XML version of their API... where your request could look like this: abc=123&def=456 .…

“returning a single object without an array to represent one item.” I saw the same thing when working with PHP consuming data from a SOAP endpoint. First I thought PHP is stupid. But then I realized that XML can’t model single item arrays versus single objects. They look the same. JSON is better that way. You can model empty arrays and single item arrays.

What would we do without ten different ways to represent nothing?

  if ismissing(x) then
       x = empty
  end if

  if isempty(x) then
       x = ""
  end if

  if x is nothing then
       x = new foo
  end if

  if right(typename(x), 2)  "()" then
       x = Array(x)
  end if

Re: API Practices If You Hate Your Customers

#238

Post by perl4ever » Fri, Dec 13, 2019, 1:42 AM UTC

favorited wrote: post #28

gtsteve wrote: post #2

I was expecting to see two of my pet hates - returning a null array to represent no items, and returning a single object without an array to represent one item. I also once worked with an API where you had to send the data in POST format - abc=123&def=456. After much pressure from their customers, they finally relented and added an XML version of their API... where your request could look like this: abc=123&def=456 .…

There's often a semantic difference between null and empty. Like, if I'm checking the result of a batch processing job, I want to know if the job finished & resulted in an empty set, or there is simply no result yet. It's "the thing you're looking for doesn't exist" vs. "the thing you're looking for exists, but is empty."

Of course there is, but generally speaking there's an infinite number of possible reasons for an empty or null value, not precisely two. That's why if you don't watch out they multiply.

Re: API Practices If You Hate Your Customers

#239

Post by perl4ever » Fri, Dec 13, 2019, 1:46 AM UTC

arethuza wrote: post #168

jiggawatts wrote: post #64

Earlier quoted context omitted.

I had this happen with a vendor that was told that they needed to add REST support to their legacy Java app. They turned up with an API that encoded XML as base64 and put that in a POST variable on HTML page. Then they added their own crypto on top with hard-coded RSA keys. They helpfully included both the public and private keys in their documentation. When they turned up for a meeting to present the fruits of their…

Unfortunately I have vivid memories of one API I worked with (many years ago) that included entire encoded XML documents as attribute values within top level XML. At least there was only one level of recursion. Still, perhaps not as bad as using XML documents as database keys...

A common misconception is that you can put anything in xml via CDATA so long as it doesn't have the ending delimiter embedded.

https://stackoverflow.com/questions/21087648/xml-invalid-cha...

I know about this because I received a file with illegal CDATA characters once.

Re: API Practices If You Hate Your Customers

#240

Post by celticmusic » Fri, Dec 13, 2019, 2:24 AM UTC

wwweston wrote: post #233

celticmusic wrote: post #191

Earlier quoted context omitted.

> And parsing that message field and checking conditionals to determine your client's behavior is something you'll have to write code for unless you're just relaying the error message back to the end user. Oh noooo, you have to.... handle errors coming out of API's you consume. What an imposition... It must be terrible to be you, always having to write code to handle when things don't go exactly down the happy path.…

> Oh noooo, you have to.... handle errors coming out of API's you consume. What an imposition... It must be terrible to be you, always having to write code to handle when things don't go exactly down the happy path. Do you really think this is a conversation about never wanting to handle errors, or is that sarcasm as a convenient way of getting out of actually thinking during the discussion? Good specific HTTP status…

> Do you really think this is a conversation about never wanting to handle errors, or is that sarcasm as a convenient way of getting out of actually thinking during the discussion?

It was me making fun of someone trying to argue that you would have to write code for the non-happy path in scheme A, but not B.

> Good specific HTTP status codes from the application layer help the client sort errors by type before they have to parse specifics. Or in cases where the client may not even have been prepared for the specifics.

> Have you really never found a software situation where it's useful to know what the type of error is before you get into the details?

Yes, because you could never embed that sort of information into the JSON, that's not what it's for! It's for... well I guess if you're not using it for that sort of information I don't know what it's for.

> Imagine knowing there's a whole range of 5xx status codes that allow for both that possibility and others.

https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#5xx_...

11 5xx status codes. I'm glad to know these cover all possibilities for the software you write, but you know what's even worse than anything we've discussed here?

15 different API's defining 512 to mean different things specific to their software. All in the name of software cleanliness, because parsing json is apparently icky and hard in brainfuck, their API language of choice I guess?