Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JSON Schema bundling formalised

json-schema.org

Post reply on HN
Print view View original

51–60 of 188 posts

Re: JSON Schema bundling formalised

#51

Post by matja »

ChrisMarshallNY wrote: post #44
tored wrote: post #38

Earlier quoted context omitted.

One thing I'm guilty of is designing a document format as JSON, thus the need to formally verify it. Passing data works well as JSON but documents works better as XML where you can use XML Schema.

This is where I would also develop an XML variant, even if it is never to be used, simply so that I can have a Schema to go along with it. I'll start with a data structure/class, in the code, then use a built-in transformer, to turn it into JSON or XML. That way, I know that the "kernel" of the data is the same, between them. Even though JSON is a "natural" for scalar types, it can get weird, there. For example, let'…

Even with integers (comes up often for large numeric IDs):

   JSON.stringify(JSON.parse('{"kids":10000000000000001}')) -> '{"kids":10000000000000000}'
How do JSON schemas handle "must be an integer", and having to pass a string to get the necessary value?

Re: JSON Schema bundling formalised

#52

Post by 66fm472tjy7 »

reaperducer wrote: post #28
66fm472tjy7 wrote: post #8

Earlier quoted context omitted.

There are things you cannot accept. What do you do if an order has a negative quantity, if you get a string instead of a number, if a value does not fit in your DB column, etc ? Having this as part of your interface definition, along with generated representations of the possible data in the language your choice[0] and automatic (so you can't forget it) validation that rejects invalid input with HTTP 400 bad request…

What do you do if an order has a negative quantity, if you get a string instead of a number, if a value does not fit in your DB column, etc ? Isn't that all just basic data validation and sanitation that we all do anyway? And wouldn't it still have to be done, even if the JSON came with all the schemabloat? I don't think I'm suddenly going to accept that the data presented to my function is valid and fit for purposes…

  I don't think I'm suddenly going to accept that the data presented to my function is valid and fit for purposes just because it goes through someone's JSON schema library. That's just passing the responsibility on to a third party, which feels lazy and dangerous to me.
Do you also distrust that third parties can implement HTTP, generate SQL, enforce the constraints in your DDL, do arithmetic operations, etc. correctly ?

Of course there is still a lot of semantic validation you have to do manually (e.g. for an order: does the referenced product exist, is the seller allowed to sell it, at this time, for this price, for this currency, taxes, do the line prices add up to the order price, contingents, etc., etc.).

Expressing these constraints in your interface definition is just another declarative validation layer like your implementation language's type system, database schema/constraints, generic validation libraries[0].

Also, as the server author you are usually also the author of the interface definition containing all these constraints.

[0] E.g. https://beanvalidation.org . The OpenAPI to Java generator will express constraints using BV annotations. BV also integrates with the Hibernate ORM to validate data before you send it to the DB, or you can manually annotate your DTOs/domain model and programmatically trigger the validation at any time, so you only have to know one set of annotations and rely on a single library for validation in any layer of your application.

Re: JSON Schema bundling formalised

#53

Post by tored »

ChrisMarshallNY wrote: post #44
tored wrote: post #38

Earlier quoted context omitted.

One thing I'm guilty of is designing a document format as JSON, thus the need to formally verify it. Passing data works well as JSON but documents works better as XML where you can use XML Schema.

This is where I would also develop an XML variant, even if it is never to be used, simply so that I can have a Schema to go along with it. I'll start with a data structure/class, in the code, then use a built-in transformer, to turn it into JSON or XML. That way, I know that the "kernel" of the data is the same, between them. Even though JSON is a "natural" for scalar types, it can get weird, there. For example, let'…

Perhaps create your own custom type floatable that can be either be of type float, integer or string with a regexp pattern.

Re: JSON Schema bundling formalised

#54

Post by paulddraper »

csmpltn wrote: post #2

We've come full circle back to XML SOAP. We should consider restricting the registration of .org domains to actual non-profit organizations, and restricting the use of words like "schema" and "standard" to things that have been fully certified as such by internationally accredited engineering bodies. This doesn't add anything on-top of the tools we've had 40 years ago. We're stuck with this everywhere now, all thanks…

Do you mean XSD?

XSD = JSONSchema

WSDL = OpenAPI

SOAP = there really isn't a direct analog maybe ""REST""

---

In any case, try creating a SOAP service and WSDL. It's literally so complex that most tutorials have to rely on an IDE to do it. Humans are simply not equal to the task.

Yes, the purpose is similar, but only those who have never used them would think they're the same.

Re: JSON Schema bundling formalised

#55

Post by tored »

matja wrote: post #51
ChrisMarshallNY wrote: post #44

Earlier quoted context omitted.

This is where I would also develop an XML variant, even if it is never to be used, simply so that I can have a Schema to go along with it. I'll start with a data structure/class, in the code, then use a built-in transformer, to turn it into JSON or XML. That way, I know that the "kernel" of the data is the same, between them. Even though JSON is a "natural" for scalar types, it can get weird, there. For example, let'…

Even with integers (comes up often for large numeric IDs): JSON.stringify(JSON.parse('{"kids":10000000000000001}')) -> '{"kids":10000000000000000}' How do JSON schemas handle "must be an integer", and having to pass a string to get the necessary value?

I think you can use a string type with a regexp pattern.

Re: JSON Schema bundling formalised

#57

Post by paulddraper »

RustyRussell wrote: post #23

I use jsonschema to validate my projects' output, and also generate the documentation from it. The python jsonschema support makes this possible: our testsuite is in Python already. But it's an awkward fit, and I despair of having anyone else in my team write schemas: the default of allowing additional unspecified fields must be continually overridden, otherwise your schema has no teeth, and things like "if this fiel…

> otherwise your schema has no teeth

I don't disagree, but FWIW, this is common deliberate choice.

E.g. Protobufs work exactly the same way. Except unlike JSONSchema, there's no way to disable it.

The reasoning is to permit future extensibility.

Re: JSON Schema bundling formalised

#58

Post by k__ »

paulddraper wrote: post #54
csmpltn wrote: post #2

We've come full circle back to XML SOAP. We should consider restricting the registration of .org domains to actual non-profit organizations, and restricting the use of words like "schema" and "standard" to things that have been fully certified as such by internationally accredited engineering bodies. This doesn't add anything on-top of the tools we've had 40 years ago. We're stuck with this everywhere now, all thanks…

Do you mean XSD? XSD = JSONSchema WSDL = OpenAPI SOAP = there really isn't a direct analog maybe ""REST"" --- In any case, try creating a SOAP service and WSDL. It's literally so complex that most tutorials have to rely on an IDE to do it. Humans are simply not equal to the task. Yes, the purpose is similar, but only those who have never used them would think they're the same.

I had to use SOAP once... for one API endpoint.

I'd say it was the worst dev experience of my whole career, and I fought with objective-c and app stores.

Re: JSON Schema bundling formalised

#60

Post by segphault »

The way $ref resolution works is hideously complicated, to the point where many JSON Schema implementations just don't even bother supporting external $refs. It really should be a totally separate concern from validation, especially since how you store and compose schemas may end up being specific to a given use case.
Post reply on HN