JMESPath – A query language for JSON • Hacker News Board

Re: JMESPath – A query language for JSON

#111

Post by bshacklett » Sat, Feb 17, 2018, 9:22 PM UTC

crdoconnor wrote: post #28

xchaotic wrote: post #11

The reinvention of XML in JSON is almost complete - JMESPath vs XPath, JSON Schema vs XML Schema etc. If you need semi structured data to that level, consider using XML instead - you can validate it, there's plenty of tools, it's very stable and mature etc

The "reinvention" is not complete and will never be necessary. The difference is that XPath is necessary to query XML because it's a botched horribly overcomplicated, designed-by-committee markup language. Except for tools like jq no such language is actually required for JSON because it maps on to language structures that always exist. Neither JSON schema or XML schema are particularly popular - and for good reason.…

There's a lot of benefit to being able to state what keys may be specified in a certain location, though. Look at DSLs like Cloudformation, for instance. Having schema validation could make static analysis of this kind of code much easier to handle. E.g.: Fn::Sub may be used inside of Fn::Join, but the reverse is not true, regardless of the types "returned" by each. It's certainly possible to validate via the api, but being able to do it in my editor will make finding errors much faster.

To your other point, however, dynamic code generation is becoming much more common. AWS generates a huge amount of its code from JSON definitions across multiple languages to keep its SDKs up to date. I could see schema validation being valuable in this domain as well.

Re: JMESPath – A query language for JSON

#112

Post by JetSpiegel » Sat, Feb 17, 2018, 9:36 PM UTC

jelder wrote: post #84

Finnucane wrote: post #40

Earlier quoted context omitted.

It's not very much less verbose, really. You still need a key and a value. The only thing you lose, really, is the end tag. For a complex text document in say TEI or Docbook, I don't see how this is much of an advantage.

The reason I prefer JSON over XML is the latter's ambiguity. It's completely obvious how to convert a native object into a JSON object, for example. The translation to XML brings all sorts of questions about when to use tags vs. attributes of tags. It's cognitive load I'd rather spend on solving the problem at hand, not serialization. If there had been an opinionated SGML syntax which mapped directly and unambiguousl…

If your JSON is so complex it need a schema, doesn't XML extra structure help?

Re: JMESPath – A query language for JSON

#113

Post by cookiecaper » Sat, Feb 17, 2018, 9:53 PM UTC

rozzie wrote: post #64

For those interested in arbitrarily transforming JSON objects (for example, in a communications pipeline) I’d recommend JSONata. It’s quite useful and we’re well along in a Golang port with $function extensibility. http://jsonata.org

Agreed. I like JSONata a lot, even though it's the dark horse among JSON traversal languages. I've had a good experience parsing semi-unstable JSON with it.

Re: JMESPath – A query language for JSON

#114

Post by k6k7f7uky8ki » Sat, Feb 17, 2018, 9:54 PM UTC

JetSpiegel wrote: post #112

jelder wrote: post #84

Earlier quoted context omitted.

The reason I prefer JSON over XML is the latter's ambiguity. It's completely obvious how to convert a native object into a JSON object, for example. The translation to XML brings all sorts of questions about when to use tags vs. attributes of tags. It's cognitive load I'd rather spend on solving the problem at hand, not serialization. If there had been an opinionated SGML syntax which mapped directly and unambiguousl…

If your JSON is so complex it need a schema, doesn't XML extra structure help?

How would the extra structure help?

Re: JMESPath – A query language for JSON

#115

Post by k6k7f7uky8ki » Sat, Feb 17, 2018, 9:57 PM UTC

avmich wrote: post #92

booleandilemma wrote: post #38

Earlier quoted context omitted.

JSON still doesn’t support comments though, they need to work on that.

To me this sounds almost like "S-expressions still don't support comments". Why do you need comments in a structure of nested - unique key mappings - arrays - strings - numbers - booleans - nulls ? You can include comments, just like any string, into it, just reserve a key with unique name, if you want. From JSON parser/transformer point of view, "comment" as a concept isn't a data structure piece, it's rather "inten…

Something seems odd about wanting to put comments inside the JSON instead of before it. Then people will want a way to read the comments programmatically, then they'll want annotations, etc.

Re: JMESPath – A query language for JSON

#116

Post by danbruc » Sat, Feb 17, 2018, 10:14 PM UTC

spraak wrote: post #99

danbruc wrote: post #73

Earlier quoted context omitted.

I never tried it but I am expecting the performance to be not good enough, it takes already several minutes with code specifically written to perform the calculations I am interested in. And because I don't know what exactly I am looking for I need more or less interactive speed so that I can try out many different ways to look at the data. But maybe I could use [materialized] views to convey enough information to th…

Is this for an open source project, or anything you'll be publishing? I'd be interested to follow on with the results!

Right now it is just an effort to develop a tool to diagnose and hopefully thereafter fix random performance problems we are experiencing with one of our applications in production. Despite having a small team dedicated to investigating the problems, monitoring every click and function call with Dynatrace, having had a Microsoft SQL server expert look into it, and getting the system audited by one of the big consulting companies, the problem persists since years and nobody has really any clue about what is going wrong.

The performance is never really great, it is [one of] the central applications of the company and depends on the interaction with a sizable junk of the system landscape developed over decades and therefore it is prone to be affected by incidents in a lot of systems but most of the time it is good enough. But once every couple of weeks or months something goes badly wrong an requests, it's a web application, start taking several seconds or even minutes to complete. Minutes later everything is back to normal.

But I digress. If I would manage to come up with a reusable and somewhat general tool to analyze data similar to what I am looking at, I would consider releasing it. It could either be a somewhat general data analysis and visualization tool, think R, or it could be more specifically tailored towards looking for anomalies in data sets like the one I am investigating. But as of now I am struggling to come up with a general framework to express the analyses I am performing and therefore all I have is a rather ad hoc collection of transformations that extract and visualize aspects of the data that could lead to new insights into what is going on.

But right now it is really driven by our specific issue, I notice something in one view of the data and then come up with a new transformation to look at it in more detail or from a different angle. It is nothing that could easily be reused by anyone else and so for the moment it seems most likely that this will never become public or maybe only in the form of a blog article explaining what kind of information might be useful to look at and how to derive it from logs that look rather uninteresting at first glance.

Re: JMESPath – A query language for JSON

#117

Post by avmich » Sat, Feb 17, 2018, 10:29 PM UTC

bshacklett wrote: post #109

avmich wrote: post #88

Earlier quoted context omitted.

XML Stylesheet Transformation, the language to describe transformations of XML documents into another XML documents. Isn't jq or JMESPath an example of such a language for transformations?

No, those are query languages akin to XPath.

What's the difference between query and transformation then? Isn't transformation a (vector) function ^f of input ^x, with n dimensions of x and m dimensions of f where f_i = f_i(x_1, ... x_n)?

Re: JMESPath – A query language for JSON

#118

Post by murukesh_s » Sun, Feb 18, 2018, 4:57 AM UTC

bufferoverflow wrote: post #15

xchaotic wrote: post #11

The reinvention of XML in JSON is almost complete - JMESPath vs XPath, JSON Schema vs XML Schema etc. If you need semi structured data to that level, consider using XML instead - you can validate it, there's plenty of tools, it's very stable and mature etc

We still need XSLT for JSON to complete the circle. Though it seems like XSLT should support JSON: https://www.w3.org/TR/xslt-30/#json

Working on that, was thinking of naming it JST.. building it as part of a high level programming platform we are building.. will be open sourced once complete.

Re: JMESPath – A query language for JSON

#119

Post by granitosaurus » Sun, Feb 18, 2018, 5:36 AM UTC

Man I hate that every json parser has the need to introduce their own language.

We already have a bunch of well adopted and extremely powerful languages such as Xpath, why reinvent the wheel?

I recently made a simple tool for enabling xpath parsing for json https://github.com/Granitosaurus/pq - it just converts json to xml and lets you go at it with xpath or css selectors. Easy, beautiful and convient, except it's really slow with bigger files :|

Re: JMESPath – A query language for JSON

#120

Post by nurettin » Sun, Feb 18, 2018, 5:39 AM UTC

I already have a query language for json. I insert json into mssql 2017 community edition and query it there. https://docs.microsoft.com/en-us/sql/relational-databases/js...