Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JSON Canvas – An open file format for infinite canvas data

jsoncanvas.org

Post reply on HN
Print view View original

81–90 of 168 posts

Re: JSON Canvas – An open file format for infinite canvas data

#81

Post by DrDeadCrash »

aragonite wrote: post #20
regus wrote: post #6

What is the upside of using an infinite canvas? I tried playing around with them in the past but I think my brain's mental model of how to parse data is incompatible with "clutter" (for lack of a better term) of an infinite canvas.

Check out this video from one of my favoriate infinite canvas softwares, especially beginning around 1:25 (which I'm deep-linking to): https://youtu.be/GblI7GI0jQ4?t=85 (Alas, the original desktop app (iMapping, in Java) as shown in the video is no longer being developed, and now they are only developing the web app (Infinitymaps) only run in the browser, which imo is not the best fit for infinite canvas apps which c…

Imo it would need to show varying levels of information density depending on scale. Otherwise, as another commenter stated it's overwhelming....

Re: JSON Canvas – An open file format for infinite canvas data

#82

Post by asa400 »

desmondl wrote: post #65
asa400 wrote: post #52

Earlier quoted context omitted.

Valid points and I generally agree! I think you're probably right that the SQLite dependency is possibly too much for some applications to pull in, though I will ask why we're so often willing to pull in more or less anything else regardless of weight (DOM manipulation, state management, animation, etc.) but a robust data layer is often a bridge too far. I wish I knew why this is, as it seems to be one of the larger…

I think you're talking about the trade offs between supporting features like "DOM manipulation, state management, animation, etc." and "shipping updates" out of the box, versus only storing the data as simple files and leaving everything else to the implementation. Sqlite as an application file format is great [1], but for a knowledge base / note taking app the benefits are not worth the tradeoffs in my opinion. Sqli…

> I think you're talking about the trade offs between supporting features like "DOM manipulation, state management, animation, etc." and "shipping updates" out of the box, versus only storing the data as simple files and leaving everything else to the implementation.

I'm not sure I understand. Can you clarify?

> Sqlite is more performant more performant and provides lots of built-in features. However, most note taking users do not have enough notes or files to benefit from that performance.

For a graph with lots (thousands+) of nodes/edges, SQLite is probably capable of being more performant than a JSON file, depending on whatever specific kind of performance we're measuring. That said, to me, the most interesting thing that SQLite gives for applications when compared to flat files is data integrity, via transactions and schemas/constraints. Performance is nice but not even close to the most interesting thing about SQLite in most applications. Performance has never been the reason I've chosen SQLite over flat files for my applications.

> Sqlite will also lock the user into the application, whereas a "pack of files" can be used in the shell as a text editor. Using markdown files + a open json format has the benefit of being supported by multiple applications (e.g. sometimes i open my obsidian vault in vscode), while a sqlite database would need a proprietary schema coupled with a single application

In a sense you're right about this! I'll grant it's easier to open a JSON file in vscode and edit it if you already know vscode and JSON. That said, SQLite is in the public domain with a well-defined, stable format and there are countless free and open source database editors/viewers out there.

SQLite is self-describing, also. You open `sqlite3` and type `.schema` and it shows you the database schema. You enter a query and get some results. It's all right there. So, while a database might have a schema that was designed for a particular application, that doesn't mean you as the end user can't tinker with it, and the number of people who know SQL is rather large.

Re: JSON Canvas – An open file format for infinite canvas data

#83

Post by eichin »

remram wrote: post #34
kepano wrote: post #15

Whoa! Didn't expect this to bubble up to the top of HN. Some context about why we created JSON Canvas: https://obsidian.md/blog/json-canvas/ We just released it today, so this is still a very nascent project. A little over a year ago we released Obsidian Canvas. The .canvas file format has felt stable enough to give it a name and resources that other apps can freely use. See the original Show HN: https://news.ycombin…

Who was involved in this open spec? Or was it built by Obsidian with a hope that it works for everyone else? Which existing formats were considered before building your own, e.g. SVG/Excalidraw/draw.io/...?

Yeah, didn't see any discussion in the other thread, do you have any notes on why this isn't SVG?

Re: JSON Canvas – An open file format for infinite canvas data

#84

Post by chatmasta »

Macha wrote: post #45

One of the big reasons I use Obsidian is the data portability - while it provides some nice enhancements, if Obsidian went evil tomorrow and I had to use my notes in VS Code tomorrow, then even reduced to just "a folder full of markdown", I'd get 90% of the value without relying on any convoluted importers/exporters. I've stayed away from their Canvas feature largely because it is.. not that. Not because the Obsidian…

I've stayed away from their Canvas feature because the Excalidraw plugin for Obsidian is unbelievably good. I love it.

Re: JSON Canvas – An open file format for infinite canvas data

#85

Post by asa400 »

exceptione wrote: post #78
asa400 wrote: post #30

I've worked a few jobs now where application data was stored in text files of various kinds (homegrown as well as well-defined formats, JSON included) and it pretty quickly becomes a mess when you start talking about modifying it over time, evolving its schema, validating it in the face of end-user edits, ensuring threads don't write to it concurrently, etc. This strikes me as exactly the type of application data tha…

SQLite has no real types. Use SQLite if data quality is of no concern. The applicability domain of SQLite is far smaller than people think. If you want to improve on JSON, you would have to go into an other direction. Maybe something like postgis would be helpful for extremely large canvasses. JSON Schema is pretty powerful by the way. Checkout the documentation. SQLite is absolutely no match there. What Sqlite could…

SQLite's types are quite loose by default, no doubt.

There is a somewhat recent STRICT mode that strengthens them: https://www.sqlite.org/stricttables.html

Re: JSON Canvas – An open file format for infinite canvas data

#86

Post by [deleted] »

exceptione wrote: post #78
asa400 wrote: post #30

I've worked a few jobs now where application data was stored in text files of various kinds (homegrown as well as well-defined formats, JSON included) and it pretty quickly becomes a mess when you start talking about modifying it over time, evolving its schema, validating it in the face of end-user edits, ensuring threads don't write to it concurrently, etc. This strikes me as exactly the type of application data tha…

SQLite has no real types. Use SQLite if data quality is of no concern. The applicability domain of SQLite is far smaller than people think. If you want to improve on JSON, you would have to go into an other direction. Maybe something like postgis would be helpful for extremely large canvasses. JSON Schema is pretty powerful by the way. Checkout the documentation. SQLite is absolutely no match there. What Sqlite could…

[deleted]

Re: JSON Canvas – An open file format for infinite canvas data

#87

Post by asa400 »

LordDragonfang wrote: post #75
asa400 wrote: post #53

Earlier quoted context omitted.

Oh I read the post and a few of the other linked posts so I'm aware. If we're storing a file, we're already storing a file of some kind, so we're just talking about what kind of file it is. I'm just talking about a different set of tradeoffs one can make if that file is something other than JSON.

>If we're storing a file, we're already storing a file of some kind It's not just a file "of some kind", it's a text file. That's one of Obsidian's key selling points - that a vault is simply a collection of text files. The "text" part is important to both Obsidian's philosophy and the majority of its users.

Sure! And in a lot of ways this is about values more than technology.

Re: JSON Canvas – An open file format for infinite canvas data

#88

Post by cstrahan »

desmondl wrote: post #76
exceptione wrote: post #71

Earlier quoted context omitted.

> 6. YAML Please don't, it has one of the most confusing syntax out there with lists and maps, and it won't do well for parsing.

I haven’t had any issues with yaml in markdown frontmatter or openapi specs. What kind of issues do you see with list and maps that make you against yaml? I agree that for computers and consistency json is preferred. I already use a linter for my markdown files so I would do the same with yaml to keep lists and maps consistent

YAML is kind of like C++:

> You like C++ because you're only using 20% of it. And that's fine, everyone only uses 20% of C++, the problem is that everyone uses a different 20% :)

https://eli.thegreenplace.net/2009/10/17/the-c-bashing-seaso...

The YAML footguns are too numerous to reproduce here, so here are some sources:

https://stackoverflow.com/questions/3790454/how-do-i-break-a...

https://www.arp242.net/yaml-config.html

https://noyaml.com/

YAML isn't terrible if you only ever have to read what you wrote. Now consider that there are 63 different ways to write multi-line strings in YAML -- how many of those have you committed to memory? Yeah... now throw 10-100 developers into the mix, each with their own favorite alternative syntaxes -- good luck making sense of your YAML.

Re: JSON Canvas – An open file format for infinite canvas data

#89

Post by desmondl »

cstrahan wrote: post #88
desmondl wrote: post #76

Earlier quoted context omitted.

I haven’t had any issues with yaml in markdown frontmatter or openapi specs. What kind of issues do you see with list and maps that make you against yaml? I agree that for computers and consistency json is preferred. I already use a linter for my markdown files so I would do the same with yaml to keep lists and maps consistent

YAML is kind of like C++: > You like C++ because you're only using 20% of it. And that's fine, everyone only uses 20% of C++, the problem is that everyone uses a different 20% :) https://eli.thegreenplace.net/2009/10/17/the-c-bashing-seaso... The YAML footguns are too numerous to reproduce here, so here are some sources: https://stackoverflow.com/questions/3790454/how-do-i-break-a... https://www.arp242.net/yaml-confi…

Yeah true, I'm starting to remember the headaches with yaml when I was using kubernetes or cloudformation....

Re: JSON Canvas – An open file format for infinite canvas data

#90

Post by exceptione »

desmondl wrote: post #76
exceptione wrote: post #71

Earlier quoted context omitted.

> 6. YAML Please don't, it has one of the most confusing syntax out there with lists and maps, and it won't do well for parsing.

I haven’t had any issues with yaml in markdown frontmatter or openapi specs. What kind of issues do you see with list and maps that make you against yaml? I agree that for computers and consistency json is preferred. I already use a linter for my markdown files so I would do the same with yaml to keep lists and maps consistent

In the past I had to craft yaml files. Sometimes I needed quotes for a string, sometimes I had to put in a dash in front of a key, or just not. You basically needed to have the whole schema in your head.

There can only be so much nesting before you lose track of what item belong to which parent. Copying some yaml structures over to another level requires care, as the result might look correct, but the white space parser thinks otherwise.

I have lost hours of debugging yaml files when a dash was missing somewhere or when I needed one more leading space. The parser accepts it happily, but half of the typical javascript programs will only detect things are wrong when it has already executed on half of your spec. The other half will just run with input that wasn't intended that way.

I remember writing artillery.io test specs where all those problems pop up.

Now the good thing from JSON is JSON Schema. The latest spec allows you to specify quite advanced validations. Yaml has no such thing.

As to your remark: Yaml for front matter is defensible, as you dont have deeply nested structures. Though, as an obsidian user you want to make sure your front-matter is conforming to your own schema. That would require writing a json spec and then have your yaml internally converted to json before handing it over to the validator.

A spec is worthless if you cannot validate against it. Json and xml have a good story there. I concede that yaml is more human-readable than json without an editor. Correctness is the holy grail though.

Post reply on HN