Live data from Hacker News

JSON Canvas – An open file format for infinite canvas data

jsoncanvas.org

61–70 of 168 posts

Re: JSON Canvas – An open file format for infinite canvas data

#61
post #21

If ids are unique strings, then why are `nodes` and `edges` arrays, as opposed to id-string-keyed maps?

An object in JSON is meant to be unordered [1], so arrays are better if you want order to be preserved. [1] https://www.json.org/json-en.html

Sure, but why do you want order to be preserved? The spec never says the order of nodes is pertinent. Is there undocumented z-index behavior?

Re: JSON Canvas – An open file format for infinite canvas data

#63
post #61

Earlier quoted context omitted.

An object in JSON is meant to be unordered [1], so arrays are better if you want order to be preserved. [1] https://www.json.org/json-en.html

Sure, but why do you want order to be preserved? The spec never says the order of nodes is pertinent. Is there undocumented z-index behavior?

Looking at their reference impl, it does seem that array index implies z-index, in which case fair enough, I suppose an array does make the most sense - they should probably document the z-indexing though!

Re: JSON Canvas – An open file format for infinite canvas data

#64
post #30

I've worked a few jobs now where application data was stored in text files of various kinds (homegrown as well as well-defined formats, JSON included) and it pretty quickly becomes a mess when you start talking about modifying it over time, evolving its schema, validating it in the face of end-user edits, ensuring threads don't write to it concurrently, etc. This strikes me as exactly the type of application data tha…

Agree that SQLite is a great local format for this sort of thing, but seems to get tricky when trying to sync across clients without conflicts. I've seen most CDRT schemes working with json documents rather than SQLite, but curious if there's solid conflict-free syncing out there for SQLite files being used inside applications?

Re: JSON Canvas – An open file format for infinite canvas data

#65
post #52
post #32

Earlier quoted context omitted.

I'm usually the first person to suggest SQLite for just about anything, but in this particular case I do feel like JSON is a better default format. Interacting with SQLite from different programming languages is easier than most other formats, but you still need a SQLite binding. They're available for every language but that's still a not-completely-trivial dependency. I expect most tools that people build against JS…

Valid points and I generally agree! I think you're probably right that the SQLite dependency is possibly too much for some applications to pull in, though I will ask why we're so often willing to pull in more or less anything else regardless of weight (DOM manipulation, state management, animation, etc.) but a robust data layer is often a bridge too far. I wish I knew why this is, as it seems to be one of the larger…

I think you're talking about the trade offs between supporting features like "DOM manipulation, state management, animation, etc." and "shipping updates" out of the box, versus only storing the data as simple files and leaving everything else to the implementation.

Sqlite as an application file format is great [1], but for a knowledge base / note taking app the benefits are not worth the tradeoffs in my opinion.

Sqlite is more performant more performant and provides lots of built-in features. However, most note taking users do not have enough notes or files to benefit from that performance. Sqlite will also lock the user into the application, whereas a "pack of files" can be used in the shell as a text editor. Using markdown files + a open json format has the benefit of being supported by multiple applications (e.g. sometimes i open my obsidian vault in vscode), while a sqlite database would need a proprietary schema coupled with a single application

I prefer an open file format that isn't tied to a vendor. A "data bridge" might handle syncing and diffing more efficiently than plain files, but it is still tied to the vendor. For example, I prefer not to pay for Obsidian Sync, and I'm able to use a git plugin and storing my files on nextcloud to sync between my devices. This leverages existing tech without having to implement features from the ground up

[1] https://www.sqlite.org/appfileformat.html

Re: JSON Canvas – An open file format for infinite canvas data

#66
post #30

I've worked a few jobs now where application data was stored in text files of various kinds (homegrown as well as well-defined formats, JSON included) and it pretty quickly becomes a mess when you start talking about modifying it over time, evolving its schema, validating it in the face of end-user edits, ensuring threads don't write to it concurrently, etc. This strikes me as exactly the type of application data tha…

SQLite is awesome, but it's still an order of magnitude more complex than JSON. If it wants to be the one file format to rule them all, we're going to need high quality and heavily used implementations in most languages.

Adding C sqlite to a golang project adds a significant hit to build times and cross-compilation/static linking complexity[0]. When I looked into the native Go implementations of sqlite I came away with the feeling it wasn't worth the tradeoffs compared to using the C version, but now I still have to deal with the issues above.

I haven't looked deeply into how sqlite works, but my instincts tell me the reason we don't have high quality implementations in every language is because it's actually too complex to treat as a protocol.

I would love to see something fill the void between plain text and sqlite.

[0]: https://www.arp242.net/static-go.html

Re: JSON Canvas – An open file format for infinite canvas data

#67
post #18

As a maintainer of an open source JavaScript infinite canvas application [1], I was very interested, and now I am a little disappointed. The set of supported objects on the canvas is quite minimal. [1] https://github.com/lovasoa/whitebophir

It's an early revision. Now sounds like the perfect time for you to share your experience and suggestions to improve it.

Re: JSON Canvas – An open file format for infinite canvas data

#68
post #61

Earlier quoted context omitted.

An object in JSON is meant to be unordered [1], so arrays are better if you want order to be preserved. [1] https://www.json.org/json-en.html

Sure, but why do you want order to be preserved? The spec never says the order of nodes is pertinent. Is there undocumented z-index behavior?

Sometimes you have nodes that overlap each other, so you want to control whether or not a node is in front of or behind another node.

Though yes, they could have explicitly defined a z-index or defined a convention on how the ordering should work (first nodes top and last nodes bottom or vice versa?). It's interesting to think about the trade offs between explicitly defining these things vs. leaving the application to implicitly make the choice. JSON Canvas seems to be designed to use in tandem with markdown files in a note taking app, so it makes sense why they opted for the more implicit design to be similar to markdown

Re: JSON Canvas – An open file format for infinite canvas data

#69
Obsidian is one of the few closed-source applications I would consider relying on, due to their commitment to building around simple, open file formats.

Sure, they could screw me over and start charging absurd amounts of money for their app, but high quality open source alternatives would pop up immediately.

Meanwhile, as long as they don't screw me over, it's unlikely an open source alternative is going to be able to catch up to a profitable business that keeps their users happy.

It's an interesting approach, focused on incentive alignment, which is the best way to ensure quality long term.

Re: JSON Canvas – An open file format for infinite canvas data

#70
post #24

I would love to see this and excalidraw converge feature & spec wise.

Any reference for obsidian's specs? The closest thing I could find is this: https://docs.excalidraw.com/docs/codebase/json-schema but it seems to be really minimal and doesn't go into detail on what properties belong in elements
Post reply on HN