Live data from Hacker News

How to store your app's entire state in the url

scottantipa.com

161–170 of 416 posts

Re: How to store your app's entire state in the url

#161
I imagine I'm not the first one to think of this, but a little while back I wrote a toy pastebin-like webpage that stored the code being rendered directly in the URL (with base64 and gzip compression to make it a bit smaller, although it still was quite large by usual standards). My thinking was that rather than having a centralized app for this publicly available, making a small page that could be statically hosted would make it easy for people or organizations to host their own page with whatever security they need (e.g. behind a firewall or VPN) and then the links could be shared privately on Slack or whatever without a need to add functionality for determining who should have access or how long to keep the code snippet around and instead just piggyback off of the privacy of whatever channel/DM they sent it across and how long the Slack instance is configured to retain history.

When I first came up with this idea, I was going to write some sort of server backend rather than just doing everything in the frontend, but once I started actually working on it, I realized that it wouldn't really add any value and it would make it a bit more annoying for people to self-host. Since then, I've wondered a bit how viable "static pages with state in the URL that can be easily self-hosted" would be as an alternative to a desktop GUI or Electron app for more technical users. One of the nicest parts of this approach to me is that I was able to make in only a few hours one evening despite knowing only the basics of webdev from maybe 8-10 years ago and next to nothing about programming GUIs in general. This makes me think that it wouldn't be too hard for people to fork and modify it to their own liking (e.g. changing the hard-coded style settings for rendering the code snippets), so there could be potential for an ecosystem to grow organically around something like this.

In case anyone is curious to take a look: https://gitlab.com/saghm/pastml/

Re: How to store your app's entire state in the url

#162
post #54

I'm not into web "programming", but do people really have to abndon binary data storage these days? I think compressing more informaiton produces still more information. While JSON is by no means as heavy as, for example, XML, it's far heavier than just array of int graph[x][y];

At work we have two api endpoints. One for JSON and the other for binary.

Re: How to store your app's entire state in the url

#163
Last November I had covid and during that time I wrote a little pastebin that runs in the browser, compresses the paste data with Brotli, and puts it in the URL. It has line numbers, file naming, and syntax highlighting so it's feature complete for my own use. Here's a demo, but beware, the URLs are loooong: https://nicd.gitlab.io/t/#NhfW?Qm3F?6-&22c&CQZXuP+Aej5OXzXk7...

I find it interesting to sometimes play around with the pasted data just to see how it compresses. Sometimes adding a character or two may cut several characters from the URL length!

Re: How to store your app's entire state in the url

#164
Like some other commenters mentioned, this is a cool idea, however by storing JSON data inside base64, the length of the URL blows up very quickly as you start storing more state.

While technically URLs have no formal length limit, various sources suggest that URLs with more than around 2,048 characters start causing issues in browsers, and around 8,196 start causing issues in CDNs (https://stackoverflow.com/a/417184). You can work around this partially by not storing this state information in the query string (path/to?) and instead using a hash string (path/to#), and then extract that data in JavaScript such that it's not sent to the server at all.

IMO, the canonical way to solve this problem is using protocol buffers, which you can then serialize into a much smaller number of bytes, which is then base64 encoded. For example, as mentioned in another comment in this thread (https://news.ycombinator.com/item?id=34314578), Yahoo Finance has a 1,616 character-long URL with 1,572 characters of base64-encoded JSON state:

> {"interval":"week","periodicity":1,"timeUnit":null,"candleWidth":4.3486590038314175,"flipped":false,"volumeUnderlay":true,"adj":true,"crosshair":true,"chartType":"line","extended":false,"marketSessions":{},"aggregationType":"ohlc","chartScale":"linear","studies":{" vol undr ":{"type":"vol undr","inputs":{"id":" vol undr ","display":" vol undr "},"outputs":{"Up Volume":"#00b061","Down Volume":"#ff333a"},"panel":"chart","parameters":{"widthFactor":0.45,"chartName":"chart"}}},"panels":{"chart":{"percent":1,"display":"F","chartName":"chart","index":0,"yAxis":{"name":"chart","position":null},"yaxisLHS":[],"yaxisRHS":["chart"," vol undr "]}},"setSpan":{"multiplier":5,"base":"year","periodicity":{"period":1,"interval":"week"}},"lineWidth":2,"stripedBackground":true,"events":true,"color":"#0081f2","stripedBackgroud":true,"eventMap":{"corporate":{"divs":true,"splits":true},"sigDev":{}},"customRange":null,"symbols":[{"symbol":"F","symbolObject":{"symbol":"F","quoteType":"EQUITY","exchangeTimeZone":"America/New_York"},"periodicity":1,"interval":"week","timeUnit":null,"setSpan":{"multiplier":5,"base":"year","periodicity":{"period":1,"interval":"week"}}}]}

The un-base64'd JSON dictionary is 1,162 characters long, 569 of which is composed of just the key names (48 percent!), and most of the remaining fields likely contain their default value. If this was instead encoded in protobuf, the key names wouldn't need to be included, since fields are referenced by their field ID, and default field values would take up little-to-no space since when the default value is present it isn't encoded into the structure. I presume that, if done efficiently, you could likely make an equivalent representation in protobuf that is 1/4th or even 1/8th the size. You also get, for better or for worse, what is essentially obfuscation for free (yes, you can read out the packed format, but without an equivalent protobuf struct definition it's still hard to tell what field is which, and the entire operation becomes much more labor-intensive).

Re: How to store your app's entire state in the url

#165
post #122

Earlier quoted context omitted.

Another reason to not base64 encode is that many URLs with base64 strings will break in iMessage do to strings randomly matching various keywords that iMessage looks for. I think ‘usd’ is one such substring to look out for.

base64 does not have the problem here, iMessage has. Frustrating.

Agreed, iMessage should have URL detection that supersedes any other string parsing.

Re: How to store your app's entire state in the url

#166

App state in URL can be a good idea, but if possible I prefer readable path/query parameters instead of unreadable base64 encoding. As one comparison, this is Google Finance encoding stock chart parameters: https://www.google.com/finance/quote/F:NYSE?window=5Y Versus Yahoo! Finance doing the same: https://finance.yahoo.com/quote/F/chart?p=F#eyJpbnRlcnZhbCI6IndlZWsiLCJwZXJpb2RpY2l0eSI6MSwidGltZVVuaXQiOm51bGwsImNhbmRsZ…

In your example, absolutely the first URL is better. But if as in the case of the OP, you're trying to encode an entire flowchart, I would argue it's a bit of a pointless exercise. Maybe you could come up with something reasonably readable, but I don't think anyone would care. It really depends on what kind of state you need to store.

Re: How to store your app's entire state in the url

#167

A better idea is to use the URI fragment (the string after the # sign). It has the advantage of not having a size limit, and not needing to be sent all to the backend during the request. `window.location.hash` is your friend.

I just updated knotend so that it now uses the hashmark. I think this is a great idea, and it just loses the ability to get the urls server side, which in my case is totally fine. I also updated the blog post and acknowledged your comment, so thank you!

Re: How to store your app's entire state in the url

#169
post #145

Earlier quoted context omitted.

Harder. Not impossible. Harder. I don't want to make it sound like I am being disagreeable just for the sake of being disagreeable. If there is one thing the past decade has shown, it is that hacking is just a matter of time and whether a determined person is willing to direct resources at it.

I see you, and raise you: What if they encrypted the parameters with a one-time pad ?

Hmm. Most likely ( if not only ) way to counter that would be social hacking ( because I assume the pad is generated automatically from some source ), which seems like the best way to obtain it. Then again.. I am not an expert in this field ( but we do sometimes use parameters from link for some projects ). Is it refreshed after some time passes ( if so, maybe there is an easier way to just observe what changes )?

Re: How to store your app's entire state in the url

#170

This is pretty common and has a bunch of advantages, like the fact you can link to and bookmark a particular state. Also, if you are careful you get undo and redo for free with the browser's back button doing all the work for you. The disadvantages are that your representation of internal state becomes part of the interface - if you ever change your app you need to deal with versioning the state so your new version c…

> The disadvantages are that your representation of internal state becomes part of the interface This is the biggest reason to avoid this. URLs aren't meant to be used this way. I'd only do it if I wanted a quick and dirty solution.

> URLs aren't meant to be used this way

I disagree, URLs are supposed to point to a resource - in this case the resource is the client state of the app (aka "deep linking"). It is a useful approach.

Of course, it can easily be misused.

Post reply on HN