If anyone is looking at a free tool to quickly edit CSV data in an Excel-like editor, see https://editcsvonline.com
Show HN: Transform a CSV into a JSON and vice versa
91–100 of 111 posts
Re: Show HN: Transform a CSV into a JSON and vice versa
#92CSVJSON is fundamentally an unsolvable problem because of mismatch in data hierarchies among them.
Plus you have the type looseness for both and lack of standards for CSV.
A trivial 2-D case is handled well by Python library such as Pandas. Here OP could be an alternative.
When I say trivial I mean flat 2 dimensional data, such as you would get from Mockaroo or similar source.
However in real life - data is messy.
As you get into 3,4 and deeper hierarchies on JSON you can't really translate that into nice flat 2d CSV.
Then you have missing keys, mixed up types and you end up rolling you own hand written converters.
Re: Show HN: Transform a CSV into a JSON and vice versa
#93Earlier quoted context omitted.
Collecting usage statistics is harvesting data. This is a classic example of why you should never run random NPM modules. Or even install them as all of this is possible in a post install script too. Putting analytics in a deployed app is your prerogative. Putting it in what touts itself as a reusable component is at best frowned upon.
Ok, I see your point. I update the website. This commit deletes any kind of data harvesting (removes fingerprint and visit counting): https://github.com/erikmartinjordan/jsonmatic/commit/7f3fa89...
web-vitals is still there. What is it used for? https://github.com/erikmartinjordan/jsonmatic/blob/f926f197b...
Re: Show HN: Transform a CSV into a JSON and vice versa
#94Earlier quoted context omitted.
To parse it, you need to check the keys. If there can’t be other keys, you can just use an array which is stable on JSON and you can save on keys. So you just have an array of arrays. Or even a huge array and every X elements, it’s a new record. If each one has 2 keys, [ { key1: ‘a’, key2: ‘b’ }, { key1: ‘a’, key2: ‘b’ } ] Can become, [ [ ‘a’, ‘b’ ], [ ‘a’, ‘b’ ] ] Or just every 2 will be a new record, [ ‘a’, ‘b’, ‘a…
But why ? Why save on keys when compression will nearly eliminate them for you?
Compression mainly helps with transmission.
Trying to point out that the original structure allows for more flexibility.
If you only cared about space, this compresses better anyway and uncompressed, it still occupies less space.
Re: Show HN: Transform a CSV into a JSON and vice versa
#95Earlier quoted context omitted.
Ok, I see your point. I update the website. This commit deletes any kind of data harvesting (removes fingerprint and visit counting): https://github.com/erikmartinjordan/jsonmatic/commit/7f3fa89...
web-vitals is still there. What is it used for? https://github.com/erikmartinjordan/jsonmatic/blob/f926f197b...
It comes by default on React apps, it’s not used for anything.
Re: Show HN: Transform a CSV into a JSON and vice versa
#96Earlier quoted context omitted.
+1 - If you are looking for something JSON-compatible, JSON Lines (one json object per line - https://jsonlines.org/ ) is pretty popular as well. You could store a CSV in JSONL very easily. In fact, jsonlines' website shows it as its first example: https://jsonlines.org/examples/ And if you wanted something that is json file-wide, you can just add some commas and wrap in [] for a list of rows.
Oh no, this is so similar to http://ndjson.org/
ndjson appears to be a fork of jsonlines with the addition of a spec (see https://github.com/ndjson/ndjson.github.io/issues/1)
Re: Show HN: Transform a CSV into a JSON and vice versa
#97Earlier quoted context omitted.
Escaping breaks often as well. The general problem is that the data is inline with the format. Parquet files or something that has clear demarcation between data and file format are more ideal but probably nothing is perfect or future proof. Or accepting of past mistakes either.
You can write a perfectly isomorphic escaping printer/parser
Obligatory https://xkcd.com/927/ but also all systems would have to be proven correct or else it wouldn't work. "Forgiving" parsers are the norm and you can't rely on what they do deterministically.
Re: Show HN: Transform a CSV into a JSON and vice versa
#98Earlier quoted context omitted.
that's just CSV with unnecessary square brackets and whitespace
As someone who's written parsers for both CSV and jsonlines, I can assure you that you could not be further from the truth: 1. The whitespace is optional. It's just put there for illustrative purposes. 2. Whitespace in CSV can actually corrupt the data where some parsers make incompatible assumptions vs other CSV parsers. Eg space characters used before or after commas -- do you trim them or include them? Some will d…
Re: Show HN: Transform a CSV into a JSON and vice versa
#99CSV is more of a rumor than a standard, plus JSON can have a tree structure. It is a fun idea to think about and may be useful in some narrow cases, but will fail in almost all but those most trivial of structures.
Trees are trivially flattened, and it's literally a couple lines of comments or documentation to describe what flavor of CSV you're using.
Flattening is a lot of duplication
Re: Show HN: Transform a CSV into a JSON and vice versa
#100Earlier quoted context omitted.
That sounds incredible inefficient What was the rational for such enormous single payloads?
Without knowing more about the application, I'd guess probably caching and/or scaling. If you only need 1 payload then that can be statically generated and cached in your CDN. Which in turn reduces your dependence on the web servers so few nodes are required and/or you can scale your site more easily to demand. Also compute time is more expensive than CDN costs so there might well be some cost savings there too.
This was basically it. The dataset was the same across users so caching was simple and efficient, and the front-end had no difficulty handling this much data (and paging client-side was snappier than requesting anew each time)