Cool! Could this be used to make on-disk DataFrames that store arbitrary Julia types? Serializing/deserializing such dataframes with JLD has given me lots of trouble (both too slow and also sometimes the files get corrupted in ways that can't be debugged.)
JuliaDB already does this eg `loadtable("foo.csv", output="foo/")` will store the data in "foo/" in binary form and mmap it.
Re: Zero-copy deserialization in Julia
#31Thanks for the tip Jamii, and for the nice original post.
This is pretty close to what I've been doing, but since I'm storing Julia values rather than numeric data I still to need to convert my data to strings to store in "foo.csv" and then parse back from strings when reading in individual values using `loadtable("foo.csv", output="foo/", colparsers=[parserfortype1, parserfortype2, ...])`. As a parser for nested Julia types is fairly complicated (structs containing dictionaries as fields which contain nested arrays as values which contain ...), it'd be nice to go directly to the binary form used by JuliaDB without converting to/from the string intermediate representation needed by the csv format. Perhaps Blob isn't quite the abstraction needed, but if such a thing exists I'd love to know about it!