Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Ldump – serialize any Lua data

github.com

Post reply on HN
Print view View original

21–30 of 44 posts

Re: Show HN: Ldump – serialize any Lua data

#21

Post by gvx »

girvel wrote: post #13
lifthrasiir wrote: post #9

Maybe I'm too pedantic but allowing anything to be "deserialized", which equals to "evaluated" here, is not secure. I think it only has to accept a very limited subset of Lua anyway, so you may switch to a non-Lua format which is made easy to parse. That way the library has a total control over what is being evaluated.

This is an interesting thought. Currently, it is unsafe and intended to load only the files you trust. I should definitely include a warning into README. Overall, it would be nice to make it safer. I don't think switching to non-Lua format would make it safer, because it is intended to serialize functions too, which can have arbitrary code even if everything else would be stored as data. Maybe it is possible to make…

You could take a look at SELÖVE, a (severely out of date) fork of LÖVE that is intended to make it safe to run arbitrary .love games. (It used to be on bitbucket, but it looks like it's gone? I'm not sure if I have the repo locally :/)

Running arbitrary code was such a problem that I just completely ruled it out for bitser. Instead of serializing functions, you can register safe functions as resources. This doesn't solve the upvalue problem, though.

Re: Show HN: Ldump – serialize any Lua data

#23

Post by sebstefan »

girvel wrote: post #10
sebstefan wrote: post #6

So it also dumps functions and is able to import them back? Does the function still need to be in memory to be loaded again ("does it just dump the pointer") or can I save it to disk, shut off the interpreter, boot it again and it imports it fine (in which case it somehow dumps them as code...?)? Even in the linked test case on the readme you don't show the output/expectation of the serialization

The function (even a closure) would be fully recreated on deserialization, it is fully safe to save it to disk. It wouldn't preserve reference equality -- it would be a new function -- but the behaviour and the state (if using closures) would be equivalent. I didn't include asserts in the linked case, because I thought it would be too verbose. You can see asserts in the test, that is linked below the example. Maybe i…

That's super cool

I think you could make it clearer, try reading the readme as someone with the preconceived notion that this is Yet Another Lua Serializer that translates functions, userdata and threads to their tostring() output. There are hundreds of those projects

Re: Show HN: Ldump – serialize any Lua data

#24

Post by elpocko »

girvel wrote: post #15
myrmidon wrote: post #12

Earlier quoted context omitted.

> can I save it to disk, shut off the interpreter, boot it again and it imports it fine (in which case it somehow dumps them as code...? Yes, it dumps them as bytecode (probably not compatible between completely different interpreters). It even preserves debug metadata, so stack traces involving serialized/deserialized functions look right , and still show the original source file. This is really neat.

Thank you, it is really nice to hear. Though, I have to give credit to Lua's standard library -- the basic function serialization (without upvalues) is implemented there as `string.dump`.

Be aware that you're gonna have a bad time in scenarios where code is serialized using one Lua version and deserialized using another. Bytecode compatibility is not guaranteed between different versions of Lua(JIT).

I've shipped Love2D games as bytecode that wouldn't run on many Linux boxes because their LuaJIT installation (which is not part of Love2D but part of the system) was too old, or they stopped working after the user updated their system. There's a plethora of situations where something like that can happen.

I'm also wary of the "upvalues are preserved" feature, which sounds like a huge footgun, but I haven't looked into the details of your implementation.

Re: Show HN: Ldump – serialize any Lua data

#28

Post by myrmidon »

jhatemyjob wrote: post #27

Try running this in a repl and tell me what you get, OP: string.format('%q', 'hi\n')

If you insinuate that %q obviates the need for ldump then you are wrong.

There is not even significant overlap in what they do; all that %q does is sufficiently escape Lua strings so the interpreter can read them back. It does not serialize functions nor even tables in any shape or form.

edit: Sorry for being unreasonably harsh after misunderstanding your message.

Re: Show HN: Ldump – serialize any Lua data

#29

Post by girvel »

jhatemyjob wrote: post #27

Try running this in a repl and tell me what you get, OP: string.format('%q', 'hi\n')

On my machine it produces an equivalent string, although differently formatted. It seems that ldump preserves all special characters (`"\a\b\f\n\r\t\v\\\"\'"`), although I will need to test in on all supported versions.
Post reply on HN