Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Ldump – serialize any Lua data

github.com

Post reply on HN
Print view View original

1–10 of 44 posts

Show HN: Ldump – serialize any Lua data

#1

Post by girvel »

Some time ago, I was implementing saves for my LOVE2D game. I wanted to do a full dump of the game state -- which included closures (AI), complex graphs, sets with tables as keys and also fundamentally non-serializable data (coroutines and userdata), that require user-defined serialization/deserialization logic. I went through every Lua serialization library -- none covered all data types/cases. So I wrote my own.

It is a polished version, thoroughly annotated, tested and documented. It is made to be as functional and customizable as possible (or at least I did everything I could think of). I would be happy to hear suggestions/corrections for both code and documentation -- even nitpicky ones.

Show HN: Ldump – serialize any Lua data
github.com

Re: Show HN: Ldump – serialize any Lua data

#6

Post by sebstefan »

So it also dumps functions and is able to import them back?

Does the function still need to be in memory to be loaded again ("does it just dump the pointer") or can I save it to disk, shut off the interpreter, boot it again and it imports it fine (in which case it somehow dumps them as code...?)?

Even in the linked test case on the readme you don't show the output/expectation of the serialization

Re: Show HN: Ldump – serialize any Lua data

#7

Post by lifthrasiir »

sebstefan wrote: post #6

So it also dumps functions and is able to import them back? Does the function still need to be in memory to be loaded again ("does it just dump the pointer") or can I save it to disk, shut off the interpreter, boot it again and it imports it fine (in which case it somehow dumps them as code...?)? Even in the linked test case on the readme you don't show the output/expectation of the serialization

Functions are apparently serialized as a bytecode dump contained in a self-extracting expression. So everything can indeed be serialized as a Lua expression. Seems that the author also tried to preserve as many upvalues as possible, though I feel that is way more dangerous than I would like.

Re: Show HN: Ldump – serialize any Lua data

#9

Post by lifthrasiir »

Maybe I'm too pedantic but allowing anything to be "deserialized", which equals to "evaluated" here, is not secure. I think it only has to accept a very limited subset of Lua anyway, so you may switch to a non-Lua format which is made easy to parse. That way the library has a total control over what is being evaluated.

Re: Show HN: Ldump – serialize any Lua data

#10

Post by girvel »

sebstefan wrote: post #6

So it also dumps functions and is able to import them back? Does the function still need to be in memory to be loaded again ("does it just dump the pointer") or can I save it to disk, shut off the interpreter, boot it again and it imports it fine (in which case it somehow dumps them as code...?)? Even in the linked test case on the readme you don't show the output/expectation of the serialization

The function (even a closure) would be fully recreated on deserialization, it is fully safe to save it to disk. It wouldn't preserve reference equality -- it would be a new function -- but the behaviour and the state (if using closures) would be equivalent.

I didn't include asserts in the linked case, because I thought it would be too verbose. You can see asserts in the test, that is linked below the example. Maybe it was the wrong call, I will think about including asserts into the example itself.

Post reply on HN