Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Ldump – serialize any Lua data

github.com

Post reply on HN
Print view View original

31–40 of 44 posts

Re: Show HN: Ldump – serialize any Lua data

#31

Post by girvel »

myrmidon wrote: post #28
jhatemyjob wrote: post #27

Try running this in a repl and tell me what you get, OP: string.format('%q', 'hi\n')

If you insinuate that %q obviates the need for ldump then you are wrong. There is not even significant overlap in what they do; all that %q does is sufficiently escape Lua strings so the interpreter can read them back. It does not serialize functions nor even tables in any shape or form. edit: Sorry for being unreasonably harsh after misunderstanding your message.

I actually thought the comment was about ldump implementation: it uses %q to serialize strings, and it may not be a reliable way.

Re: Show HN: Ldump – serialize any Lua data

#32

Post by girvel »

nicoloren wrote: post #19
JourneyJourney wrote: post #16

I'm afraid I spent too much time with LUA lately and fell in love with its simplicity. Kinda hard to go back to OOP after that.

Same for me, I used Lua for a desktop software for a client and I enjoyed it a lot! I'm thinking of starting to dev a game with LOVE2D just to have an excuse to use Lua.

LOVE2D is a great gamedev framework, I can not recommend it enough. It is so pleasant to work with.

Re: Show HN: Ldump – serialize any Lua data

#33

Post by girvel »

girvel wrote: post #13
lifthrasiir wrote: post #9

Maybe I'm too pedantic but allowing anything to be "deserialized", which equals to "evaluated" here, is not secure. I think it only has to accept a very limited subset of Lua anyway, so you may switch to a non-Lua format which is made easy to parse. That way the library has a total control over what is being evaluated.

This is an interesting thought. Currently, it is unsafe and intended to load only the files you trust. I should definitely include a warning into README. Overall, it would be nice to make it safer. I don't think switching to non-Lua format would make it safer, because it is intended to serialize functions too, which can have arbitrary code even if everything else would be stored as data. Maybe it is possible to make…

I looked into it, and Lua allows limiting the environment when `load`ing -- through `env` argument since 5.2 or through setfenv before. I will add a helper function to produce a minimal needed environment for safe loading and a documentation page about safety.

Re: Show HN: Ldump – serialize any Lua data

#34

Post by jhatemyjob »

girvel wrote: post #29
jhatemyjob wrote: post #27

Try running this in a repl and tell me what you get, OP: string.format('%q', 'hi\n')

On my machine it produces an equivalent string, although differently formatted. It seems that ldump preserves all special characters (`"\a\b\f\n\r\t\v\\\"\'"`), although I will need to test in on all supported versions.

Ah, you know what, you're right. It's an equivalent string for me too:

    "hi\
    "
I didn't know Lua treated \ before newlines like that. That's cool! I made a similar Lua serialization library for myself and was using a chain of `string.match` calls to escape my strings. Now I can make it way simpler. Lol. Thanks

Re: Show HN: Ldump – serialize any Lua data

#35

Post by myrmidon »

girvel wrote: post #33
girvel wrote: post #13

Earlier quoted context omitted.

This is an interesting thought. Currently, it is unsafe and intended to load only the files you trust. I should definitely include a warning into README. Overall, it would be nice to make it safer. I don't think switching to non-Lua format would make it safer, because it is intended to serialize functions too, which can have arbitrary code even if everything else would be stored as data. Maybe it is possible to make…

I looked into it, and Lua allows limiting the environment when `load`ing -- through `env` argument since 5.2 or through setfenv before. I will add a helper function to produce a minimal needed environment for safe loading and a documentation page about safety.

Note that loading (maliciously crafted) bytecode is generally not safe in Lua; sandboxing can be escaped in more ways than what's possible when loading plaintext sourcecode, and there are no full mitigations for this currently as far as I know (and would probably be highly interpreter/version sensitive anyway)-- the only "real" mitigation strategy is to just not `load` bytecode at all.

But this is probably a non-issue for a lot of usecases.

See e.g.

https://gist.github.com/corsix/6575486

https://www.corsix.org/content/malicious-luajit-bytecode

Re: Show HN: Ldump – serialize any Lua data

#37

Post by koeng »

I've been looking for something similar! Here is what I'd like to do:

I have a long-running script. At several steps, the execution of the script has to pause for a long time for operations to be done in-real-life (biological experiments, so think wait time being like 2 days between running), before getting some data and continuing to run. From what I can see in this, I'd add yielding coroutines at data pause points, right? How would you handle that?

Re: Show HN: Ldump – serialize any Lua data

#39

Post by GranPC »

appleorchard46 wrote: post #38

Very cool! I was just needing something like this for my Defold game, this looks way better than my hacky solution. Semi-unrelated - you say you're using tables as keys in your project. I didn't know you could do that! What are you using it for?

FWIW anything in Lua can be used as a key - including functions, userdata, etc.

Re: Show HN: Ldump – serialize any Lua data

#40

Post by girvel »

myrmidon wrote: post #35
girvel wrote: post #33

Earlier quoted context omitted.

I looked into it, and Lua allows limiting the environment when `load`ing -- through `env` argument since 5.2 or through setfenv before. I will add a helper function to produce a minimal needed environment for safe loading and a documentation page about safety.

Note that loading (maliciously crafted) bytecode is generally not safe in Lua; sandboxing can be escaped in more ways than what's possible when loading plaintext sourcecode, and there are no full mitigations for this currently as far as I know (and would probably be highly interpreter/version sensitive anyway)-- the only "real" mitigation strategy is to just not `load` bytecode at all. But this is probably a non-issu…

This is fascinating. I wonder if this issue exists in Lua5.2+, where there is no jit and `load` is able to restrict used environment.
Post reply on HN