I do a lot of ML and AI work nowadays... I miss Ruby a lot especially the its culture around ergonomics.
It keeps me coming back.
Ruby itself is just such an enabler.
Pstore: Ruby Built-In Hash Persistence
11–20 of 42 posts
Re: Pstore: Ruby Built-In Hash Persistence
#12Don't use this. Marshal has too many issues. If you really need persistence and can't use something like Postgres, use the Ox gem instead. It's more reliable between versions of Ruby and easier to parse from other languages if you ever have to.
> too many issues Such as?
Marshal is Ruby's version of pickle in Python: it serializes arbitrary objects, which means that correct deserialization requires arbitrary code execution.
This is bad enough on its own, but it also makes pivoting a file read/write primitive into code execution much easier.
Re: Pstore: Ruby Built-In Hash Persistence
#13pstore has been a built-in with Ruby stdlib for as long as ruby has existed, so _over_ 20 years.
I'm assuming it pre-dates Rubygems because it really should be a gem. I can't speak for Japan but few people in the Western world seem to use it.
Re: Pstore: Ruby Built-In Hash Persistence
#14I do a lot of ML and AI work nowadays... I miss Ruby a lot especially the its culture around ergonomics.
There have been some interesting ML gems rolled in the past few years: https://ankane.org/new-ml-gems Any thoughts on what the Ruby community would need to build in order for it to become an attractive tool for AI work?
My guess is some kind of corporate sponsorship. Someone with deep pockets to maintain it, encourage new apis keeping up with the latest papers, and make sure it works out of the box with the accelerator people want to use this month.
Re: Pstore: Ruby Built-In Hash Persistence
#15Don't use this. Marshal has too many issues. If you really need persistence and can't use something like Postgres, use the Ox gem instead. It's more reliable between versions of Ruby and easier to parse from other languages if you ever have to.
> use the Ox gem
The main thing is that it's part of the standard library. If you import a gem anyway, often you'd be well off with sqlite.
As for storage format, there's also:
https://ruby-doc.org/stdlib-3.1.2/libdoc/yaml/rdoc/YAML/Stor...
Re: Pstore: Ruby Built-In Hash Persistence
#16pstore has been a built-in with Ruby stdlib for as long as ruby has existed, so _over_ 20 years.
Pstore also uses Marshal behind the scenes, so I assume has similar caveats you see in other comments on this thread.
Re: Pstore: Ruby Built-In Hash Persistence
#17I do a lot of ML and AI work nowadays... I miss Ruby a lot especially the its culture around ergonomics.
I recently had the need to build an internal system that distributed workloads across many workers via a client/server model. I did the proof-of-concept using druby [1] and it turned out to be so simple and stable that we just ran with it. It'd been years since I had used that library and instinctively I assumed we'd get the prototype out and then rebuild it using some sort of web service and utilize a high concurren…
drb is awesome. I've had the good fortune to be able to use it once. The simplicity of it compared to anything else is amazing.
Re: Pstore: Ruby Built-In Hash Persistence
#18Earlier quoted context omitted.
> too many issues Such as?
Marshal is Ruby's version of pickle in Python: it serializes arbitrary objects, which means that correct deserialization requires arbitrary code execution. This is bad enough on its own, but it also makes pivoting a file read/write primitive into code execution much easier.
https://github.com/ruby/psych defaults to only loading permitted classes since 4.0 so that seems less of a concern now?
Re: Pstore: Ruby Built-In Hash Persistence
#19Earlier quoted context omitted.
Marshal is Ruby's version of pickle in Python: it serializes arbitrary objects, which means that correct deserialization requires arbitrary code execution. This is bad enough on its own, but it also makes pivoting a file read/write primitive into code execution much easier.
https://github.com/ruby/psych defaults to only loading permitted classes since 4.0 so that seems less of a concern now?
`psych`, used for YAML, is a different thing than Marshal. pstore uses Marshal. https://ruby-doc.org/core-2.6.3/Marshal.html. I don't believe psych will be involved with pstore.
I'm honestly not sure, though, how much I should be worried about the fact that someone who has write access to my database can maybe escalate that to an arbitrary code execution if I use pstore. Literally not sure. Write access to my DB seems pretty disastrous already...
Re: Pstore: Ruby Built-In Hash Persistence
#20I do a lot of ML and AI work nowadays... I miss Ruby a lot especially the its culture around ergonomics.
There have been some interesting ML gems rolled in the past few years: https://ankane.org/new-ml-gems Any thoughts on what the Ruby community would need to build in order for it to become an attractive tool for AI work?
As a primarily Ruby dev I'd prefer the AI/ML ecosystem not be split-brained between two languages that are semantically 90% the same thing. Just learn Python and integrate the models into your Rails (or whatever) apps.