Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pstore: Ruby Built-In Hash Persistence

github.com

Post reply on HN
Print view View original

31–40 of 42 posts

Re: Pstore: Ruby Built-In Hash Persistence

#31

Post by 3pt14159 »

brunno wrote: post #25
e12e wrote: post #15

Earlier quoted context omitted.

> use the Ox gem The main thing is that it's part of the standard library. If you import a gem anyway, often you'd be well off with sqlite. As for storage format, there's also: https://ruby-doc.org/stdlib-3.1.2/libdoc/yaml/rdoc/YAML/Stor...

I love the simplicity of YAML::Store. It was introduced in Ruby 1.8, almost 20 years ago ( https://github.com/ruby/ruby/commit/55f4dc4c9a5345c28d0da750... ). I even created a little gem when I was starting with Ruby, 10 years ago, that was a very thin wrapper around it so that I could play around using an ActiveRecord like syntax ( https://github.com/brunnogomes/active_yaml ). I used in some pet projects so I could d…

The problem with YAML is that meaningful whitespace means that the size grows quickly for highly nested documents. I don't love XML, but there is a reason I recommended Ox. I've used it for real projects and it never fell over like so many of the alternatives I've tried where databases were not in the cards.

Re: Pstore: Ruby Built-In Hash Persistence

#32

Post by mattnewton »

fny wrote: post #22
cdiamand wrote: post #9

Earlier quoted context omitted.

There have been some interesting ML gems rolled in the past few years: https://ankane.org/new-ml-gems Any thoughts on what the Ruby community would need to build in order for it to become an attractive tool for AI work?

A huge cultural shift. People in scientific computing speak Python and R. Something would need to happen that makes Ruby far more attractive. Say performance parity with Crystal or Nim.

I think it’s more than that, Julia exists and adoption is still slow. Lua and torch were plenty fast and they were still replaced by pytorch. I think to compete with python you need at least a fraction of the de-facto corporate sponsorship for python in the ML space.

Re: Pstore: Ruby Built-In Hash Persistence

#35

Post by fny »

3pt14159 wrote: post #31
brunno wrote: post #25

Earlier quoted context omitted.

I love the simplicity of YAML::Store. It was introduced in Ruby 1.8, almost 20 years ago ( https://github.com/ruby/ruby/commit/55f4dc4c9a5345c28d0da750... ). I even created a little gem when I was starting with Ruby, 10 years ago, that was a very thin wrapper around it so that I could play around using an ActiveRecord like syntax ( https://github.com/brunnogomes/active_yaml ). I used in some pet projects so I could d…

The problem with YAML is that meaningful whitespace means that the size grows quickly for highly nested documents. I don't love XML, but there is a reason I recommended Ox. I've used it for real projects and it never fell over like so many of the alternatives I've tried where databases were not in the cards.

The problem with XML is that angle bracket expressions take up too much space because you need to duplicate element names. I don't love JSON, but there is a reason I recommend OJ.

...

The problem with JSON is that the keys take up too much space because they are duplicated. I don't love BSON, but there's a reason why I recommend bson-ruby.

And I could keep going... ;)

The benefit of using YAML is precisely that there's meaningful whitespace. Different strokes for different folks.

Re: Pstore: Ruby Built-In Hash Persistence

#36

Post by nurettin »

woodruffw wrote: post #12
jrochkind1 wrote: post #10

Earlier quoted context omitted.

> too many issues Such as?

Marshal is Ruby's version of pickle in Python: it serializes arbitrary objects, which means that correct deserialization requires arbitrary code execution. This is bad enough on its own, but it also makes pivoting a file read/write primitive into code execution much easier.

Why the "don't use it"? Just say "use it with caution" or, since we are being rude telling people what to do whenever pickle or marshal comes up, just don't say anything and assume people know what they are doing.

Re: Pstore: Ruby Built-In Hash Persistence

#37

Post by vr46 »

3pt14159 wrote: post #8

Don't use this. Marshal has too many issues. If you really need persistence and can't use something like Postgres, use the Ox gem instead. It's more reliable between versions of Ruby and easier to parse from other languages if you ever have to.

Is Marshal still tied to Ruby version? Boy was this fun about ten years ago for a system I inherited that Marshaled huge complex objects into TokyoTyrant and back. You try migrating or upgrading a system where the runtime version is tied to EVERY object in a database.

Re: Pstore: Ruby Built-In Hash Persistence

#38

Post by iovrthoughtthis »

prescriptivist wrote: post #7
fny wrote: post #3

I do a lot of ML and AI work nowadays... I miss Ruby a lot especially the its culture around ergonomics.

I recently had the need to build an internal system that distributed workloads across many workers via a client/server model. I did the proof-of-concept using druby [1] and it turned out to be so simple and stable that we just ran with it. It'd been years since I had used that library and instinctively I assumed we'd get the prototype out and then rebuild it using some sort of web service and utilize a high concurren…

how have i never seen this!! :D

Re: Pstore: Ruby Built-In Hash Persistence

#39

Post by Fire-Dragon-DoL »

e12e wrote: post #15
3pt14159 wrote: post #8

Don't use this. Marshal has too many issues. If you really need persistence and can't use something like Postgres, use the Ox gem instead. It's more reliable between versions of Ruby and easier to parse from other languages if you ever have to.

> use the Ox gem The main thing is that it's part of the standard library. If you import a gem anyway, often you'd be well off with sqlite. As for storage format, there's also: https://ruby-doc.org/stdlib-3.1.2/libdoc/yaml/rdoc/YAML/Stor...

I don't get the value of "it's in the standard library". Ruby has the amazing (fir scripts) require "bundler/inline" that allows you to use a single file for code and Gemfile, as well as auto installing the dependencies, so going for standard library doesn't seem to provide any practical value except offline support

Re: Pstore: Ruby Built-In Hash Persistence

#40

Post by woodruffw »

nurettin wrote: post #36
woodruffw wrote: post #12

Earlier quoted context omitted.

Marshal is Ruby's version of pickle in Python: it serializes arbitrary objects, which means that correct deserialization requires arbitrary code execution. This is bad enough on its own, but it also makes pivoting a file read/write primitive into code execution much easier.

Why the "don't use it"? Just say "use it with caution" or, since we are being rude telling people what to do whenever pickle or marshal comes up, just don't say anything and assume people know what they are doing.

I don't think I phrased that in a particularly rude way, but I'm sorry if it came across as rude.

The answer is that we have serialization techniques that are as good on all the dimensions that matter (speed, serialized size, etc.) and better in terms of security. Pickle and Marshal are, at best, footguns in otherwise very safe language ecosystems.

Post reply on HN