Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Glom – Restructured Data for Python

sedimental.org

Post reply on HN
Print view View original

21–30 of 99 posts

Re: Glom – Restructured Data for Python

#21

Post by mhashemi »

mhashemi wrote: post #12
hessammehr wrote: post #10

This might have been unintentional, but I suspect "Spectre of Structure" and "Python's missing piece" refer to Nathan Marz's specter library for clojure [1], similarly touted as clojure's missing piece. I tend to agree in the case of specter, given the mind-boggling types of transformations that are easily (and simply) expressed in it (and often run faster than idiomatic clojure as well). Highly recommended if you ev…

Total coincidence! Reading the README, Nathan and I are definitely on the same wavelength though. When I get a chance I'll add it to the analogies doc: http://glom.readthedocs.io/en/latest/by_analogy.html :)

What's really interesting is that we're approaching the same ideal state from different directions. Specter goes from Clojure's immutability to something more practical, from Python's super dynamic system to something more declarative and immutable.

Re: Glom – Restructured Data for Python

#23

Post by doublereedkurt1 »

nerdponx wrote: post #22

It's a nice idea, but i never like writing what amounts to a DSL in strings in my code (yes, that applies to in-code SQL as well, although that's often unavoidable). I prefer the `get_in()` method from Toolz: http://toolz.readthedocs.io/en/latest/api.html#toolz.dicttoo...

the only string parsing is 'a.b.c', which is mostly equivalent to T.a.b.c so you can completely ignore that capability

the very slight difference is that using T you must be explicit about attribute access vs key access whereas 'a.b.c' will try both

Re: Glom – Restructured Data for Python

#25

Post by radmin »

nerdponx wrote: post #22

It's a nice idea, but i never like writing what amounts to a DSL in strings in my code (yes, that applies to in-code SQL as well, although that's often unavoidable). I prefer the `get_in()` method from Toolz: http://toolz.readthedocs.io/en/latest/api.html#toolz.dicttoo...

My thoughts also turned to toolz.

Here's a example comparison:

glom

  glom(target, ('system.planets', ['name']))
  # ['earth', 'jupiter']
toolz

  list(pluck('name', get_in(('system', 'planets'), target)))
  # ['earth', 'jupiter']

Re: Glom – Restructured Data for Python

#26

Post by sixdimensional »

I had a quick look, but I didn't see filtering expressions, only shaping expressions. It seems like glom is more of a result shaper/mapper. Can you filter with glom (maybe with lambdas or something)? I could see the two going together quite well if you were "glomming" a big Python object.

Re: Glom – Restructured Data for Python

#27

Post by mhashemi »

radmin wrote: post #25
nerdponx wrote: post #22

It's a nice idea, but i never like writing what amounts to a DSL in strings in my code (yes, that applies to in-code SQL as well, although that's often unavoidable). I prefer the `get_in()` method from Toolz: http://toolz.readthedocs.io/en/latest/api.html#toolz.dicttoo...

My thoughts also turned to toolz. Here's a example comparison: glom glom(target, ('system.planets', ['name'])) # ['earth', 'jupiter'] toolz list(pluck('name', get_in(('system', 'planets'), target))) # ['earth', 'jupiter']

The real advantage over toolz/get_in is in the breadth of types glom can support (http://glom.readthedocs.io/en/latest/api.html#setup-and-regi...), the extensive fallback behavior (http://glom.readthedocs.io/en/latest/api.html#advanced-speci...), and "T" specifier (https://sedimental.org/glom_restructured_data.html#true-pyth...), which allows performing object-oriented traversals and calls.

Check the post for more examples of where path-based access falls short of the mark!

(Sidenote: I find it very clunky that get_in() takes a "default" kwarg and _also_ a n"o_default" kwarg. Use a Sentinel object! http://boltons.readthedocs.io/en/latest/typeutils.html#bolto... )

Re: Glom – Restructured Data for Python

#28

Post by mhashemi »

sixdimensional wrote: post #26

I had a quick look, but I didn't see filtering expressions, only shaping expressions. It seems like glom is more of a result shaper/mapper. Can you filter with glom (maybe with lambdas or something)? I could see the two going together quite well if you were "glomming" a big Python object.

Filtering is supported through the OMIT value: http://glom.readthedocs.io/en/latest/api.html#glom.OMIT (another example: http://glom.readthedocs.io/en/latest/snippets.html#filtered-... )

Lambdas and functions are always a safe fallback, but glom does its best to keep your specs readable and roundtrippable (gotta love a nice repr()).

Re: Glom – Restructured Data for Python

#29

Post by abuckenheimer »

nerdponx wrote: post #22

It's a nice idea, but i never like writing what amounts to a DSL in strings in my code (yes, that applies to in-code SQL as well, although that's often unavoidable). I prefer the `get_in()` method from Toolz: http://toolz.readthedocs.io/en/latest/api.html#toolz.dicttoo...

I feel the same way, although my instinct is generally to build a custom generator. Only costs a couple lines but is plain old python and quite explicit

    target = {'system': {'planets': [{'name': 'earth', 'moons': 1},
                                     {'name': 'jupiter', 'moons': 69}]}}

    glom(target, {'moon_count': ('system.planets', ['moons'], sum)})
    # vs
    def iter_moons(t):
        for planet in target['system']['planets']:
            yield planet['moons']

    sum(iter_moons(target))
would have to combine with `defaultdict`s if your nested data is only sometimes there though

Re: Glom – Restructured Data for Python

#30

Post by jasonpeacock »

mhashemi wrote: post #20
jasonpeacock wrote: post #19

I'm probably being dense, but I don't see a good description of the input data types supported - the CLI says "json or python". It would be great to have clarification if this is JSON only, or supports other data structures, or parsers could be plugged in?

From within Python, all objects are supported by default. If you can parse it, you can glom it. You can even register additional behaviors for specific types to keep your specs tight: http://glom.readthedocs.io/en/latest/api.html#setup-and-regi... (example: http://glom.readthedocs.io/en/latest/snippets.html#automatic... ) The CLI is in a pretty preliminary state, usable but not as robust as it will be in a few weeks.…

Aha. Thanks! I understand better now, I didn't realize it was so general-purpose :)
Post reply on HN