Live data from Hacker News

Graph Mining Library

github.com

71–80 of 113 posts

Re: Graph Mining Library

#71

Whew. Lots of complaints from people who probably will never need to use this code. If you need docs just read the .h files, they have extensive comments. I’m sure they’ll add them or maybe, just maybe, you could write some to contribute. This would have made some of my previous work much easier, it’s really nice to see google open source this.

> If you need docs just read the .h files

curious if this is typical dev experience inside google..

Re: Graph Mining Library

#72
post #54

Graph mining was "so hot right now" ten years ago. Remember GraphX ( https://spark.apache.org/graphx/ ) and GraphLab ( https://en.wikipedia.org/wiki/GraphLab ) ? Or graph databases? I guess it coincided with the social network phenomenon. Much more recently geometric learning (ML on graphs and other structures) shone, until LLMs stole their thunder. I still think geometric learning has a lot of life left in it, and I…

There are "graph databases" which see graphs as a universal approach to data, see RDF and SPARQL and numerous pretenders. For that matter, think of a C program where the master data structure is a graph of pointers. In a graph like that there is usually a huge number of different edge types such as "is married to", "has yearly average temperature", ... Then there are "graph algorithms" such as PageRank, graph central…

1. Graph algorithms like the ones you mentioned are processed not by graph databases like Neo4j, but graph processing libraries like the titular Google library.

2. Geometric learning is the broader category that subsumes graph neural networks.

https://geometricdeeplearning.com/

Re: Graph Mining Library

#73

Can someone with familiarity with Bazel give any clues how to build? `bazel build` does something, but I end up with `bazel-build` and `bazel-build` with no obvious build artefacts.

Interesting. I always had in back of my mind this notion that I ought to check bazel one of these days. So, one of these days is then today. In order to install bazel, recommended way seems to be to install bazelisk first and just rename that to bazel and move it somewhere on the path like /usr/local/bin/bazel.. fine. Now when I run query it warned me about JDK.. huh. Now when I run build it errored and failed due to missing JAVA with "WARNING: Ignoring JAVA_HOME, because it must point to a JDK, not a JRE.". Ok, I'm not using Java - let's check which Java JDK/JRE to use these days and after few minutes of googling I'm not up for it anymore and that, ladies and gentlemen, is where this day is then up for another day after all. Pathetic how cargo and even npm/yarm spoiled us.

edit: thanks to https://sdkman.io/ it's up and running. It wasn't _that_ bad after all.

Re: Graph Mining Library

#74

Whew. Lots of complaints from people who probably will never need to use this code. If you need docs just read the .h files, they have extensive comments. I’m sure they’ll add them or maybe, just maybe, you could write some to contribute. This would have made some of my previous work much easier, it’s really nice to see google open source this.

> If you need docs just read the .h files curious if this is typical dev experience inside google..

I think in most cases, back when I worked there, I would have instead searched the monorepo for targets that depended on this library (an easy lookup), and look at how they used it.

Some code libraries had excellent docs (recordio, sstable, mapreduce). But yes, reading the header file was often the best place to start.

Re: Graph Mining Library

#75

Earlier quoted context omitted.

I'd say both of those are actively harmful products (like PFOS or cigarettes) that hurt Google's competition by being open sourced. Google wrecked their own productivity, the least they could do was wreck everybody else's.

And why would any of those be harmful? Care to elaborate?

They take a process a small team could complete quickly with high quality and low cost maintenance and turn it into a process a huge team completes slowly with poor quality and high maintenance cost. Google can afford this because of huge profits from their advertising monopoly that they don’t know how to spend.

Go look at the manuals for IBM's Parallel Sysplex for mainframes and compare the simplicity of that to K8S for instance.

Or for that matter look at DCOM and the family of systems which Microsoft built around it which are truly atrocious but look like a model of simplicity compared to gRPC. (At least Don Box wrote a really great book about COM that showed people in the Microsoftsphere how to write good documentation.)

Or for that matter try doing something with AWS, Azure or some off-brand cloud and Google Cloud from zero (no account) and time yourself with a stopwatch. Well, it will be a stopwatch for AWS but you will probably need a calendar for Google Cloud.

Re: Graph Mining Library

#76
post #10

Github says it is C, C++, and Starland. What is Starland ?

It's Starlark, the language for configuring the build system Bazel. Bazel is the open source port of Google's internal build system, Blaze. Starlark is a subset of Python.

This list of corporate project name associations makes me wonder where Galactus comes in. :P

https://www.youtube.com/watch?v=y8OnoxKotPQ

Re: Graph Mining Library

#77

Earlier quoted context omitted.

I’m not saying it’s too much to ask for, but also, when you’re doing distributed in memory graph mining (which means you’ve got an application with a big enough graph that you need to do this, and the technical expertise to need the algorithms in this open source package) maybe it’s expected that you can read the bazel files and bazel docs yourself and figure it out. Or just write a make file and cut all the bazel bu…

Yeah, and somebody who is that smart can probably pack their data structures efficiently and find an approximation to do the job on a macbook pro that people with too many resources need a 1000 machine cluster to do. And get the coding and the computation done in the time that the C++ compiler is still chewing on the headers of the bloated library. (At times I’ve been that guy.) But seriously, there is such a thing a…

I agree with all of this.

Its just frustrating that all the comments about an interesting library seem to be customer service complaints from people who never need to reach for this library. I was hoping for a real discussion, something I could learn from.

Re: Graph Mining Library

#78

Whew. Lots of complaints from people who probably will never need to use this code. If you need docs just read the .h files, they have extensive comments. I’m sure they’ll add them or maybe, just maybe, you could write some to contribute. This would have made some of my previous work much easier, it’s really nice to see google open source this.

> If you need docs just read the .h files curious if this is typical dev experience inside google..

I’m not at google so I’ve got no idea.

Reading the code, especially the header files, seems to be pretty standard as far as what I see in non-open source code. So, it’s been my typical dev experience, I’d say if you’re somewhere that has gleaming, easy to understand docs that are actually up to date with the code you all have too much time on your hands, but I serially work at startups that are running to market.

Re: Graph Mining Library

#79
post #54

Graph mining was "so hot right now" ten years ago. Remember GraphX ( https://spark.apache.org/graphx/ ) and GraphLab ( https://en.wikipedia.org/wiki/GraphLab ) ? Or graph databases? I guess it coincided with the social network phenomenon. Much more recently geometric learning (ML on graphs and other structures) shone, until LLMs stole their thunder. I still think geometric learning has a lot of life left in it, and I…

[deleted]

Re: Graph Mining Library

#80
post #54

Graph mining was "so hot right now" ten years ago. Remember GraphX ( https://spark.apache.org/graphx/ ) and GraphLab ( https://en.wikipedia.org/wiki/GraphLab ) ? Or graph databases? I guess it coincided with the social network phenomenon. Much more recently geometric learning (ML on graphs and other structures) shone, until LLMs stole their thunder. I still think geometric learning has a lot of life left in it, and I…

[deleted]
Post reply on HN