Can someone explain what this library might be useful for?
Graph Mining Library
81–90 of 113 posts
Re: Graph Mining Library
#82Earlier quoted context omitted.
There are "graph databases" which see graphs as a universal approach to data, see RDF and SPARQL and numerous pretenders. For that matter, think of a C program where the master data structure is a graph of pointers. In a graph like that there is usually a huge number of different edge types such as "is married to", "has yearly average temperature", ... Then there are "graph algorithms" such as PageRank, graph central…
1. Graph algorithms like the ones you mentioned are processed not by graph databases like Neo4j, but graph processing libraries like the titular Google library. 2. Geometric learning is the broader category that subsumes graph neural networks. https://geometricdeeplearning.com/
Depends, some graph databases have some support for graph algorithms.
I’ll also say I think graph algorithms are overrated, I mean you know the diameter of some graph: who cares? Physicists (like me back in the day) are notorious for plotting some statistics on log-log paper, seeing that the points sorta kinda fall on a line if you squint and decide that three of the points are really bug splats and then yelling “UNIVERSIALITY” and sending it to Physical Review E but the only thing that is universal is that none of them have ever heard of a statistical estimator or a significance test for power law distributions. Node 7741 is the “most central” node, but does that make a difference? Maybe if you kill the top 1% central nodes that will disrupt the terrorist network but for most of us I don’t see high quality insights coming out of graph algorithms most of the time.
Re: Graph Mining Library
#83Earlier quoted context omitted.
1. Graph algorithms like the ones you mentioned are processed not by graph databases like Neo4j, but graph processing libraries like the titular Google library. 2. Geometric learning is the broader category that subsumes graph neural networks. https://geometricdeeplearning.com/
Depends, some graph databases have some support for graph algorithms. I’ll also say I think graph algorithms are overrated, I mean you know the diameter of some graph: who cares? Physicists (like me back in the day) are notorious for plotting some statistics on log-log paper, seeing that the points sorta kinda fall on a line if you squint and decide that three of the points are really bug splats and then yelling “UNI…
> Physicists (like me back in the day) are notorious for plotting some statistics on log-log paper...
For people who've missed it: So You Think You Have a Power Law — Well Isn't That Special? (http://bactra.org/weblog/491.html) :)
Re: Graph Mining Library
#84Earlier quoted context omitted.
> If you need docs just read the .h files curious if this is typical dev experience inside google..
I’m not at google so I’ve got no idea. Reading the code, especially the header files, seems to be pretty standard as far as what I see in non-open source code. So, it’s been my typical dev experience, I’d say if you’re somewhere that has gleaming, easy to understand docs that are actually up to date with the code you all have too much time on your hands, but I serially work at startups that are running to market.
Header file gives you a view into some narrow window of the system, API, pipeline, and you probably have no idea which header files are important and which are part of some internal implementation.
10 mins spent on readme with some high level details is investment with 100x return for lib users.
Re: Graph Mining Library
#85Graph mining was "so hot right now" ten years ago. Remember GraphX ( https://spark.apache.org/graphx/ ) and GraphLab ( https://en.wikipedia.org/wiki/GraphLab ) ? Or graph databases? I guess it coincided with the social network phenomenon. Much more recently geometric learning (ML on graphs and other structures) shone, until LLMs stole their thunder. I still think geometric learning has a lot of life left in it, and I…
There are "graph databases" which see graphs as a universal approach to data, see RDF and SPARQL and numerous pretenders. For that matter, think of a C program where the master data structure is a graph of pointers. In a graph like that there is usually a huge number of different edge types such as "is married to", "has yearly average temperature", ... Then there are "graph algorithms" such as PageRank, graph central…
> a universal approach to data, see RDF and SPARQL and numerous pretenders. For that matter, think of a C program where the master data structure is a graph of pointers.
A graph of typed pointers. As you likely know, the basic element of RDF is not “foo has a relationship with bar”, but “foo has a relationship with bar of type baz”.
Also, the types themselves can be part of relationships as in “baz has a relationship with quux of type foobar”
> The thing about graphs is, in general, they are amorphous and could have any structure (or lack of structure) at all which can be a disaster from a memory latency perspective
But that’s an implementation detail ;-)
In theory, the engine you use to store the graph could automatically optimize memory layout for both the data and the types of query that are run on it.
Practice is different.
> Thus practitioners tend to be skeptical about general purpose graph processing libraries
I am, too. I think the thing they’re mostly good for is producing PhD’s, both on the theory of querying them, ignoring performance, and on improving performance of implementations.
Re: Graph Mining Library
#86Can someone explain what this library might be useful for?
Clustering. I used the correlation clusterer from here for a problem that I could represent as a graph of nodes with similarity measures (this data looks like this other data) and strong repelling features (this data is known to be different from this other, so never merge them).
Re: Graph Mining Library
#87Earlier quoted context omitted.
There are "graph databases" which see graphs as a universal approach to data, see RDF and SPARQL and numerous pretenders. For that matter, think of a C program where the master data structure is a graph of pointers. In a graph like that there is usually a huge number of different edge types such as "is married to", "has yearly average temperature", ... Then there are "graph algorithms" such as PageRank, graph central…
> a universal approach to data, see RDF and SPARQL and numerous pretenders. For that matter, think of a C program where the master data structure is a graph of pointers. A graph of typed pointers. As you likely know, the basic element of RDF is not “ foo has a relationship with bar ”, but “ foo has a relationship with bar of type baz ”. Also, the types themselves can be part of relationships as in “ baz has a relatio…
Funny, the core table of salesforce.com is triples but they got a patent circa 2000 on a system that builds indexes and materializes views based on query profiling so the performance is good (w/ gold plated hardware). That patent is one reason why graph databases sucked for a long time.
Now the Lotus notes patents have been long expired so I’d like to see some graph database based products that can do what Notes did 30 years ago but it is lost technology like the pyramids, stonehenge and how to make HTML form applications without React.
Re: Graph Mining Library
#88Graph algorithms cry out for some standardization. Think blas and lapack.
Re: Graph Mining Library
#89Graph algorithms cry out for some standardization. Think blas and lapack.
Consider: https://graphblas.org
Re: Graph Mining Library
#90Earlier quoted context omitted.
Yeah, and somebody who is that smart can probably pack their data structures efficiently and find an approximation to do the job on a macbook pro that people with too many resources need a 1000 machine cluster to do. And get the coding and the computation done in the time that the C++ compiler is still chewing on the headers of the bloated library. (At times I’ve been that guy.) But seriously, there is such a thing a…
I agree with all of this. Its just frustrating that all the comments about an interesting library seem to be customer service complaints from people who never need to reach for this library. I was hoping for a real discussion, something I could learn from.
Really though an open source product has not really been released until there is documentation walking through setting it up and doing some simple thing with it. As it is I am really not so sure what it is, what kind of hardware it can run on, etc. Do you really think it got 117 Github stars from people who were qualified to evaluate it?
(I’d consider myself qualified to evaluate it.. If I put two weeks into futzing with it.)
Every open source release I’ve done that’s been successful has involved me spending almost as much time in documentation, packaging and fit-and-finish work as I did getting working it well enough for me. It’s why I dread the thought of an open source YOShInOn as much as I get asked for it.
Sometimes though it is just a bitch. I have some image curation projects and was thinking of setting up some “booru” software and found there wasn’t much out there that was easy to install because there are so many moving parts and figured I’d go for the motherf4r of them all because at least the docker compose here is finite
https://github.com/danbooru/danbooru
even if it means downloading 345TB of images over my DSL connection.