Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to build a graph visualization engine

memgraph.com

Post reply on HN
Print view View original

41–50 of 73 posts

Re: How to build a graph visualization engine

#41

Post by cizl »

erwinh wrote: post #2

Recently a new web-based fully gpu driven graph visualisation engine was released and it could handle quite some big graphs directly in the browser: https://twitter.com/hoogerwoord/status/1568188361503907840?s... Happy that people are building new graph visualisation engines and I think still a very open space for innovation in interactively exploring graph/network structures.

Cosmos really looks amazing when it comes to performace. Besides visual rendering on the GPU, they also do the graph layout simulation calculations on the GPU as well, which is a first. SigmaJS for example uses graphology for the calculations, and WebGL for rendering.

Cosmos is a really new lib, so when we did our initial research it wasn't even out yet. You can read more about our research over here: https://memgraph.com/blog/you-want-a-fast-easy-to-use-and-po...

From what we've understood, it's somewhat limited when it comes to graph styling. But an amazing technology nontheless.

Re: How to build a graph visualization engine

#44

Post by stuntkite »

taubek wrote: post #32
stuntkite wrote: post #10

Earlier quoted context omitted.

Heh. You and me both. When I worked there something I really wanted to get to was GIS integration and abstraction with graph. They are doing some really, really cool stuff and I think their future offerings will be things to take note of, but it's a service too and it's not open source. If you find a solution to what you're talking about, I'm interested. Let me know. Also, I talk to Leo. If you have specific requests…

What kind of integration with GIS where you considering? What did you want to accomplish?

Well I haven't let go of the idea and am in the process of releasing a GIS data processing platform, but that's a story for another day. My thought is that all data, especially large datasets from the top down only has a few ways it can be displayed and sliced. Relationships, multi temporal (slicing and or playing by time but then also playing back multiple time sections so they can be compared. compute vs. real time, time to exec on server vs time experienced by user, and also finding various time slices to compare. So multi temporal being that your timeline playback isn't just a film strip), and spatial. The ground truth for all data is that it happens somewhere in the world so a graph should be able to be put into a GIS space for display, bonus points for 3D, but then also you should be able to build composite fake GIS spaces and visualize the relationships between them. Like for instance, your products network traffic all going to colocation spaces and to users and back again. That has normal GIS locations and that can provide constraints for graph display but also, especially in a distributed system there may be artificial geography that can be defined, lets call it "The Astral Plane" that has relationships that are important and grouping just like state and country boundaries that can be defined in shape files and put into a spatial database in a projection you just make up or is defined by the graph data and physical locations and you should be able to slide between all those things. That's where I wanna go and still intend to get to.

EDIT: Obviously when I say everything has GIS coordinates I'm not quite talking about outer space, but I think outer space is even covered by the rest of this idea. So is tiny space. This realization came to me when I was reading about people using PostGIS as a database for chemistry simulation. I have no link for that, I haven't thought about it in years, but now that I'm talking about it, I'll try to find it and post a link if I find it as I would like to readdress.

Re: How to build a graph visualization engine

#45

Post by kasupe »

vpavicic wrote: post #39
kasupe wrote: post #29

If it’s at least a bit easier to use than D3.js, I like it :’) Seems like a really cool project.

I tried it a bit and it does seem easier and it renders way faster

I see that it draws on canvas, and I used to draw on svg. This is awesome!

Re: How to build a graph visualization engine

#46

Post by stuntkite »

mbuda wrote: post #30
stuntkite wrote: post #18

Earlier quoted context omitted.

This looks pretty clunky. Is this what Neo4j uses in their viewer? It feels like it's related.

It's not, it's also based on d3 (like Neo4j viewer I think), but build from from 0

The code giving a "lot of thought to software design" must be pretty good, because this looks like Java Graybeard garbage to me. Yeah, that's a caustic statement. I'm not trying to say that people who made it didn't do a good job, but the display is just.... Not good enough. They need to find a team member to help them get out of 20 years ago with this stuff. If anyone that is working on this reads this. That's my thought. I am not taking shots at your work. It does look well organized, but I cannot see a reason I would choose this.

Re: How to build a graph visualization engine

#47

Post by cizl »

kasupe wrote: post #37
stuntkite wrote: post #18

Earlier quoted context omitted.

This looks pretty clunky. Is this what Neo4j uses in their viewer? It feels like it's related.

What does Neo4j use?

Neo4j uses a fork of VisJS. They call it NeoVis. We didn't go that route because vis-network is tightly coupled and has a lot of calls to the browser window reference which doesn't work in a WebWorker environment. So simulations end up blocking the main thread.

Re: How to build a graph visualization engine

#48

Post by stuntkite »

jylam wrote: post #26
stuntkite wrote: post #16

Earlier quoted context omitted.

It depends on how you value your time and effort. Not every idea should be valued by its market fit. I don't think people should write code they are uninterested in writing. That's my litmus test for myself. Usually when I'm really, really interested the thing I'm aiming at doing initially isn't even about what I'm really trying to solve. It's picking away at the cruft over something and then I can see what's on the…

I don't thing Parent was talking about business at all. Personal projects are not things you do because of potential business opportunities in my book. I wrote a GameBoy emulator, several raytracers, countless demos on 8bits machines, countless "IoT" things, an automatic cat feeder, a 3d scanner and much more, I never ever had the goal to transform that into money, that was for fun. Of course you won't write anything…

I see what you're getting at and concede that you may be totally right that I'm not on the same page as Parent, but if the reason for not doing something dumb that you'll probably fail at that other people are doing better at, that lets say you also hate, isn't for building a business... What are you doing that for? Pathological self harm?

I think you and I agree. I'm not being crass, but I'd if you had more thoughts on what I'm trying to say about what you're trying to say I'd be interested in further discussion.

Re: How to build a graph visualization engine

#49

Post by xani_ »

stuntkite wrote: post #25
xani_ wrote: post #23

Earlier quoted context omitted.

I agree. Making a personal blog engine gave me new hatred for anything frontend and a deal of compassion for people that have to deal with that mess professionally.

What blog engine and why did you and I assume your partners feel like the world needed another one? Not being glib. I'm genuinely curious.

It didn't need another one which is why I won't be showing it ;p. It was also project where I learned Go and did a bunch of redesigns along the way so it is a but of a mess. I'd also hate idea of someone else actually using it, or worse, reporting bugs, because just using some static site generator would've been much better idea and I kinda designed it for my workflow.

It did a lot of performance wankery, like templating system (not my lib, I just used available one) being just Go code embedded in HTML that needed to be compiled with the rest of the app, or pre-generating HTML from Markdown on load. Using same language to write app as to write templates was nice tho.

Most of it was mostly "how far I can go without cutting features" because really going from 10ms per page to 0.2 ms per page has no difference after client RTT is involved.

Hell, even on localhost for some reasons Chrome always have few ms delay before starting download compared to any cli client, chrome shows anywhere between 3 and 8ms to start downloading, while FF sits at 0ms

I originally planned to have some fun with HTTP2.0 PUSH but, well, while I kinda believed on authority that it is useful for something it turned out to be entirely stupid idea which apparently nobody bothered to test before pushing it into standard so I didn't get to do it. Maybe I should try again with HTTP3.0 hints.

Re: How to build a graph visualization engine

#50

Post by stuntkite »

xani_ wrote: post #49
stuntkite wrote: post #25

Earlier quoted context omitted.

What blog engine and why did you and I assume your partners feel like the world needed another one? Not being glib. I'm genuinely curious.

It didn't need another one which is why I won't be showing it ;p. It was also project where I learned Go and did a bunch of redesigns along the way so it is a but of a mess. I'd also hate idea of someone else actually using it, or worse, reporting bugs, because just using some static site generator would've been much better idea and I kinda designed it for my workflow. It did a lot of performance wankery, like templa…

It sounds like you learned a lot... but a blog platform to do that? Like why not just pivot to something that might need the enhancements you're providing with go?

I'm asking this for rhetorical purposes. I know, I know deeply why. heh. I'm glad you came out of it ok. Thank you for your honest response.

Post reply on HN