Earlier quoted context omitted.
What kind of integration with GIS where you considering? What did you want to accomplish?
Well I haven't let go of the idea and am in the process of releasing a GIS data processing platform, but that's a story for another day. My thought is that all data, especially large datasets from the top down only has a few ways it can be displayed and sliced. Relationships, multi temporal (slicing and or playing by time but then also playing back multiple time sections so they can be compared. compute vs. real time…
How to build a graph visualization engine
51–60 of 73 posts
Re: How to build a graph visualization engine
#52Recently a new web-based fully gpu driven graph visualisation engine was released and it could handle quite some big graphs directly in the browser: https://twitter.com/hoogerwoord/status/1568188361503907840?s... Happy that people are building new graph visualisation engines and I think still a very open space for innovation in interactively exploring graph/network structures.
Re: How to build a graph visualization engine
#53Earlier quoted context omitted.
It's not, it's also based on d3 (like Neo4j viewer I think), but build from from 0
The code giving a "lot of thought to software design" must be pretty good, because this looks like Java Graybeard garbage to me. Yeah, that's a caustic statement. I'm not trying to say that people who made it didn't do a good job, but the display is just.... Not good enough. They need to find a team member to help them get out of 20 years ago with this stuff. If anyone that is working on this reads this. That's my th…
Just as a note, Orb is not there to compete with high volume graph visualizations like Cosmograph, Graphistry, Linkurious. It is more as a child from d3 and vis.js, which are great libraries, that uses d3 simulation and vis-like canvas rendering. We really liked what vis.js team did with the styling of the graph and how you can customize it - this is often a limitation for high volume graph visualizations.
We could also discuss about the analytics usability of seeing a graph with 1 billion nodes. It is definitely awesome, but it is too much data to grasp on as a user seeing it. Clustering or other graph algorithms would help. I think the question is: What is the maximum graph size (number of nodes/edges) when it becomes hard to get any useful visual information expect the graph global state? (e.g. seeing a bar chart with 365 columns (days) is harder to read than a bar chart with a smaller sampling, e.g. per week or month).
I don't know the answer to this, but maybe you will have due to your experience with graph visualizations.
Re: How to build a graph visualization engine
#54"And why you shouldn't" doesn't deserve equal billing in the headline. The article structure is * first two sentences: You shouldn't build from scratch * next four sentences: How not building from scratch failed for us * entire rest of article: How we built from scratch and made an amazing product you should try. I guess the implication is "you shouldn't build an engine because we just built the engine to end all eng…
Even though "end all engines" might not be 100% correct, because the initial idea of the Orb is to make a single interface where the background (simulation and rendering engine) can be changed.
To add to your title, "you shouldn't build an engine because we just built the engine to unify all engines". :)
Re: How to build a graph visualization engine
#55If it’s at least a bit easier to use than D3.js, I like it :’) Seems like a really cool project.
Re: How to build a graph visualization engine
#56Earlier quoted context omitted.
I tried it a bit and it does seem easier and it renders way faster
I see that it draws on canvas, and I used to draw on svg. This is awesome!
Re: How to build a graph visualization engine
#57I was an early employee at the first round of graphistry.com. I'm really proud of Leo and that he took it from the brink and put it back on the market and is growing it. I think more people should build more graph visualization engines, but you're going to have a hard time competing with how slick pygraphistry is, but there is not enough alternatives that are worth a shit. Graphistry, when I left was able to do half…
Did you mean a million? 500k is half a million not billion
Re: How to build a graph visualization engine
#58Earlier quoted context omitted.
It didn't need another one which is why I won't be showing it ;p. It was also project where I learned Go and did a bunch of redesigns along the way so it is a but of a mess. I'd also hate idea of someone else actually using it, or worse, reporting bugs, because just using some static site generator would've been much better idea and I kinda designed it for my workflow. It did a lot of performance wankery, like templa…
It sounds like you learned a lot... but a blog platform to do that? Like why not just pivot to something that might need the enhancements you're providing with go? I'm asking this for rhetorical purposes. I know, I know deeply why. heh. I'm glad you came out of it ok. Thank you for your honest response.
I wanted something simple that was just markdown for actual content, no database or anything more fancy. I have considered static gen and even eventually migrated my old blog to it as archive (not in english, and cringe anyway). But it really started as "well, it looks like fun thing to do and I will be scratching the itch I had, why not".
Mind you, that was in 2012, the first version was in asynchronous(!) Perl, Ghost didn't even exist at that point (and I didn't wanted to touch JS anyway), let alone any other alternative. It even did respectable ~3ms per render of the page.
I rewrote it in Go to learn some stuff and in the process also yeeted comment processing and just farmed it out to externals (self hosted, not written by me) app, as that's probably the most annoying and thankless part of blog engine when you include all kinds of spam detection that would need to be written.
On the nearby cementery I also have unfinished Go z80 emu (I did learn a bit about using Dear ImGui from it) and it rewrite in Rust (because what's better first project than that?), just coz I wanted to see just how much faster CRust is compared to CGo interop(answer = a lot, rendering part got from >4ms to below 0.5ms).
I did it to the point where it could run some code, and quite fast too, I think it was down to few ns per instruction, and 8 byte prog ran in like 11 ns, which I kinda didn't expect from Go. It didn't emulate instruction delay tho, which would be required to emulate it with peripherals.
Program decoder was just...256 byte array with function pointers generated out of operator list, which probably helped
Re: How to build a graph visualization engine
#59"And why you shouldn't" doesn't deserve equal billing in the headline. The article structure is * first two sentences: You shouldn't build from scratch * next four sentences: How not building from scratch failed for us * entire rest of article: How we built from scratch and made an amazing product you should try. I guess the implication is "you shouldn't build an engine because we just built the engine to end all eng…
Re: How to build a graph visualization engine
#60Earlier quoted context omitted.
It sounds like you learned a lot... but a blog platform to do that? Like why not just pivot to something that might need the enhancements you're providing with go? I'm asking this for rhetorical purposes. I know, I know deeply why. heh. I'm glad you came out of it ok. Thank you for your honest response.
Well it started after wordpress fucked up my formatting and replaced -- with — (em dash) one too many times. The whole PHP stack isn't exactly pleasant to manage in the best of days, and looming threat of WP having another bug and getting owned of no fault of my own was also a factor. I wanted something simple that was just markdown for actual content, no database or anything more fancy. I have considered static gen…