Live data from Hacker News

Market Map

marketmap.one

11–20 of 31 posts

Re: Market Map

#12
post #3

What good is placement within a matrix if you can't see what the axes are?

I guess it's a Principle Component Analysis (PCA) dimensionality reduction so the axes are not necessarily concepts/features with names. More just "abstract dimensions of similarity."

The underlying UMAP model is actually pretty interesting. It's linked to in the tour, though I would have expected it to be featured more prominently: https://pair-code.github.io/understanding-umap/

Re: Market Map

#13
post #9

I wish they had comments about what the clusters are and what the "empty loops" are

I suppose an unavoidable effect of the dimensionality reduction process is that clusters of correlations across dozens of dimensions are by nature hard to describe.

This actually seems like a potentially great application for LLMs – generating a semantic description of an n-dimensional correlation.

Re: Market Map

#14
Sorry to place this here. I've been working on some stock market data visualization over the past couple days -- not as pretty but it makes the point -- and would love some feedback.

Analyzing the Direct Correlation between Federal Reserve's Reverse Repo Operations and S&P 500 Stock Prices

https://github.com/adam-s/fred-reverse-repo-analysis/blob/ma...

Re: Market Map

#15
Incredible viz.

I've experimented with zillions of 3d graphing layouts, usually in the context of PDM/ERP for manufacturing/logistics. Couple of roadblocks I've encountered that are also obstacles here (although he does a MUCH better job than I did in overcoming them, including dynamic distance between nodes, which I can't get away with, sad to say)

First is parallax, the phenomenon of things appearing larger when they are closer to the observer. What this means is that the node size CAN'T be significant in a 3d network unless the perspective is set to orthographic / isometric - because it's going to screw with parallax. How can you tell if the node is actually larger, or if it's just closer?

Second interesting thing about 3d networks is how the (Levenshtein or whatever parm) distance resolves in 3d space, and how that's going to be legible given that we don't have a fourth dimension to stick a camera in. On a 2d surface, the distance-driven force resolves in a 2d vector, so that looking down on it from above, no matter where the force vectors go, all the nodes will be theoretically visible. If you just plot plain distance as a force into 3 dimensions, just using geodesic or straightest line distance, the most tightly gathered nodes will disappear, i.e., be completely occluded. You won't see them!

One possible resolution for this problem, I've found, is classification of distance and assigning this class / category to a specific axis. For example, X axis can be time, Y axis can be a single vector (like, say, military / civilian adoption of a particular dual use part number, expressed as n), and Z axis can represent actual "real" distance (based on tokens, references, "where used", and whatever other factors, either all of them or some of them). This gives you structure where dimensions in the data viz are immediately significant, and simple isometric distance doesn't pile all the nodes in front of each other because they share the same space as the audience.

The takeaway here is that a 3d graph can't just use the same parameters as a 2d graph. The data has to be summarized differently so that the graph remains meaningful. Nothing WRONG with just dumping distance into straight 3d distance, but from the perspective of visual storytelling, it's not optimal.

Also, use isometric cameras. Sure, it's very pretty to have a camera swoop and dive, but it's not going to tell the data's story as well as an isometric camera. (Yes I know I am misusing "isometric" here, but it's the word most people recognize).

Re: Market Map

#16
post #9

I wish they had comments about what the clusters are and what the "empty loops" are

I suppose an unavoidable effect of the dimensionality reduction process is that clusters of correlations across dozens of dimensions are by nature hard to describe. This actually seems like a potentially great application for LLMs – generating a semantic description of an n-dimensional correlation.

This does sound like a fun use case, but I wouldn't even mind being allowed to browse 2d representations to get an intuition of the clusterings

Re: Market Map

#18
Nice work @ernaem! It’s pretty and has some freshness to it. I browsed it on my iPhone, so didn’t do any deep dive.

Would you consider sharing anything from the stack, data and tech side?

Re: Market Map

#19
post #3

What good is placement within a matrix if you can't see what the axes are?

I guess it's a Principle Component Analysis (PCA) dimensionality reduction so the axes are not necessarily concepts/features with names. More just "abstract dimensions of similarity."

Then why is TSM so far away from ASML? What are the UMAP parameters? Perhaps it says somewhere but isn't make prominent.

Re: Market Map

#20
post #4
post #3

What good is placement within a matrix if you can't see what the axes are?

I don't get it as well. The color dimension is configurable, so it makes sense. But I don't get what the 3 dimensions of spatial coordinates are, and how to change them. --- Edit Alright my bad it's actually explained in the "tour": https://pair-code.github.io/understanding-umap/ It seems to be some kind of multivariate PCA.

> It seems to be some kind of multivariate PCA.

No, UMAP is nonlinear. The general idea is that you generate a neighborhood graph of your data points, do a spectral embedding on that to get your initial result, and then do gradient descent to make its neighborhood graph closer to the high-dimensional one.

Post reply on HN