Viewing profile — tuned

trying to generate images and sounds from eigenvectors as harmonic basis

Measuring structural information. Check out the paper, notebooks and Python library

ok, thanks. I am taking it slow then

no, from my point of view is being more domain-focused instead of going full-orthogonal.

right. this is a proposal that needs to be tested. I started testing it on 30M parameters then I will move to a 100M and evaluate the generation on domain-specific assisting tasks

> This is obviously not powerful enough to express non-linear relationships - like graph relationships. the distance metrics used is based on energy-informed graphs that encode ene…

also: precomputing a sparse Laplacian for N vectors at dimension D (NxD) is infinitely cheaper (if using `arrowspace`, my previous paper) than computing distances on the same full …

if you have a corpus of code snippets to train the manifold (Laplacian) on (and a good embedding model), it is definitely possible to try something like this.

it made sense to me as it is a very simple idea I guess: causal self-attention compute QKV distances computing on the full vectors for Q,K and V; the topological transformer can pr…

it most-likely will in terms of performance as it uses 50% less memory (for sure it will at inference time that is the most used operation on web services), because it can leverage…

Thanks to all that have read. I would be glad to answer further scoped questions on the content of the post and the paper. I answered some comments that may clarify the ideas from …

the idea is to have a lot of "narrow" models to work with RAG instead of one model for all the knowledge domains or also distil the metadata that is currently in enterprise Knowled…

exactly, that is the current objective. To proove that generation for a specific domain is on-par with causal attention models

comparisons will be run when the quality of generation will be on pair with other available models. It is useless to have preformance if the quality is not at lease on par. The pap…

This is a novel re-interpretation of the Transformer, based on my previous research made with a library called `arrowspace`. It is somehow what is called a "Grassmann-like flow" bu…

thanks for linking. Yes the paper compares the new architecture (that is also a fork of my implementation of nanoGPT) with Karpathy's nanoGPT. There are also links to the code and …

thanks for reading. I cannot retrain an existing model as the self-attention mechanism has been completely redesigned. The Keys and Values in self-attention are stored as scalars, …