Functions are vectors

You process the operations between such equations -sum, inner, outer- by "unpacking" them into Matrix equations format (populating the Matrix with the vector values), doing the operations, and returning back to the vector format.

So in essence you are working with Matrix equations, but with a more compact format as vector.

For being able to work with such equations' compact format, its needed to follow some rules restricted to the purpose, case contrary, as in reality they are vectors per se, the geometries could mix dimensions were the equations' proprieties are different.

The article (maybe for good reason) completely ignores the (usually quite hard) choice in functional analysis which vectorspace to use.

And the one outlined here, where functions are defined pointwise, is amost always the most useless one to consider. Although I get that it was done to teach the general outline of the subject, which is quite valuable on its own.

>Clearly there is no Fourier transformation possible on all functions?

You don't even get a useful metric for such a space.

Traditionally, in regression analysis, you'd use dummy variables for the nominal/binary data, turning all of them into binary {0, 1}. This is GF(2), a bonified vector space. Education level you've got some choices. You can also encode these as binary, potentially losing some information given they're ordered. But if you choose to map them to integers or real numbers, what is the justification for the distance function you're defining? Is MD > PhD? How much greater is PhD than BA than high school diploma?

If you stick to one-hot encoding, there's justification in performing regression analysis still, because you end up in a case where most of the values are 0 and drop out and you get multiple models, one for each case, so the matrix operations being performed to generate the model only operate on the remaining real-valued vectors, and the one case that is 1 becomes the constant term. This is just basic ANCOVA.

You have another issue, though, right? Are "height" and "income" and "age" really from the same vector space? Obviously not. Some modelers choose to ignore this. Some will assume they're all normally distributed and tranform the individual values to a Z-score or something and then they're all from the same vector space. But this still may be dubious. For one thing, they're not actually normally distributed. Height and age have a strictly zero lower-bound. They have some upper-bound, whatever it may be. Income is severely right-skewed. But they're probably all at least approximately normal over some restricted range of the most common values.

We're starting to see the issue. Modeling in this way might work reasonably well within some restricted range of common values, but extreme cases are left out. We need some other techniques that don't rely on matrix math, because once we have to admit all of our elements from not from the same field, we admit we don't really have a matrix, even if a computer will gladly let us pretend anything we can encode as a number is actually a number.

I think your question is how can we encode data such that it all ends up in the same vector space? It's easy if your data is imagery. Color channel intensity of each pixel is all real-valued, possibly within different ranges, but easily normed. If all you have is text, representing the corpus by an incidence matrix puts all of your elements in GF(2). Plenty of other word embeddings are well-justified. But when you start looking at longitudinal data from epidemology, the shit they do in econometrics, it starts to get less math and more black magic. By the time you're a data scientist at Netflix, I'm sure they have reason to believe whatever they're doing works, but it may be hard to reason about why it works.

I don't really know if there any field of mathematics studying ways of encoding data such that doing things like adding and multipying height by favorite color actually means anything, but it's an interesting question.