I immediately found the results suspect, and think I have found what is actually going on. The dataset it was trained on was 2770 images, minus 982 of those used for validation. I posit that the system did not actually read any pictures from the brains, but simply overfitted all the training images into the network itself. For example, if one looks at a picture of a teddy bear, you'd get an overfitted picture of anot…
What are you talking about? They didn't train a model for this. That's why it's so impressive.
Quoting from the paper,
The only training required in our method is to con-
struct linear models that map fMRI signals to each LDM
component, and no training or fine-tuning of deep-learning
models is needed.
...
To construct models from fMRI to the components of
LDM, we used L2-regularized linear regression, and all
models were built on a per subject basis. Weights were
estimated from training data, and regularization parame-
ters were explored during the training using 5-fold cross-
validation.