Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Functions are vectors

thenumb.at

Post reply on HN
Print view View original

91–100 of 124 posts

Re: Functions are vectors

#91

Post by bigbillheck »

cykotic wrote: post #25
thaumasiotes wrote: post #13

Earlier quoted context omitted.

That is the definition of a function, but it's purely a formalism. In reality, and in theoretical math, we are frequently interested in functions that generate multiple values, and it's very easy to represent those in terms of the formal definition: 1. When we want to consider both positive and negative square roots, we can just say that instead of the function sqrt(x): ℝ ⟶ ℝ which always gives a nonnegative real num…

We don’t allow multiple values for a given input of a function. Ever. This is never done. f(x) = x^2 is not invertible as a real valued function of a real variable. On the restricted domain of [0, infinity) it is invertible and that inverse is sqrt(x). If you want to talk about negative roots you use the function -1*sqrt(x). You can construct a new function as you did to encapsulate this fact but one never has a func…

> You will not find this in the literature.

It would have cost you nothing except a minute or so of your time to have refuted yourself before posting: https://arxiv.org/abs/1912.08274 https://arxiv.org/abs/1703.01700

(See also: https://en.wikipedia.org/wiki/Multivalued_function)

Re: Functions are vectors

#92

Post by smokel »

GolDDranks wrote: post #78

I haven't read the article yet, but I've known that functions are (infinite) vectors for some years. However, there's something that has been bothering me: most of my understanding of linear algebra comes from 2D and 3D spaces, and then in different context of machine learning, datasets that have from tens to even millions of dimensions. In the former, geometric context, the connection between the dimensions are clea…

This is bothering me a little as well. Perhaps someone more knowledgeable could guide us into the light.

As a concrete example, I am toying with a reinforcement learning to solve the game of Yahtzee. When trying to formulate the state space, I have to lump in together the states of the dice (1..6), the current state of the score card (13 boolean values, and 13 integer scores in the range 0..50), and the current turn (first attempt, second attempt). This could be formulated as a 5+13+13+2 = 34 dimensional vector, but I might as well use one-hot encoding for some aspects, and even go all the way up to 56+13+56+13+30+30+1+1+1+1+30 = 180 dimensions. Which formulation would be the most "natural"?

Then again, what may seem (un)natural to a human, might be the other way around for a neural network.

(And, yes, I am aware of papers out there describing how to solve Yahtzee with reinforcement learning, but I'm still at a stage where avoiding those seems to increase the amount of fun.)

Re: Functions are vectors

#94

Post by rcme »

LudwigNagasena wrote: post #89

I think the article has it backwards and provides bad intuition. It is not input that makes functions form a vector space, it is the output. Functions from any set X to a field F can form a vector space, even if X is unordered.

It’s neither the input nor the output. It’s the mapping of inputs to outputs, aka the function :).

Re: Functions are vectors

#95

Post by cykotic »

bigbillheck wrote: post #91
cykotic wrote: post #25

Earlier quoted context omitted.

We don’t allow multiple values for a given input of a function. Ever. This is never done. f(x) = x^2 is not invertible as a real valued function of a real variable. On the restricted domain of [0, infinity) it is invertible and that inverse is sqrt(x). If you want to talk about negative roots you use the function -1*sqrt(x). You can construct a new function as you did to encapsulate this fact but one never has a func…

> You will not find this in the literature. It would have cost you nothing except a minute or so of your time to have refuted yourself before posting: https://arxiv.org/abs/1912.08274 https://arxiv.org/abs/1703.01700 (See also: https://en.wikipedia.org/wiki/Multivalued_function )

A multivalued function is not a function. Hence the need to call it something else. In complex analysis you have branch cuts and things like that. We don’t call something a function that doesn’t have a unique output for a given input.

EDIT: Complex analysis was developed well before the formal definition of functions, relations, set theory. As such there are legacy terms in use. Mathematics has a lot of abuse of notation. For instance x=2 can mean assignment or an equation to solve. It depends on context.

In each branch of a multi-function the mapping is a function and, as a convenience, a single term “multi-function” is used to encapsulate all this. It cuts out the verbosity. A multi-function is not a function though.

My pedantry in this comes from the blog post. It was poorly written and left out some, from a mathematical point of view, important details. For instance the author uses a Schauder basis and has an uncountable sum at one point whilst ignoring the need distinguish between a Hamel basis and Schauder basis or the need to use a linear ordering in the reals to properly make sense of the uncountable sum.

Re: Functions are vectors

#96

Post by cykotic »

thaumasiotes wrote: post #56
cykotic wrote: post #25

Earlier quoted context omitted.

We don’t allow multiple values for a given input of a function. Ever. This is never done. f(x) = x^2 is not invertible as a real valued function of a real variable. On the restricted domain of [0, infinity) it is invertible and that inverse is sqrt(x). If you want to talk about negative roots you use the function -1*sqrt(x). You can construct a new function as you did to encapsulate this fact but one never has a func…

Don't be ridiculous. It's a common need, it's easily satisfied, and so it is commonly done. Compare the concept of the "inverse element" of a value given by an invertible function with the "inverse image" of a value given by a non-invertible function. Of course the function that gives you inverse images is nothing more than an inverse function that is allowed to give multiple values. Why do we do that? Because our ne…

There is no high principle other than the agreed upon definition of what a function is.

Of course the function that gives you inverse images is nothing more than an inverse function that is allowed to give multiple values.

You can’t use the word function for something that isn’t a function. It makes no sense to say inverse function that is allowed to give multiple values. Hence the need for terms like pre-image of a set.

In all contexts it is important that a function yield a single value for a given input because that is the definition of the word.

Re: Functions are vectors

#97

Post by javajosh »

This looks absolutely fantastic and I want to go over it in more detail later. You'll go over most of this in a typical physics degree. But like a good movie or book, the concepts are interesting enough to go over more than once.

I will say that as a programmer some of these techniques look a lot like hacks. You start off with a perfectly reasonable integer index. And then you realize you can generalize the index, effectively cramming more information into the index than was originally intended. The really shocking thing is that these stupid, abusive ideas seem to always lead to something really insightful and useful down the road. It's a little bit magical.

Re: Functions are vectors

#98

Post by sheepscreek »

ubj wrote: post #41

I wish I could upvote this twice. This is the best basic introduction to concepts in functional analysis that I've seen. Another great overview that goes deeper into the math is [1]. Another fantastic application that the website doesn't mention is the composition / Koopman operator. In control theory (e.g. autonomous drones, cars, robot arms, etc.), most real-world systems are described by nonlinear dynamics which a…

My first thought was about its striking conceptual similarity to Fourier transforms. Truly fascinating. Going to explore this a bit more. Thanks for sharing.

Re: Functions are vectors

#99

Post by fritzo »

Plug for the Funsor library, written by Eli Bingham and me for use in the Pyro and NumPyro probabilistic programming languages. We tried to take the "functions are tensors" perspective and make a numpy-like library for functions, aimed mostly at the log-density functions of probability distributions.

Paper: "Functional Tensors for Probabilistic Programming" (2019) https://arxiv.org/abs/1910.10775

Code: https://github.com/pyro-ppl/funsor

Post reply on HN