Here are some harder examples-
abs(x) + abs(y) = 1
is a square of side 1.
x^2 + y^2 = 1
is a circle of radius 1.
But
abs(x) = positive sqrt(x^2)
so squares and circles ought to have the same 2D parent, since given the x & y, its just a matter of applying the right transform.
y = ax^2 + bx
is a parabola through the origin.
y = bx
is a line through the origin.
so lines and parabolas ought to have a common 1D curve as a parent, because a parabola reduces to a line when a=0.
clearly a line shifted by a constant is still a line.
so also, a line rotated clockwise or anti clockwise through say 90 degree is still a line.
so then i take a horizontal line, shift it once and rotate it twice, join these 4 lines and get a square.
so squares ought to be a container type with 4 lines ? or wait, if i collapse the two verticals ie. set the height to 0, then a square is just a straight line with no height, which is absolutely true geometrically, but think about the damage you’d do to your type-theory.
but didn’t we just say lines are just degenerate parabolas. so then we ought to be able to take a square, grab its 4 sides which are lines, transformed each line into a parabola, and tell me what that gives you.
and you haven’t even gotten me started on activation functions. since all logistics are richards curves, a gompertz is just a particular richards. so then a relu should inherit from a gompertz and a swish should be a child of relu and somewhere in here we need the hyperbolic tangent, either as a parent or a child or a sibling or the notorious c++ friend function.
now go ask your friendly neighborhood tensorflow colleague why the activations don’t share a common parent, why lines aren’t parabolas and squares aren’t lines and so in and do forth.