(I vouched for this comment since it exposes a common misconception people have about derivatives, and I'd rather reply than have it buried)
It's unfortunate that the connection between matrices and linear transformations, and thereby between derivatives and linear transformations, is not adequately emphasized in the math courses that most people take.
The derivative (or differential, or total derivative) of a function f : R^m -> R^n at a point x ∈ R^m is defined, formally, to be a linear transformation A : R^n -> R^m such that
|| f(x+h) - (f(x) + A(h)) ||
lim ------------------------ = 0
h->0 ||h||
Where ||.|| indicates a vector norm that measures the magnitude of its argument. Unpacking a bit, notice that
> h is a vector in R^m
> f(x+h) is the exact value of f at x+h
> f(x)+A(h) is a linear approximation to f(x+h), using x as the base point and A as the linear approximator
So this just a formal way of saying that the dervative A should be a good linear approximation to f.
Every linear transformation has a corresponding matrix, and every matrix corresponds to a linear transformation (with respect to a choice of basis).
So, when we say "the derivative of f at x is the matrix A", we really mean that "the derivative of f at x is the linear transformation represented by the matrix A".
EDIT: In response to the parent poster, who has dug their heels in by posting a now-dead rant in reply to this post, I encourage you to refer to a copy of Rudin, "Principles of Mathematical Analysis", for a rigorous treatment of derivatives. See also: representation theory and group actions.