Here's how you can apply this interpretation of matrix multiplication:
Think of the columns of the matrix A as basis vectors of a coordinate system represented in global coordinates, and the vector v as the components of a vector in that coordinate system. Then the product A * v transforms the vector v into the global coordinate system.
Carrying this forward, matrix multiplication A * B gives the combined representation of two coordinate transformations.
Another way to look at it: A matrix is a linear transformation, and multiplying a vector by a matrix is how you apply the transformation. But linear transformations are really just changes of basis. How do you change your basis? You find the dot product of a vector with each new basis vector. And that's exactly what matrix multiplication is. When you multiply your column vector by a row in the matrix, you're finding the dot product, doing the projection in your change of basis.
Hmm. As someone who has never understood/visualized/been taught matrix multiplication in any other way than how it's shown on the linked page, could someone explain the alternative? I.e., if this is new, what mental model do you currently have? Presumably that one would be new and useful for me.
Edit: "Typically this visualization isn't taught" -- that's what I"m asking. What visualization were you taught, if not this one? I can't think of any other that makes any sense.
Typically, this visualization isn't taught, in my experience. What's taught is the formula for computing each cell of the result matrix (cell i,j being the dot product of row i of the first matrix with column j of the second). While this is, of course correct, and also the most efficient way to compute the multiplication manually, it's not always clear why the formula is correct.
Hopefully the visualization on the linked page makes this formula's origin clear.
Really? Interesting. We spent some time discussing multiplication by block decompositions in my 100-level linear algebra class -- is this really uncommon?
Unless I'm missing something, this is precisely the kind of visualization that explains how to compute the result, but not why the computation makes sense. I.e. why is matrix multiplication defined this way at all.
You usually think of the product of a matrix with a vector as the matrix acting on the vector, i.e. every row of the matrix tells you what the corresponding component of the resulting vector is. This turns the thing on its head and describes the result as the vector acting on the matrix. Because you usually use matrices to describe operations, e.g. transformations like translations or rotations in (computer) geometry or observables in quantum physics, it seem more natural to think of the matrix as acting on the vector, i.e. a point of the geometry or a physical state, not the other way round.
Yes, but if you think of the matrix as a collection of basis vectors, then something like a rotation in R3 becomes crystal clear. Your using the same coordinate but moving in arbitrary (orthogonal) directions rather than just right,up,forward. This makes more sense using scalar*vector operations than using dot products.
I was taught to go element-by-element: element c_ij of the product = the linear combination a_ik*b_kj. What's presented here is the "same" but represents a higher level of abstraction, so that you think in terms of linear combinations of rows or columns, rather than linear combinations of scalars.
Matrices represent maps (functions), and matrix multiplication is map (function) composition. This is very useful because you can easily see how linear algebra generalises to functional analysis and you can easily get an intuitive feel for many theorems from functional analysis if you convince yourself that infinite-dimensional bases are not that special.
One alternative that I find interesting (not the one I was taught first though) is to visualize matrix multiplication as the sum of the rank 1 matrices obtained by multiplying each column of the left matrix by the corresponding row of the right one.
+1 for the Strang videos. He's an amazing lecturer. e.g. in Lecture 1 explains very clearly matrix multiplication in the "column picture" and the "row picture". More intuition than you can shake a stick at:
http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-...
This is the way I "understood" linear algebra.
I even encoded it here in a small c++ header file when I used to think it wise to implement these things yourself:
> when I used to think it wise to implement these things yourself
It's a very good thing that you thought it wise at some point. There's nothing like reinventing a few fundamental wheels for a better understanding of how they work.
C has the same number of rows
as A and the same number of columns
as B, and
each column j of C is from
column j of B acting as
coefficients in a linear
combination of all the columns
of A.
Next: For the set of
real numbers R,
positive integers m and n,
m by n real matrix A,
real numbers a and b,
and n by 1 x and y,
we have that
A(ax + by) = (aA)x + (bA)y
so that A is linear.
If we regard x and y as vectors
in the
n-dimensional real vector space
R^n, then we have that
A is a function
A: R^n --> R^m
and a linear operator
which can be good to know.
Then the subset of R^n
K = { x | x in R^n and Ax = 0 }
(where 0 is the m by 1 matrix of all
zeros)
is important to understand.
E.g., K = [0] if and only if
function A is 1-1. If m = n,
then A has an inverse
A^(-1) if and only if A is 1-1.
With C = AB, the matrix
product is the same as
function composition so that
Cx = (AB)x = A(Bx)
which also can be good to know and,
of course, uses just the
associative law of matrix
multiplication.
That matrix multiplication is
associative is a biggie --
sometimes says some big things,
e.g., is the core of duality
theory in linear programming.
And the situation is entirely
similar for the complex numbers
C in place of the real numbers R.
I don't know why but it seems to make more sense to me if I think of the multiplication in terms of rotating or transposing and moving the vector or matrix so that the columns or rows line up with the ones they are to be multiplied with, the coloured guides are entirely useless to me.
What is the point of this blog post? Matrix multiplication is shown like this, and in many other similar ways, in many introductory linear algebra textbooks.
I'm actually studying linear algebra right now, and this site has given me insights my text book (which is actually pretty good!) just hasn't. Thanks for the link :-)
Think of the columns of the matrix A as basis vectors of a coordinate system represented in global coordinates, and the vector v as the components of a vector in that coordinate system. Then the product A * v transforms the vector v into the global coordinate system.
Carrying this forward, matrix multiplication A * B gives the combined representation of two coordinate transformations.