Having picked up a rudimentary understanding of tensors from reading mechanics papers and Wikipedia, I tend to think of rank-2 tensors simply as square matrices (along with appropriate transformation rules). Certainly, if the distinction between vectors and dual vectors is ignored, a rank 2 tensor $T$ seems to be simply a multilinear map $V \times V \rightarrow \mathbb{R}$, and (I think) any such map can be represented by a matrix $\mathbf{A}$ using the mapping $(\mathbf{v},\mathbf{w}) \mapsto \mathbf{v}^T\mathbf{Aw}$.
My question is this: Is this a reasonable way of thinking about things, at least as long as you’re working in $\mathbb{R}^n$? Are there any obvious problems or subtle misunderstandings that this naive approach can cause? Does it break down when you deal with something other than $\mathbb{R}^n$? In short, is it “morally wrong”?
You’re absolutely right.
Maybe someone will find useful a couple of remarks telling the same story in coordinate-free way:
What happens here is indeed identification of space with its dual: so a bilinear map $T\colon V\times V\to\mathbb{R}$ is rewritten as $V\times V^*\to\mathbb{R}$ — which is exactly the same thing as a linear operator $A\colon V\to V$;
An identification of $V$ and $V^*$ is exactly the same thing as a scalar product on $V$, and using this scalar product one can write $T(v,w)=(v,Aw)$;
So orthogonal change of basis preserves this identification — in terms of Qiaochu Yuan’s answer one can see this from the fact that for orthogonal matrix $B^T=B^{-1}$ (moral of the story: if you have a canonical scalar product, there is no difference between $T$ and $A$ whatsoever; and if you don’t have one — see Qiaochu Yuan’s answer.)
It’s not misleading as long as you change your notion of equivalence. When a matrix represents a linear transformation $V \to V$, the correct notion of equivalence is similarity: $M \simeq B^{-1} MB$ where $B$ is invertible. When a matrix represents a bilinear form $V \times V \to \mathbb{R}$, the correct notion of equivalence is congruence: $M \simeq B^TMB$ where $B$ is invertible. As long as you keep this distinction in mind, you’re fine.