Hello, I am confused about the use of the expression ‘Tensor’. Insofar as Deep Learning (or any other aspect of Data Science) is concerned, it just simply appears to be any rectangular, cuboid, or higher dimensional entity that satisfies the rules of Linear Algebra. On the other hand, a Tensor is more than just some entity that satisfies the rules of Linear Algebra. It must also possess Geometrical and Topological properties (e.g. Covariant and Contravariant Transformation laws, Continuity of small neighborhoods). So where is that aspect in Deep Learning? Or should we just simply refer to a Tensor as an ‘n-Dim Matrix’ ?

Hello, it may help ; I don’t remember where I picked this thoughts :

The defining characteristic of a vector, matrix or tensor is how they transform under a coordinate transformation, not the dimensionality of the array that stores the numbers.

In fact, a scalar can be an element of any Field (e.g. the complex numbers C, which is actually isomorphic with R^2, i.e. a 2-space on the real numbers; or quaternions, which are isomorphic with a 4-dimensional vector space, yet operations defined on them make them behave like scalars, …).

And a vector in an N-dimensional Vector Space (e.g. R^N, a vector space on the real numbers), is N-dimensional.

And a matrix representing a linear transformation on an N-dimensional vector space is an NxN table of numbers

So, the dimensionality of the array of numbers representing a scalar, a vector, a matrix or a tensor is related to the rank of that linear algebra entity, not its dimensionality.

It looks like an unnecessary complication but unless you fully understand the roles and mathematical properties of these entities you will never be able to unlock the full power of Vector Spaces and Linear Algebra beyond being able to program deeper layers of arrays of numbers.

Yes, I probably did mean Rank. But outside of this property, no other aspect of satisfying the conditions for Tensorial behavior occur in Deep Learning, that is why I wanted to simply describe the entities as ‘Rank-N matrix’.