Images are represented as tensors on deep learning algorithms

Even though the technique of neural networks are universal, it can be challenging to transform the problem into a space that the NN can understand. I remember a private student doing a masters, and how heard it was for him to transform reality into a space that the neural model could understand.

For images, this space is called tensors . Tensors were widely used for developing the general theory of relativity, and it is used for similar reasons in computer vision. It seems to me Einstein made this concept popular: the concept of tensors.

In simple terms: tensors are used to represent several dimensions problems. Do not waste your time trying to see it visually, we cannot.

I have to be honest: tensors are hard to understand. It took me time to understand. They are used to dimensions we cannot see.

Here it goes:

1. one dimension -> vector;

2. two dimensions -> matrix

3. three dimensions -> matrixes of matrixes - this is a tensor!

If did say operations research, and used Excel, it is pretty hard when we have relations that are 3D. Generally, you must create matrixes that relates to matrixes: a mass. One line of math takes several spreadsheets.

Not sure the best way to explain. I like the image below, maybe because I did mechanics of materials, and we used it a lot for calculating tension on materials. Thus, I had time to create a mental abstraction of the situation.

For image, we need to go deeper. For a normal image, we have one dimension for width, other for height, and other for color (being a 3D inside a 1D): this is nuts, I know. My brain still has difficulties to grasp.

We shall need to TensorFlow.js. Yes, TensorFlow means “flows of tensor”, since each layer is a tensor, mathematically speaking. Then, the output is another tensor, and so forth. This is indeed a “Tensor Flow”.

When I am coding in TensorFlow.js, if I do not pay attention, I can make a mess trying to image what the tensor looks like.

A grey scale image, seen as tensor. A colorful image would be like mountains on each square, instead of flat; a more information image, such as infrared for depth, would be more one dimension, not possible to say anything verbally. Source: Deep Learning with JavaScript: Neural Networks in TensorFlow.js. Book by Eric D. Nielsen, Shanqing Cai, and Stanley M. Bileschi

The core advantage of tensors, and there are many, like all mathematical tricks, is that you can work without concerning about dimensions. This is a general way to make reasoning, without worrying about applications. This is the magic of human abstraction, in action.