Tensors — Representation of Data In Neural Networks

Source: Deep Learning on Medium

Tensors — Representation of Data In Neural Networks

What you will learn?

  • Tensors
  • Key Attributes Of Tensor
  • Real-World Examples of Tensors

What is Tensor?

It is a container of Data, which helps to store different dimensions of Data in Neural Networks

Google’s Machine Learning Library TensorFlow was named after them.

Scalar or Rank 0 or 0-D Tensors

  • A tensor that contains only one number is called a scalar.
  • A Scalar tensor has 0 axes (ndim == 0)
  • The number of axes is called a rank of the tensor.

Code :

ignition = tf.Variable(451, tf.int16)

floating = tf.Variable(3.14159265359, tf.float64)

its_complicated = tf.Variable(12.3–4.85j, tf.complex64)

Vector or Rank 1 or 1-D Tensors

  • An array of numbers is called a vector, or 1-D Tensor.


mystr = tf.Variable([“Hello”], tf.string)

cool_numbers = tf.Variable([3.14159, 2.71828], tf.float32)

first_primes = tf.Variable([2, 3, 5, 7, 11], tf.int32)

its_very_complicated = tf.Variable([12.3–4.85j, 7.5–6.23j], tf.complex64)

Matrices or Rank 2 or 2-D Tensor

  • An array of vectors is a matrix or 2-D Tensor
  • It consists of two axes, row and columns


mymat = tf.Variable([[7],[11]], tf.int16)

myxor = tf.Variable([[False, True],[True, False]], tf.bool)

3-D Tensor and Higher Dimensional Tensor

  • If you pack such matrices in a new array, you obtain a 3-D Tensor.
  • By packing 3D tensors in an array, you can create a 4D tensor, and so on.


mymat = tf.Variable([[7],[11]], tf.int16)

myxor = tf.Variable([[False, True],[True, False]], tf.bool)

linear_squares = tf.Variable([[4], [9], [16], [25]], tf.int32)

squarish_squares = tf.Variable([ [4, 9], [16, 25] ], tf.int32)

rank_of_squares = tf.rank(squarish_squares)

mymatC = tf.Variable([[7],[11]], tf.int32)

Key Attributes Of Tensors

  • Number of Axes(Rank)
  • Shape

This is a tuple of integers that describes how many dimensions the tensor has along each axis.

This is the type of data contained in the tensor

Ex: float32, uint8, float64, and so on.

Real-World Examples of Tensors

  • Vector Data: 2D tensors of shape (samples, features)
  • Time-Series Data or Sequence Data:

3D tensors of shape (samples, timesteps,features)

4D tensors of shape (samples, height, width, channels)

5D tensors of shape (samples, frames, height, width, channels)

Vector Data

This is an array of vectors, where the first axis is the samples axis(sample dimensions) and the second axis is the features axis.


An actuarial dataset of people, where we consider each person’s age, ZIP code, and income. Each person can be characterized as a vector of 3 values, and thus an entire dataset of 100,000 people can be stored in a 2D tensor of shape (100000, 3).

Time- Series or Sequence Data

Whenever time matters in your data (or the notion of sequence order), it makes sense to store it in a 3D tensor with an explicit time axis.

  • (samples, timesteps, features)

Example: A dataset of stock prices. Every minute, we store the current price of the stock, the highest price in the past minute, and the lowest price in the past minute. Thus every minute is encoded as a 3D vector, an entire day of trading is encoded as a 2D tensor of shape (390, 3) (there are 390 minutes in a trading day), and 250 days’ worth of data can be stored in a 3D tensor of shape (250,390, 3). Here, each sample would be one day’s worth of data.

Image Data

  • Images typically have three dimensions: height, width, and colour depth.
  • Grayscale Images contains only one channel.
  • RGB Images has three colour channels.
  • (samples, height, width, channels)


A batch of 128 grayscale images of size 256 × 256 could thus be stored in a tensor of shape (128, 256, 256, 1), and a batch of 128 color images could be stored in a tensor of shape (128, 256, 256, 3)

  • RGB Images : (128, 256, 256, 3)
  • Grayscale Images : (128, 256, 256, 1)

Video Data

Video data is one of the few types of real-world data for which you’ll need 5D tensors. A video can be understood as a sequence of frames, each frame being a color image. Because each frame can be stored in a 3D tensor (height, width, color_depth), a sequence of frames can be stored in a 4D tensor (frames, height, width, color_depth), and thus a batch of different videos can be stored in a 5D tensor of shape (samples, frames, height, width, color_depth).

For instance, a 60-second, 144 × 256 YouTube video clip sampled at 4 frames per second would have 240 frames. A batch of four such video clips would be stored in a tensor of shape (4, 240, 144, 256, 3). That’s a total of 106,168,320 values! If the dtype of the tensor was float32, then each value would be stored in 32 bits, so the tensor would represent 405 MB. Heavy! Videos you encounter in real life are much lighter, because they aren’t stored in float32, and they’re typically compressed by a large factor (such as in the MPEG format).

Thank You!

Stay Tuned & Stay Connected With #MLAIT

Follow Us For More blogs on ML, AI and Cloud…