Introduction to Tensor with Tensorflow

Tensor is a multi-dimensional array used to store data in machine learning and deep learning frameworks such as TensorFlow. Tensors are the fundamental data structure in TensorFlow and they represent the flow of data through a computation graph. Tensors generalize scalars, vectors and matrices to higher dimensions.

Types of Tensors

Tensors in TensorFlow can take various forms depending on their number of dimensions.

• Scalar (0D tensor): A single number, such as 5 or -3.14.
• Vector (1D tensor): A one-dimensional array, such as [1, 2, 3, 4].
• Matrix (2D tensor): A two-dimensional array, like a table with rows and columns: [[1, 2], [3, 4]].
• 3D Tensor: A three-dimensional array, like a stack of matrices: [[[1, 2], [3, 4]], [[5, 6], [7, 8]]].

Higher-dimensional tensors: Tensors with more than three dimensions are often used to represent more complex data, such as color images (which might be represented as a 4D tensor with shape [batch_size, height, width, channels]).

Representation of Tensors

TensorFlow framework is designed for high-performance numerical computation, operates primarily using tensors. When you use TensorFlow, you define your model, train it and perform operations using tensors.

A tensor in TensorFlow is represented as an object that has:

• Shape: The dimensions of the tensor (e.g., [2, 3] for a matrix with 2 rows and 3 columns).
• Rank: The number of dimensions of the tensor (e.g., a scalar has rank 0, a vector has rank 1, a matrix has rank 2, etc.).
• Data type: The type of the elements in the tensor, such as float32, int32 or string.
• Device: The device on which the tensor resides (e.g., CPU, GPU).

TensorFlow provides a variety of operations that can be applied to tensors, including mathematical operations, transformations and reshaping.

Basic Tensor Operations in TensorFlow

TensorFlow provides a large set of tensor operations, allowing for efficient manipulation of data. Below are some of the most commonly used tensor operations in TensorFlow:

1. Creating Tensors

You can create tensors using TensorFlow’s tf.Tensor() or its various helper functions, such as tf.constant(), tf.Variable() or tf.zeros():

import tensorflow as tf

scalar_tensor = tf.constant(5)
vector_tensor = tf.constant([1, 2, 3, 4])

matrix_tensor = tf.constant([[1, 2], [3, 4]])
tensor_3d = tf.constant([[[1, 2], [3, 4]], [[5, 6], [7, 8]]])
zeros_tensor = tf.zeros([3, 3])
ones_tensor = tf.ones([2, 2])

Output:

2. Tensor Operations

TensorFlow supports various operations that can be performed on tensors, such as element-wise operations, matrix multiplication, reshaping and more.

import tensorflow as tf

matrix_tensor = tf.constant([[1, 2], [3, 4]])

ones_tensor = tf.ones_like(matrix_tensor)
tensor_add = tf.add(matrix_tensor, ones_tensor)
matrix_mult = tf.matmul(matrix_tensor, matrix_tensor)
reshaped_tensor = tf.reshape(matrix_tensor, [4, 1])

transpose_tensor = tf.transpose(matrix_tensor)

Output:

3. Accessing Elements in a Tensor

You can access specific elements within a tensor using indices. Similar to how you access elements in Python lists or NumPy arrays, TensorFlow provides slicing and indexing operations.

import tensorflow as tf

vector_tensor = tf.constant([1, 2, 3, 4])
first_element = vector_tensor[0]
matrix_tensor = tf.constant([[1, 2], [3, 4]])

matrix_slice = matrix_tensor[:2, :]

Output:

4. Changing Tensor Shape

You can change the shape of a tensor by reshaping it. This is often used when you need to feed data into a model with specific input dimensions.

reshaped_tensor = tf.reshape(matrix_tensor, [4, 1])

Output:

Tensors in Neural Networks

In neural networks, tensors represent various forms of data throughout the model’s architecture. For example:

• Input data (e.g., images, text or tabular data) is represented as tensors. For an image, it might be a 4D tensor with the shape [batch_size, height, width, channels].
• Weights and biases of layers are represented as tensors. These are the parameters the model learns during training.
• Output of layers after applying operations like convolutions, activations and fully connected layers is also represented as tensors.

How Tensors Flow Through a Neural Network in TensorFlow?

In this example:

• The input data X_train is a tensor of shape [100, 10], representing 100 samples with 10 features.
• The output of the model is a tensor of shape [100, 1], representing the predictions for each of the 100 samples.
• The weights and biases of the model layers are also tensors and they are updated during training via backpropagation.

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

X_train = tf.random.normal([100, 10])
y_train = tf.random.normal([100, 1])

model = Sequential([
    Dense(64, activation='relu', input_shape=(10,)),
    Dense(32, activation='relu'),
    Dense(1)
])

model.compile(optimizer='adam', loss='mse')

model.fit(X_train, y_train, epochs=10)

Output: