Every Index based Operation you’ll ever need in Pytorch

Original article was published on Deep Learning on Medium

There are two types of operations in Pytorch, In-place Operations & Out-of-place Operations. If you are programming in python for some time, you have certainly used them somewhere but may not be familiar with the terminology.

Tutorials Point tutorial has a very apt definition for In-place Operation,

In-place operation is an operation that directly changes the content of a given linear algebra, vector, matrices (Tensor) without making a copy.

In PyTorch, all operations on the tensor that operate in-place on it will have an _ postfix. Eg. we have both add() and add_(). However, some operations only have one type of version like .narrow has only out-of-place and .fill have only in-place version.

In torch.tensor, we have 10 Index Operations based functions.

  • index_add_
  • index_add
  • index_copy_
  • index_copy
  • index_fill_
  • index_fill
  • index_put_
  • index_put
  • index_select