Original article was published on Deep Learning on Medium
There are two types of operations in Pytorch, In-place Operations & Out-of-place Operations. If you are programming in python for some time, you have certainly used them somewhere but may not be familiar with the terminology.
Tutorials Point tutorial has a very apt definition for In-place Operation,
In-place operation is an operation that directly changes the content of a given linear algebra, vector, matrices (Tensor) without making a copy.
In PyTorch, all operations on the tensor that operate in-place on it will have an _ postfix. Eg. we have both add() and add_(). However, some operations only have one type of version like .narrow has only out-of-place and .fill have only in-place version.
In torch.tensor, we have 10 Index Operations based functions.