Pytorch tensor shape

Question

I have a simple question regarding the shapes of 2 different tensors - tensor_1 and tensor_2.

tensor_1.shape outputs torch.Size([784, 1]);
tensor_2.shape outputs torch.Size([784]).

I understand that the first one is rank-2 tensor, whereas the second is rank-1. What's hard for me is to conceptualize the difference between shape [784, 1] and [784].

Is it correct to think that tensor_1 has 784 rows and 1 column with a scalar inside each place? If so, why can't we call it simply a column vector (which is, in fact, rank-1 tensor), which also has values displayed vertically?

Similarly, can the shape of the second tensor ([784]) be imagined as 784 values inside a horizontal vector?

You’re a bit too hopeful about mapping Pytorch concepts onto mathematical concepts. There isn’t a distinction in Pytorch between row vectors and column vectors, for instance. There’s just rank-1 vectors. — Arya McCarthy
– Arya McCarthy, Commented Feb 17, 2021 at 18:32

Prajot Kuvalekar · Accepted Answer · 2021-02-17 18:25:14Z

3

You cant call tensor_1 as column vector because of its dimension . indexing that particular tensor is done in 2D
eg . tensor_1[1,1]

Coming to tensor_2 , its a scalar tensor having only one dimension.
And of course you can make it have a shape of tensor_1, just do

tensor_2 = tensor_2.unsqueeze(1)   #This method will make tensor_2 have a shape of tensor_1

answered Feb 17, 2021 at 18:25

Prajot Kuvalekar

6,7943 gold badges28 silver badges34 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Martynas Venckus Over a year ago

Thank you, Prajot. It makes sense. Maybe this is a dumb question, but what is there a specific reason behind having a tensor_1 of shape [784, 1] instead of just [784] (same as tensor_2)? We can easily reshape [784, 1] into [784] by tensor_1.view(784) and reduce the dimensionality.

Prajot Kuvalekar Over a year ago

there are many reason for having shape of [784, 1] instaed of [784]. One reason would be batches.when you have single value inputs, & you want to send it through a batches,and your loss function expects your value in 2D shape rather than 1D shape.

Collectives™ on Stack Overflow

Pytorch tensor shape

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related