Skip to content

why using np.arange(n) rather than one-hot as labels? #301

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
leviome opened this issue Nov 10, 2022 · 2 comments
Closed

why using np.arange(n) rather than one-hot as labels? #301

leviome opened this issue Nov 10, 2022 · 2 comments

Comments

@leviome
Copy link

leviome commented Nov 10, 2022

Thanks for sharing the nice work. But I didn't fully understand the labels usage:

labels = np.arange(n) # (0, 1, 2, 3, 4, ...)
loss_i = cross_entropy_loss(logits, labels, axis=0)
loss_t = cross_entropy_loss(logits, labels, axis=1)
loss = (loss_i + loss_t)/2

the logits is the cosine similarity matrix for n image-text pairs. But cross entropy loss will make the rows/cols close to [0, 1, 2, 3, ...] (np.arange(n)), leading to logits[0][0]=0, equally the similarity of image[0] and text[0] is getting to 0.
But our goal is to enlarge the cosine similarity of image[0] and text[0].

So, could some awesome guys help me understand why using np.arange(n) rather than using one-hot as labels?

@leviome
Copy link
Author

leviome commented Dec 5, 2022

Ok! I worked it out

@leviome leviome closed this as completed Dec 5, 2022
@yeknafar
Copy link

can you expalin this ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants