Skip to content

Softmax implementation in OneVersusAllModelParameters returns NaN even for scores of ~90.0f #3648

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rauhs opened this issue May 2, 2019 · 0 comments · Fixed by #3676
Closed
Assignees
Labels
bug Something isn't working

Comments

@rauhs
Copy link
Contributor

rauhs commented May 2, 2019

version: 0.11

This implementation:

private void NormalizeSoftmax(float[] scores, int count)
{
float sum = 0;
for (int i = 0; i < count; i++)
{
scores[i] = (float)Math.Exp(scores[i]);
sum += scores[i];
}
for (int i = 0; i < count; i++)
scores[i] = scores[i] / sum;
}

will result in NaN as the output even for moderate scores of around 90f:

      var foo = new[] { 90f, 2, -2f };
      NormalizeSoftmax(foo, 3);

Seen very easily in the debugger (the float cast will result in Inf). I'm getting back a score of roughly 102 for some of my data sets and models. Though they're extremely rare.

@rauhs rauhs changed the title Softmax imlementation in OneVersusAllModelParameters returns NaN even for scores of ~90.0f Softmax implementation in OneVersusAllModelParameters returns NaN even for scores of ~90.0f May 2, 2019
@najeeb-kazmi najeeb-kazmi self-assigned this May 7, 2019
@najeeb-kazmi najeeb-kazmi added the bug Something isn't working label May 7, 2019
@ghost ghost locked as resolved and limited conversation to collaborators Mar 22, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants