[BUG] npm audit ambiguously states "found 0 vulnerabilities"

[simonua]

Current Behavior:

We use Azure Artifacts which is a part of Azure DevOps to host our npm packages. We configured an upstream to the public npm registry to obtain public packages.
When executing npm audit against the public npm registry, we see a proper list of vulnerabilities for a large private package we maintain but when executing the same command against Azure Artifacts, we are ambiguously told "found 0 vulnerabilities".

This isn't new behavior, but it's still evident in npm 7. We have known that Azure Artifacts does not support the endpoint that is called by the audit. I don't recall the response it issues, but I suspect a 4xx of sorts.

This is problematic because it gives a false sense of security to developers, devops engineers, etc.

Expected Behavior:

What would be more appropriate to display would be a message that the audit-related endpoint is not implemented, errored, etc. and that the npm cli was thus unable to determine the status of vulnerabilities.

Steps To Reproduce:

1. Create a package with dependencies that have known vulnerabilities.
2. Point the npm cli to Azure Artifacts (set up a free Azure account, if necessary).
3. Execute npm audit to see the package appears to have zero vulnerabilities.

Environment:

The environment is fairly irrelevant here, but I run NodeJS 14.13.1 and npm 7.0.0.

[simonua]

@evilpacket, maybe something for your interest, good sir?

[isaacs]

I agree, the design intention is to not print audit information at all if the endpoint didn't return anything, so "0 vulnerabilities" is a bug. But I think a message indicating that we tried to audit and failed would be more informative and helpful.

[simonua]

Thank you! I'm very happy to debug, if needed, and give you a proper status code, etc. if you give me instructions on how to enable that.

[simonua]

Follow-up on what this looks like after installing 7.0.1. I no longer see an ambiguous message upon install, and npm audit correctly tells me about the error that was encountered. Thank you, @isaacs!