Windows paths are not always properly normalized

[Boddlnagg]

I'm having a problem where cquery is not able to find certain header files, and I'm suspecting that the reason has something to do with paths in Windows (maybe case-insensitivity?).

I found that all the files where it does not work have their cache files in
c@@Develop@git@src with relative file paths within, while those that work have their cache files in @c@@Develop@git@src with absolute file paths (such as C@@Develop@git@src@some_file.cpp) within.

And I'm getting messages of the form:

[Error - 09:20:41] Request textDocument/codeLens failed.
  Message: Unable to find file c:/Develop/git/src/some_file.cpp
  Code: -32603 

... although c:/Develop/git/src/some_file.cpp exists.

[jacobdufault]

Message: Unable to find file c:/Develop/git/src/some_file.cpp

We should change this message to be more clear, ie, 'File <foo.cc> is not indexed'. Are you blacklisting certain files from being indexed? A header file is only indexed if a cc file in the project which is indexed includes it.

[Boddlnagg]

I don't even know how I would blacklist a file from being indexed. The message appears for cc/cpp files, which are explicitly listed in compile_commands.json.

[Boddlnagg]

Another interesting observation: Things break only after I open the affected files in VS Code. When cquery starts indexing and I don't yet have the files open, I see the correct warning for the files, which means that the files can be analyzed correctly (and all headers are found). Only once I open the file in the editor, something breaks and the file becomes red all over because of header files which cannot be found.

[jacobdufault]

Can you reproduce on a small sample project that I can try locally?

[Boddlnagg]

Probably not, but I have some more information, which hopefully helps you to find the root cause:

As I said, the cache file that's working is in @c@@Develop@git@src\C@@Develop@git@src@source_file.cpp, and the one that does not work is in c@@Develop@git@src\source_file.cpp. I haven't yet understood under what conditions cquery/VSCode loads the one or the other, but it seems that it loads the bad file when I actually open source_file.cpp in the editor (but sometimes after a while it "fixes itself" and loads the correct one??).

I analyzed the difference between these two json files:

• The good one has absolute paths starting with C:/, whereas the bad one has them starting with c:/
• The bad one contains different (wrong!) args (see below)
• The "includes" array contains all the included headers in the good one, but only some of them in the bad one. The ones that are missing are exactly those of which VSCode then reports that it can't find them (and subsequent errors in the file are reported for things that refer to definitions out of those headers).
• The comments field in the types array contains has newlines encoded as \n in the good one, and \r\n\ in the bad one (and the bad one is missing types from the headers which it could not find).

Further analyzing the args:
What's in compile_commands.json:

{
    "directory": "C:\\Develop\\git\\out",
    "command": "cl.exe ... some other args ... -c -Fosource_file.obj C:/Develop/git/src/source_file.cpp",
    "file": "C:/Develop/git/src/source_file.cpp"
}

What's in the "good" cache file:

[
"cl.exe",
"-working-directory=C:\\Develop\\git\\out",
... some other args ...
"-FC:/Develop/git/out/osource_file.obj",
"C:/Develop/git/src/source_file.cpp",
"-resource-dir=C:/Develop/cquery/build/release/lib/LLVM-6.0.0-win64/lib/clang/6.0.0",
"-Wno-unknown-warning-option",
"-fparse-all-comments"
]

What's in the bad cache file:

[
"cl.exe",
"-working-directory=C:\\Develop\\git\\out",
"-fms-extensions",
"-fms-compatibility",
"-fdelayed-template-parsing",
... some other args taken from a different TU `other_source_file.cpp` from the same compile_commands.json ...
"-FC:/Develop/git/out/oother_source_file.obj",
"c:/Develop/git/src/source_file.cpp",
"-resource-dir=C:/Develop/cquery/build/release/lib/LLVM-6.0.0-win64/lib/clang/6.0.0",
"-Wno-unknown-warning-option",
"-fparse-all-comments"
]

There are at least two problems here:

• The bad one contains arguments from another TU, which is ... weird
• The -Fo parameter is parsed incorrectly (it is parsed as -F with o being part of its value, which turns -Fosource_file.obj into -FC:/Develop/git/out/osource_file.obj instead of -FoC:/Develop/git/out/source_file.obj)

[dschaefer]

Saw the same thing, I changed the case of C: in the compile_commands.json file to c: and it got further. Doesn't look like the lookup is taking case insensitivity on Windows (and to some degree on Mac) into account. But haven't dug further into it yet.

[Boddlnagg]

Indeed, when I change the case in compile_commands.json, it seems to work.

[jacobdufault]

I'm a bit surprised changing the case fixes this, but it sounds like we're missing normalization somewhere. Maybe project.cc is not normalizing.

[jacobdufault]

There have been a bunch of normalization changes. Please reopen if this is still an issue.

[Boddlnagg]

I just pulled from master, and the issue perists:

I'm still getting messages like Unable to find file c:/Develop/git/source_file.cpp, even though the file exists, and analyzing it fails because of missing headers (just like before).

Postprocessing compile_commands.json with sed --expression='s/C:\//c:\//g' makes everything work.

I'm also getting warnings Failed to normalize C:\Develop\git\out/osource_file.obj (related to wrong parsing of the -Fo parameter as mentioned above). I don't know if these have been there before or not.

[Boddlnagg]

Okay, I was finally able to build an minimal example to reproduce this (it's slightly extended from the one for #594):

cquery-test.zip

The point is that compile_commands.json (with uppercase C:/) doesn't work, while the version in compile_commands.fixed.json (with lowercase c:/) does work.

"doesn't work" means that I get Message: Unable to find file c:/Develop/cquery-test/src/test.cpp even though that file exists. In this small example it doesn't lead to headers not being found, but I guess that you'll still be able to analyze the root cause with this reproduction.