Skip to content

[clang][ExtractAPI] Create extractapi::RecordLocation #65891

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Arsenic-ATG
Copy link
Member

Create and use extractapi::RecordLocation instead of conventional clang::PresumedLoc to track the location of an APIRecord, this reduces the dependency of APISet on SourceManager and would help if someone wants to create APISet from JSON Serialized SymbolGraph.

These changes also add extractapi::CommentLine which is similar to RawComment::CommentLine but use RecordLocation instead of PresumedLoc.

This was initially being tracked on llvm Phabricator
Differential Revision: https://reviews.llvm.org/D157810

Create and use extractapi::RecordLocation instead of conventional
clang::PresumedLoc to track the location of an APIRecord, this reduces
the dependency of APISet on SourceManager and would help if someone
wants to create APISet from JSON Serialized SymbolGraph.

These changes also add extractapi::CommentLine which is similar to
RawComment::CommentLine but use RecordLocation instead of PresumedLoc.

Differential Revision: https://reviews.llvm.org/D157810
@Arsenic-ATG Arsenic-ATG added the clang Clang issues not falling into any other category label Sep 10, 2023
@Arsenic-ATG Arsenic-ATG self-assigned this Sep 10, 2023
@Arsenic-ATG Arsenic-ATG requested a review from a team as a code owner September 10, 2023 11:26
@llvmbot
Copy link
Member

llvmbot commented Sep 10, 2023

@llvm/pr-subscribers-clang

Changes

Create and use extractapi::RecordLocation instead of conventional clang::PresumedLoc to track the location of an APIRecord, this reduces the dependency of APISet on SourceManager and would help if someone wants to create APISet from JSON Serialized SymbolGraph.

These changes also add extractapi::CommentLine which is similar to RawComment::CommentLine but use RecordLocation instead of PresumedLoc.

This was initially being tracked on llvm Phabricator
Differential Revision: https://reviews.llvm.org/D157810

Patch is 92.55 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/65891.diff

5 Files Affected:

  • (modified) clang/include/clang/ExtractAPI/API.h (+212-151)
  • (modified) clang/include/clang/ExtractAPI/ExtractAPIVisitor.h (+43-135)
  • (modified) clang/lib/ExtractAPI/API.cpp (+46-44)
  • (modified) clang/lib/ExtractAPI/Serialization/SymbolGraphSerializer.cpp (+5-5)
  • (modified) clang/tools/libclang/CXExtractAPI.cpp (+6-3)
diff --git a/clang/include/clang/ExtractAPI/API.h b/clang/include/clang/ExtractAPI/API.h
index b4c0e0ad39cdf2a..d4367caf647550f 100644
--- a/clang/include/clang/ExtractAPI/API.h
+++ b/clang/include/clang/ExtractAPI/API.h
@@ -134,7 +134,48 @@ class Template {
   bool empty() const { return Parameters.empty() && Constraints.empty(); }
 };
 
-/// DocComment is a vector of RawComment::CommentLine.
+/// Slightly cut down version of PresumedLoc to suit the needs of
+/// ExtractAPI.
+class RecordLocation {
+  unsigned Line, Col;
+  std::string Filename;
+
+public:
+  RecordLocation(const unsigned Line, const unsigned Col,
+                 std::string Filename = "")
+      : Line(Line), Col(Col), Filename(Filename) {}
+  RecordLocation(const PresumedLoc &Location)
+      : Line(Location.getLine()), Col(Location.getColumn()),
+        Filename(Location.getFilename()) {}
+  RecordLocation() = default;
+
+  bool isInvalid() const { return Filename.empty(); }
+  bool isValid() const { return !isInvalid(); }
+
+  const char *getFilename() const { return Filename.c_str(); }
+
+  unsigned getLine() const { return Line; }
+
+  unsigned getColumn() const { return Col; }
+};
+
+/// Store a documentation comment line of an APIRecord
+///
+/// Similar to RawComment::CommentLine but use RecordLocation instead
+/// of PresumedLoc.
+struct CommentLine {
+
+  std::string Text;
+  RecordLocation Begin;
+  RecordLocation End;
+
+  CommentLine(llvm::StringRef Text, RecordLocation Begin, RecordLocation End)
+      : Text(Text), Begin(Begin), End(End) {}
+  CommentLine(const RawComment::CommentLine RComment)
+      : Text(RComment.Text), Begin(RComment.Begin), End(RComment.End) {}
+};
+
+/// DocComment is a vector of extractapi::CommentLine.
 ///
 /// Each line represents one line of striped documentation comment,
 /// with source range information. This simplifies calculating the source
@@ -147,7 +188,7 @@ class Template {
 ///   ///     with multiple lines.
 ///       ^~~~~~~~~~~~~~~~~~~~~~~'         Second line.
 /// \endcode
-using DocComment = std::vector;
+using DocComment = std::vector;
 
 // Classes deriving from APIRecord need to have USR be the first constructor
 // argument. This is so that they are compatible with `addTopLevelRecord`
@@ -223,7 +264,7 @@ struct APIRecord {
 
   StringRef USR;
   StringRef Name;
-  PresumedLoc Location;
+  RecordLocation Location;
   AvailabilitySet Availabilities;
   LinkageInfo Linkage;
 
@@ -256,7 +297,7 @@ struct APIRecord {
   APIRecord() = delete;
 
   APIRecord(RecordKind Kind, StringRef USR, StringRef Name,
-            PresumedLoc Location, AvailabilitySet Availabilities,
+            RecordLocation Location, AvailabilitySet Availabilities,
             LinkageInfo Linkage, const DocComment &Comment,
             DeclarationFragments Declaration, DeclarationFragments SubHeading,
             bool IsFromSystemHeader)
@@ -273,7 +314,7 @@ struct APIRecord {
 };
 
 struct NamespaceRecord : APIRecord {
-  NamespaceRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  NamespaceRecord(StringRef USR, StringRef Name, RecordLocation Loc,
                   AvailabilitySet Availabilities, LinkageInfo Linkage,
                   const DocComment &Comment, DeclarationFragments Declaration,
                   DeclarationFragments SubHeading, bool IsFromSystemHeader)
@@ -290,19 +331,20 @@ struct NamespaceRecord : APIRecord {
 struct GlobalFunctionRecord : APIRecord {
   FunctionSignature Signature;
 
-  GlobalFunctionRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  GlobalFunctionRecord(StringRef USR, StringRef Name,
+                       RecordLocation RecordLocation,
                        AvailabilitySet Availabilities, LinkageInfo Linkage,
                        const DocComment &Comment,
                        DeclarationFragments Declaration,
                        DeclarationFragments SubHeading,
                        FunctionSignature Signature, bool IsFromSystemHeader)
-      : APIRecord(RK_GlobalFunction, USR, Name, Loc, std::move(Availabilities),
-                  Linkage, Comment, Declaration, SubHeading,
-                  IsFromSystemHeader),
+      : APIRecord(RK_GlobalFunction, USR, Name, RecordLocation,
+                  std::move(Availabilities), Linkage, Comment, Declaration,
+                  SubHeading, IsFromSystemHeader),
         Signature(Signature) {}
 
   GlobalFunctionRecord(RecordKind Kind, StringRef USR, StringRef Name,
-                       PresumedLoc Loc, AvailabilitySet Availabilities,
+                       RecordLocation Loc, AvailabilitySet Availabilities,
                        LinkageInfo Linkage, const DocComment &Comment,
                        DeclarationFragments Declaration,
                        DeclarationFragments SubHeading,
@@ -322,7 +364,8 @@ struct GlobalFunctionRecord : APIRecord {
 struct GlobalFunctionTemplateRecord : GlobalFunctionRecord {
   Template Templ;
 
-  GlobalFunctionTemplateRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  GlobalFunctionTemplateRecord(StringRef USR, StringRef Name,
+                               RecordLocation Loc,
                                AvailabilitySet Availabilities,
                                LinkageInfo Linkage, const DocComment &Comment,
                                DeclarationFragments Declaration,
@@ -342,7 +385,7 @@ struct GlobalFunctionTemplateRecord : GlobalFunctionRecord {
 
 struct GlobalFunctionTemplateSpecializationRecord : GlobalFunctionRecord {
   GlobalFunctionTemplateSpecializationRecord(
-      StringRef USR, StringRef Name, PresumedLoc Loc,
+      StringRef USR, StringRef Name, RecordLocation Loc,
       AvailabilitySet Availabilities, LinkageInfo Linkage,
       const DocComment &Comment, DeclarationFragments Declaration,
       DeclarationFragments SubHeading, FunctionSignature Signature,
@@ -359,17 +402,18 @@ struct GlobalFunctionTemplateSpecializationRecord : GlobalFunctionRecord {
 
 /// This holds information associated with global functions.
 struct GlobalVariableRecord : APIRecord {
-  GlobalVariableRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  GlobalVariableRecord(StringRef USR, StringRef Name,
+                       RecordLocation RecordLocation,
                        AvailabilitySet Availabilities, LinkageInfo Linkage,
                        const DocComment &Comment,
                        DeclarationFragments Declaration,
                        DeclarationFragments SubHeading, bool IsFromSystemHeader)
-      : APIRecord(RK_GlobalVariable, USR, Name, Loc, std::move(Availabilities),
-                  Linkage, Comment, Declaration, SubHeading,
-                  IsFromSystemHeader) {}
+      : APIRecord(RK_GlobalVariable, USR, Name, RecordLocation,
+                  std::move(Availabilities), Linkage, Comment, Declaration,
+                  SubHeading, IsFromSystemHeader) {}
 
   GlobalVariableRecord(RecordKind Kind, StringRef USR, StringRef Name,
-                       PresumedLoc Loc, AvailabilitySet Availabilities,
+                       RecordLocation Loc, AvailabilitySet Availabilities,
                        LinkageInfo Linkage, const DocComment &Comment,
                        DeclarationFragments Declaration,
                        DeclarationFragments SubHeading, bool IsFromSystemHeader)
@@ -387,7 +431,8 @@ struct GlobalVariableRecord : APIRecord {
 struct GlobalVariableTemplateRecord : GlobalVariableRecord {
   Template Templ;
 
-  GlobalVariableTemplateRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  GlobalVariableTemplateRecord(StringRef USR, StringRef Name,
+                               RecordLocation Loc,
                                AvailabilitySet Availabilities,
                                LinkageInfo Linkage, const DocComment &Comment,
                                DeclarationFragments Declaration,
@@ -405,7 +450,7 @@ struct GlobalVariableTemplateRecord : GlobalVariableRecord {
 
 struct GlobalVariableTemplateSpecializationRecord : GlobalVariableRecord {
   GlobalVariableTemplateSpecializationRecord(
-      StringRef USR, StringRef Name, PresumedLoc Loc,
+      StringRef USR, StringRef Name, RecordLocation Loc,
       AvailabilitySet Availabilities, LinkageInfo Linkage,
       const DocComment &Comment, DeclarationFragments Declaration,
       DeclarationFragments SubHeading, bool IsFromSystemHeader)
@@ -423,7 +468,7 @@ struct GlobalVariableTemplatePartialSpecializationRecord
   Template Templ;
 
   GlobalVariableTemplatePartialSpecializationRecord(
-      StringRef USR, StringRef Name, PresumedLoc Loc,
+      StringRef USR, StringRef Name, RecordLocation Loc,
       AvailabilitySet Availabilities, LinkageInfo Linkage,
       const DocComment &Comment, DeclarationFragments Declaration,
       DeclarationFragments SubHeading, class Template Template,
@@ -441,13 +486,14 @@ struct GlobalVariableTemplatePartialSpecializationRecord
 
 /// This holds information associated with enum constants.
 struct EnumConstantRecord : APIRecord {
-  EnumConstantRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  EnumConstantRecord(StringRef USR, StringRef Name,
+                     RecordLocation RecordLocation,
                      AvailabilitySet Availabilities, const DocComment &Comment,
                      DeclarationFragments Declaration,
                      DeclarationFragments SubHeading, bool IsFromSystemHeader)
-      : APIRecord(RK_EnumConstant, USR, Name, Loc, std::move(Availabilities),
-                  LinkageInfo::none(), Comment, Declaration, SubHeading,
-                  IsFromSystemHeader) {}
+      : APIRecord(RK_EnumConstant, USR, Name, RecordLocation,
+                  std::move(Availabilities), LinkageInfo::none(), Comment,
+                  Declaration, SubHeading, IsFromSystemHeader) {}
 
   static bool classof(const APIRecord *Record) {
     return Record->getKind() == RK_EnumConstant;
@@ -461,11 +507,11 @@ struct EnumConstantRecord : APIRecord {
 struct EnumRecord : APIRecord {
   SmallVector> Constants;
 
-  EnumRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  EnumRecord(StringRef USR, StringRef Name, RecordLocation RecordLocation,
              AvailabilitySet Availabilities, const DocComment &Comment,
              DeclarationFragments Declaration, DeclarationFragments SubHeading,
              bool IsFromSystemHeader)
-      : APIRecord(RK_Enum, USR, Name, Loc, std::move(Availabilities),
+      : APIRecord(RK_Enum, USR, Name, RecordLocation, std::move(Availabilities),
                   LinkageInfo::none(), Comment, Declaration, SubHeading,
                   IsFromSystemHeader) {}
 
@@ -479,13 +525,14 @@ struct EnumRecord : APIRecord {
 
 /// This holds information associated with struct fields.
 struct StructFieldRecord : APIRecord {
-  StructFieldRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  StructFieldRecord(StringRef USR, StringRef Name,
+                    RecordLocation RecordLocation,
                     AvailabilitySet Availabilities, const DocComment &Comment,
                     DeclarationFragments Declaration,
                     DeclarationFragments SubHeading, bool IsFromSystemHeader)
-      : APIRecord(RK_StructField, USR, Name, Loc, std::move(Availabilities),
-                  LinkageInfo::none(), Comment, Declaration, SubHeading,
-                  IsFromSystemHeader) {}
+      : APIRecord(RK_StructField, USR, Name, RecordLocation,
+                  std::move(Availabilities), LinkageInfo::none(), Comment,
+                  Declaration, SubHeading, IsFromSystemHeader) {}
 
   static bool classof(const APIRecord *Record) {
     return Record->getKind() == RK_StructField;
@@ -499,13 +546,13 @@ struct StructFieldRecord : APIRecord {
 struct StructRecord : APIRecord {
   SmallVector> Fields;
 
-  StructRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  StructRecord(StringRef USR, StringRef Name, RecordLocation RecordLocation,
                AvailabilitySet Availabilities, const DocComment &Comment,
                DeclarationFragments Declaration,
                DeclarationFragments SubHeading, bool IsFromSystemHeader)
-      : APIRecord(RK_Struct, USR, Name, Loc, std::move(Availabilities),
-                  LinkageInfo::none(), Comment, Declaration, SubHeading,
-                  IsFromSystemHeader) {}
+      : APIRecord(RK_Struct, USR, Name, RecordLocation,
+                  std::move(Availabilities), LinkageInfo::none(), Comment,
+                  Declaration, SubHeading, IsFromSystemHeader) {}
 
   static bool classof(const APIRecord *Record) {
     return Record->getKind() == RK_Struct;
@@ -518,22 +565,22 @@ struct StructRecord : APIRecord {
 struct CXXFieldRecord : APIRecord {
   AccessControl Access;
 
-  CXXFieldRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  CXXFieldRecord(StringRef USR, StringRef Name, RecordLocation RecordLocation,
                  AvailabilitySet Availabilities, const DocComment &Comment,
                  DeclarationFragments Declaration,
                  DeclarationFragments SubHeading, AccessControl Access,
                  bool IsFromSystemHeader)
-      : APIRecord(RK_CXXField, USR, Name, Loc, std::move(Availabilities),
-                  LinkageInfo::none(), Comment, Declaration, SubHeading,
-                  IsFromSystemHeader),
+      : APIRecord(RK_CXXField, USR, Name, RecordLocation,
+                  std::move(Availabilities), LinkageInfo::none(), Comment,
+                  Declaration, SubHeading, IsFromSystemHeader),
         Access(Access) {}
 
   CXXFieldRecord(RecordKind Kind, StringRef USR, StringRef Name,
-                 PresumedLoc Loc, AvailabilitySet Availabilities,
+                 RecordLocation RecordLocation, AvailabilitySet Availabilities,
                  const DocComment &Comment, DeclarationFragments Declaration,
                  DeclarationFragments SubHeading, AccessControl Access,
                  bool IsFromSystemHeader)
-      : APIRecord(Kind, USR, Name, Loc, std::move(Availabilities),
+      : APIRecord(Kind, USR, Name, RecordLocation, std::move(Availabilities),
                   LinkageInfo::none(), Comment, Declaration, SubHeading,
                   IsFromSystemHeader),
         Access(Access) {}
@@ -549,7 +596,7 @@ struct CXXFieldRecord : APIRecord {
 struct CXXFieldTemplateRecord : CXXFieldRecord {
   Template Templ;
 
-  CXXFieldTemplateRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  CXXFieldTemplateRecord(StringRef USR, StringRef Name, RecordLocation Loc,
                          AvailabilitySet Availabilities,
                          const DocComment &Comment,
                          DeclarationFragments Declaration,
@@ -572,11 +619,11 @@ struct CXXMethodRecord : APIRecord {
   CXXMethodRecord() = delete;
 
   CXXMethodRecord(RecordKind Kind, StringRef USR, StringRef Name,
-                  PresumedLoc Loc, AvailabilitySet Availabilities,
+                  RecordLocation RecordLocation, AvailabilitySet Availabilities,
                   const DocComment &Comment, DeclarationFragments Declaration,
                   DeclarationFragments SubHeading, FunctionSignature Signature,
                   AccessControl Access, bool IsFromSystemHeader)
-      : APIRecord(Kind, USR, Name, Loc, std::move(Availabilities),
+      : APIRecord(Kind, USR, Name, RecordLocation, std::move(Availabilities),
                   LinkageInfo::none(), Comment, Declaration, SubHeading,
                   IsFromSystemHeader),
         Signature(Signature), Access(Access) {}
@@ -585,14 +632,15 @@ struct CXXMethodRecord : APIRecord {
 };
 
 struct CXXConstructorRecord : CXXMethodRecord {
-  CXXConstructorRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  CXXConstructorRecord(StringRef USR, StringRef Name,
+                       RecordLocation RecordLocation,
                        AvailabilitySet Availabilities,
                        const DocComment &Comment,
                        DeclarationFragments Declaration,
                        DeclarationFragments SubHeading,
                        FunctionSignature Signature, AccessControl Access,
                        bool IsFromSystemHeader)
-      : CXXMethodRecord(RK_CXXConstructorMethod, USR, Name, Loc,
+      : CXXMethodRecord(RK_CXXConstructorMethod, USR, Name, RecordLocation,
                         std::move(Availabilities), Comment, Declaration,
                         SubHeading, Signature, Access, IsFromSystemHeader) {}
   static bool classof(const APIRecord *Record) {
@@ -604,13 +652,14 @@ struct CXXConstructorRecord : CXXMethodRecord {
 };
 
 struct CXXDestructorRecord : CXXMethodRecord {
-  CXXDestructorRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  CXXDestructorRecord(StringRef USR, StringRef Name,
+                      RecordLocation RecordLocation,
                       AvailabilitySet Availabilities, const DocComment &Comment,
                       DeclarationFragments Declaration,
                       DeclarationFragments SubHeading,
                       FunctionSignature Signature, AccessControl Access,
                       bool IsFromSystemHeader)
-      : CXXMethodRecord(RK_CXXDestructorMethod, USR, Name, Loc,
+      : CXXMethodRecord(RK_CXXDestructorMethod, USR, Name, RecordLocation,
                         std::move(Availabilities), Comment, Declaration,
                         SubHeading, Signature, Access, IsFromSystemHeader) {}
   static bool classof(const APIRecord *Record) {
@@ -622,14 +671,15 @@ struct CXXDestructorRecord : CXXMethodRecord {
 };
 
 struct CXXStaticMethodRecord : CXXMethodRecord {
-  CXXStaticMethodRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  CXXStaticMethodRecord(StringRef USR, StringRef Name,
+                        RecordLocation RecordLocation,
                         AvailabilitySet Availabilities,
                         const DocComment &Comment,
                         DeclarationFragments Declaration,
                         DeclarationFragments SubHeading,
                         FunctionSignature Signature, AccessControl Access,
                         bool IsFromSystemHeader)
-      : CXXMethodRecord(RK_CXXStaticMethod, USR, Name, Loc,
+      : CXXMethodRecord(RK_CXXStaticMethod, USR, Name, RecordLocation,
                         std::move(Availabilities), Comment, Declaration,
                         SubHeading, Signature, Access, IsFromSystemHeader) {}
   static bool classof(const APIRecord *Record) {
@@ -641,14 +691,15 @@ struct CXXStaticMethodRecord : CXXMethodRecord {
 };
 
 struct CXXInstanceMethodRecord : CXXMethodRecord {
-  CXXInstanceMethodRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  CXXInstanceMethodRecord(StringRef USR, StringRef Name,
+                          RecordLocation RecordLocation,
                           AvailabilitySet Availabilities,
                           const DocComment &Comment,
                           DeclarationFragments Declaration,
                           DeclarationFragments SubHeading,
                           FunctionSignature Signature, AccessControl Access,
                           bool IsFromSystemHeader)
-      : CXXMethodRecord(RK_CXXInstanceMethod, USR, Name, Loc,
+      : CXXMethodRecord(RK_CXXInstanceMethod, USR, Name, RecordLocation,
                         std::move(Availabilities), Comment, Declaration,
                         SubHeading, Signature, Access, IsFromSystemHeader) {}
 
@@ -663,7 +714,7 @@ struct CXXInstanceMethodRecord : CXXMethodRecord {
 struct CXXMethodTemplateRecord : CXXMethodRecord {
   Template Templ;
 
-  CXXMethodTemplateRecord(StringRef USR, StringRef Name, PresumedLoc Loc,
+  CXXMethodTemplateRecord(StringRef USR, StringRef Name, RecordLocation Loc,
                           AvailabilitySet Availabilities,
                           const DocComment &Comment,
                           DeclarationFragments Declaration,
@@ -682,7 +733,7 @@ struct CXXMet...

/// ExtractAPI.
class RecordLocation {
unsigned Line, Col;
std::string Filename;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Due to some reasons, I am currently unable to open the original deferential revision (https://reviews.llvm.org/D157810).

So for those who are facing the same issue, It had the following review comment from @ributzka

There is an opportunity for optimization by avoiding the allocation of separate strings for each source location, especially since many source locations will be in the same file. As an example, APISet utilizes a BumpPtrAllocator to allocate and deduplicate strings. It is recommended to consider using the same allocator or a similar concept.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang Clang issues not falling into any other category
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants