[win][x64] Unwind v2 3/n: Add support for emitting unwind v2 information (equivalent to MSVC /d2epilogunwind) #129142
Conversation
llvmbot
added
clang
|
@llvm/pr-subscribers-backend-arm @llvm/pr-subscribers-backend-x86 Author: Daniel Paoliello (dpaoliello) ChangesAdds support for emitting Windows x64 Unwind V2 information, includes support Unwind v2 adds information about the epilogs in functions such that the unwinder can unwind even in the middle of an epilog, without having to disassembly the function to see what has or has not been cleaned up. Unwind v2 requires that all epilogs are in "canonical" form:
This change adds a pass to validate epilogs in modules that have Unwind v2 enabled and, if they pass, emits new pseudo instructions to MC that 1) note that the function is using unwind v2 and 2) mark the start of the epilog (this is either the first Note that the unwind v2 table only marks the size of the epilog in the "header" unwind code, but it's possible for epilogs to use different terminator instructions thus they are not all the same size. As a work around for this, MC will assume that all terminator instructions are 1-byte long - this still works correctly with the Windows unwinder as it is only using the size to do a range check to see if a thread is in an epilog or not, and since the instruction pointer will never be in the middle of an instruction and the terminator is always at the end of an epilog the range check will function correctly. This does mean, however, that the "at end" optimization (where an epilog unwind code can be elided if the last epilog is at the end of the function) can only be used if the terminator is 1-byte long. One other complication with the implementation is that the unwind table for a function is emitted during streaming, however we can't calculate the distance between an epilog and the end of the function at that time as layout hasn't been completed yet (thus some instructions may be relaxed). To work around this, epilog unwind codes are emitted via a fixup. This also means that we can't pre-emptively downgrade a function to Unwind v1 if one of these offsets is too large, so instead we raise an error (but I've passed through the location information, so the user will know which of their functions is problematic). Patch is 45.91 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/129142.diff 22 Files Affected:
diff --git a/clang/include/clang/Basic/CodeGenOptions.def b/clang/include/clang/Basic/CodeGenOptions.def
index a7f5f1abbb825..399bce726e08f 100644
--- a/clang/include/clang/Basic/CodeGenOptions.def
+++ b/clang/include/clang/Basic/CodeGenOptions.def
@@ -476,6 +476,9 @@ CODEGENOPT(ImportCallOptimization, 1, 0)
/// (BlocksRuntime) on Windows.
CODEGENOPT(StaticClosure, 1, 0)
+/// Enables unwind v2 (epilog) information for x64 Windows.
+CODEGENOPT(WinX64EHUnwindV2, 1, 0)
+
/// FIXME: Make DebugOptions its own top-level .def file.
#include "DebugOptions.def"
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 1cf62ab466134..4061d313970d3 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -7613,6 +7613,10 @@ def import_call_optimization : Flag<["-"], "import-call-optimization">,
"by the Windows kernel to enable import call optimization">,
MarshallingInfoFlag<CodeGenOpts<"ImportCallOptimization">>;
+def epilog_unwind : Flag<["-"], "winx64-eh-unwindv2">,
+ HelpText<"Enable unwind v2 (epilog) information for x64 Windows">,
+ MarshallingInfoFlag<CodeGenOpts<"WinX64EHUnwindV2">>;
+
} // let Visibility = [CC1Option]
//===----------------------------------------------------------------------===//
@@ -8725,6 +8729,8 @@ def _SLASH_M_Group : OptionGroup<"</M group>">, Group<cl_compile_Group>;
def _SLASH_volatile_Group : OptionGroup<"</volatile group>">,
Group<cl_compile_Group>;
+def _SLASH_d2epilogunwind : CLFlag<"d2epilogunwind">,
+ HelpText<"Enable unwind v2 (epilog) information for x64 Windows">;
def _SLASH_EH : CLJoined<"EH">, HelpText<"Set exception handling model">;
def _SLASH_EP : CLFlag<"EP">,
HelpText<"Disable linemarker output and preprocess to stdout">;
diff --git a/clang/lib/CodeGen/CodeGenModule.cpp b/clang/lib/CodeGen/CodeGenModule.cpp
index 7924c32fcf633..1f53a2d055516 100644
--- a/clang/lib/CodeGen/CodeGenModule.cpp
+++ b/clang/lib/CodeGen/CodeGenModule.cpp
@@ -1302,6 +1302,10 @@ void CodeGenModule::Release() {
getModule().addModuleFlag(llvm::Module::Warning, "import-call-optimization",
1);
+ // Enable unwind v2 (epilog).
+ if (CodeGenOpts.WinX64EHUnwindV2)
+ getModule().addModuleFlag(llvm::Module::Warning, "winx64-eh-unwindv2", 1);
+
// Indicate whether this Module was compiled with -fopenmp
if (getLangOpts().OpenMP && !getLangOpts().OpenMPSimd)
getModule().addModuleFlag(llvm::Module::Max, "openmp", LangOpts.OpenMP);
diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp
index 5deafa2ad0f4a..e96a271cb47d9 100644
--- a/clang/lib/Driver/ToolChains/Clang.cpp
+++ b/clang/lib/Driver/ToolChains/Clang.cpp
@@ -8532,6 +8532,9 @@ void Clang::AddClangCLArgs(const ArgList &Args, types::ID InputType,
if (Args.hasArg(options::OPT__SLASH_kernel))
CmdArgs.push_back("-fms-kernel");
+ if (Args.hasArg(options::OPT__SLASH_d2epilogunwind))
+ CmdArgs.push_back("-winx64-eh-unwindv2");
+
for (const Arg *A : Args.filtered(options::OPT__SLASH_guard)) {
StringRef GuardArgs = A->getValue();
// The only valid options are "cf", "cf,nochecks", "cf-", "ehcont" and
diff --git a/clang/test/CodeGen/epilog-unwind.c b/clang/test/CodeGen/epilog-unwind.c
new file mode 100644
index 0000000000000..73bba28f94b29
--- /dev/null
+++ b/clang/test/CodeGen/epilog-unwind.c
@@ -0,0 +1,5 @@
+// RUN: %clang_cc1 -winx64-eh-unwindv2 -emit-llvm %s -o - | FileCheck %s
+
+void f(void) {}
+
+// CHECK: !"winx64-eh-unwindv2", i32 1}
diff --git a/clang/test/Driver/cl-options.c b/clang/test/Driver/cl-options.c
index 9f9ca1bf1a8fd..c0031e9d96d09 100644
--- a/clang/test/Driver/cl-options.c
+++ b/clang/test/Driver/cl-options.c
@@ -817,4 +817,7 @@
// RUN: %clang_cl -vctoolsdir "" /arm64EC /c -target x86_64-pc-windows-msvc -### -- %s 2>&1 | FileCheck --check-prefix=ARM64EC_OVERRIDE %s
// ARM64EC_OVERRIDE: warning: /arm64EC has been overridden by specified target: x86_64-pc-windows-msvc; option ignored
+// RUN: %clang_cl /d2epilogunwind /c -### -- %s 2>&1 | FileCheck %s --check-prefix=EPILOGUNWIND
+// EPILOGUNWIND: -winx64-eh-unwindv2
+
void f(void) { }
diff --git a/llvm/include/llvm/MC/MCStreamer.h b/llvm/include/llvm/MC/MCStreamer.h
index 4fb2158f26e1e..1d6ed8e159e13 100644
--- a/llvm/include/llvm/MC/MCStreamer.h
+++ b/llvm/include/llvm/MC/MCStreamer.h
@@ -1066,6 +1066,8 @@ class MCStreamer {
virtual void emitWinCFIEndProlog(SMLoc Loc = SMLoc());
virtual void emitWinCFIBeginEpilogue(SMLoc Loc = SMLoc());
virtual void emitWinCFIEndEpilogue(SMLoc Loc = SMLoc());
+ virtual void emitWinCFIUnwindV2Start(SMLoc Loc = SMLoc());
+ virtual void emitWinCFIUnwindVersion(uint8_t Version, SMLoc Loc = SMLoc());
virtual void emitWinEHHandler(const MCSymbol *Sym, bool Unwind, bool Except,
SMLoc Loc = SMLoc());
virtual void emitWinEHHandlerData(SMLoc Loc = SMLoc());
diff --git a/llvm/include/llvm/MC/MCWinEH.h b/llvm/include/llvm/MC/MCWinEH.h
index fcce2dcd54837..c603aa896883d 100644
--- a/llvm/include/llvm/MC/MCWinEH.h
+++ b/llvm/include/llvm/MC/MCWinEH.h
@@ -10,6 +10,7 @@
#define LLVM_MC_MCWINEH_H
#include "llvm/ADT/MapVector.h"
+#include "llvm/Support/SMLoc.h"
#include <vector>
namespace llvm {
@@ -42,6 +43,7 @@ struct FrameInfo {
const MCSymbol *FuncletOrFuncEnd = nullptr;
const MCSymbol *ExceptionHandler = nullptr;
const MCSymbol *Function = nullptr;
+ SMLoc FunctionLoc;
const MCSymbol *PrologEnd = nullptr;
const MCSymbol *Symbol = nullptr;
MCSection *TextSection = nullptr;
@@ -52,6 +54,8 @@ struct FrameInfo {
bool HandlesExceptions = false;
bool EmitAttempted = false;
bool Fragment = false;
+ constexpr static uint8_t DefaultVersion = 1;
+ uint8_t Version = DefaultVersion;
int LastFrameInst = -1;
const FrameInfo *ChainedParent = nullptr;
@@ -59,7 +63,9 @@ struct FrameInfo {
struct Epilog {
std::vector<Instruction> Instructions;
unsigned Condition;
- MCSymbol *End;
+ const MCSymbol *End = nullptr;
+ const MCSymbol *UnwindV2Start = nullptr;
+ SMLoc Loc;
};
MapVector<MCSymbol *, Epilog> EpilogMap;
diff --git a/llvm/lib/MC/MCAsmStreamer.cpp b/llvm/lib/MC/MCAsmStreamer.cpp
index 4db405051f7f3..1b74b8d73165b 100644
--- a/llvm/lib/MC/MCAsmStreamer.cpp
+++ b/llvm/lib/MC/MCAsmStreamer.cpp
@@ -399,6 +399,8 @@ class MCAsmStreamer final : public MCStreamer {
void emitWinCFIEndProlog(SMLoc Loc) override;
void emitWinCFIBeginEpilogue(SMLoc Loc) override;
void emitWinCFIEndEpilogue(SMLoc Loc) override;
+ void emitWinCFIUnwindV2Start(SMLoc Loc) override;
+ void emitWinCFIUnwindVersion(uint8_t Version, SMLoc Loc) override;
void emitWinEHHandler(const MCSymbol *Sym, bool Unwind, bool Except,
SMLoc Loc) override;
@@ -2351,6 +2353,20 @@ void MCAsmStreamer::emitWinCFIEndEpilogue(SMLoc Loc) {
EmitEOL();
}
+void MCAsmStreamer::emitWinCFIUnwindV2Start(SMLoc Loc) {
+ MCStreamer::emitWinCFIUnwindV2Start(Loc);
+
+ OS << "\t.seh_unwindv2start";
+ EmitEOL();
+}
+
+void MCAsmStreamer::emitWinCFIUnwindVersion(uint8_t Version, SMLoc Loc) {
+ MCStreamer::emitWinCFIUnwindVersion(Version, Loc);
+
+ OS << "\t.seh_unwindversion " << (unsigned)Version;
+ EmitEOL();
+}
+
void MCAsmStreamer::emitCGProfileEntry(const MCSymbolRefExpr *From,
const MCSymbolRefExpr *To,
uint64_t Count) {
diff --git a/llvm/lib/MC/MCParser/COFFAsmParser.cpp b/llvm/lib/MC/MCParser/COFFAsmParser.cpp
index 4618e5675e47b..4632df9f32540 100644
--- a/llvm/lib/MC/MCParser/COFFAsmParser.cpp
+++ b/llvm/lib/MC/MCParser/COFFAsmParser.cpp
@@ -96,6 +96,10 @@ class COFFAsmParser : public MCAsmParserExtension {
".seh_startepilogue");
addDirectiveHandler<&COFFAsmParser::ParseSEHDirectiveEndEpilog>(
".seh_endepilogue");
+ addDirectiveHandler<&COFFAsmParser::ParseSEHDirectiveUnwindV2Start>(
+ ".seh_unwindv2start");
+ addDirectiveHandler<&COFFAsmParser::ParseSEHDirectiveUnwindVersion>(
+ ".seh_unwindversion");
}
bool parseSectionDirectiveText(StringRef, SMLoc) {
@@ -147,6 +151,8 @@ class COFFAsmParser : public MCAsmParserExtension {
bool parseSEHDirectiveEndProlog(StringRef, SMLoc);
bool ParseSEHDirectiveBeginEpilog(StringRef, SMLoc);
bool ParseSEHDirectiveEndEpilog(StringRef, SMLoc);
+ bool ParseSEHDirectiveUnwindV2Start(StringRef, SMLoc);
+ bool ParseSEHDirectiveUnwindVersion(StringRef, SMLoc);
bool parseAtUnwindOrAtExcept(bool &unwind, bool &except);
bool parseDirectiveSymbolAttribute(StringRef Directive, SMLoc);
@@ -767,6 +773,28 @@ bool COFFAsmParser::ParseSEHDirectiveEndEpilog(StringRef, SMLoc Loc) {
return false;
}
+bool COFFAsmParser::ParseSEHDirectiveUnwindV2Start(StringRef, SMLoc Loc) {
+ Lex();
+ getStreamer().emitWinCFIUnwindV2Start(Loc);
+ return false;
+}
+
+bool COFFAsmParser::ParseSEHDirectiveUnwindVersion(StringRef, SMLoc Loc) {
+ int64_t Version;
+ if (getParser().parseIntToken(Version, "expected unwind version number"))
+ return true;
+
+ if ((Version < 1) || (Version > UINT8_MAX))
+ return Error(Loc, "invalid unwind version");
+
+ if (getLexer().isNot(AsmToken::EndOfStatement))
+ return TokError("unexpected token in directive");
+
+ Lex();
+ getStreamer().emitWinCFIUnwindVersion(Version, Loc);
+ return false;
+}
+
bool COFFAsmParser::parseAtUnwindOrAtExcept(bool &unwind, bool &except) {
StringRef identifier;
if (getLexer().isNot(AsmToken::At) && getLexer().isNot(AsmToken::Percent))
diff --git a/llvm/lib/MC/MCStreamer.cpp b/llvm/lib/MC/MCStreamer.cpp
index dd04e1e30720a..dab98feac1cb1 100644
--- a/llvm/lib/MC/MCStreamer.cpp
+++ b/llvm/lib/MC/MCStreamer.cpp
@@ -500,7 +500,8 @@ MCSymbol *MCStreamer::emitLineTableLabel() {
MCSymbol *MCStreamer::emitCFILabel() {
// Return a dummy non-null value so that label fields appear filled in when
// generating textual assembly.
- return (MCSymbol *)1;
+ static size_t DummyLabelValue = 0;
+ return (MCSymbol *)(++DummyLabelValue);
}
void MCStreamer::emitCFIDefCfa(int64_t Register, int64_t Offset, SMLoc Loc) {
@@ -767,6 +768,7 @@ void MCStreamer::emitWinCFIStartProc(const MCSymbol *Symbol, SMLoc Loc) {
std::make_unique<WinEH::FrameInfo>(Symbol, StartProc));
CurrentWinFrameInfo = WinFrameInfos.back().get();
CurrentWinFrameInfo->TextSection = getCurrentSectionOnly();
+ CurrentWinFrameInfo->FunctionLoc = Loc;
}
void MCStreamer::emitWinCFIEndProc(SMLoc Loc) {
@@ -1026,6 +1028,7 @@ void MCStreamer::emitWinCFIBeginEpilogue(SMLoc Loc) {
InEpilogCFI = true;
CurrentEpilog = emitCFILabel();
+ CurFrame->EpilogMap[CurrentEpilog].Loc = Loc;
}
void MCStreamer::emitWinCFIEndEpilogue(SMLoc Loc) {
@@ -1037,12 +1040,53 @@ void MCStreamer::emitWinCFIEndEpilogue(SMLoc Loc) {
return getContext().reportError(Loc, "Stray .seh_endepilogue in " +
CurFrame->Function->getName());
+ assert(CurrentEpilog);
+ if ((CurFrame->Version >= 2) &&
+ !CurFrame->EpilogMap[CurrentEpilog].UnwindV2Start)
+ return getContext().reportError(Loc, "Missing .seh_unwindv2start in " +
+ CurFrame->Function->getName());
+
InEpilogCFI = false;
MCSymbol *Label = emitCFILabel();
CurFrame->EpilogMap[CurrentEpilog].End = Label;
CurrentEpilog = nullptr;
}
+void MCStreamer::emitWinCFIUnwindV2Start(SMLoc Loc) {
+ WinEH::FrameInfo *CurFrame = EnsureValidWinFrameInfo(Loc);
+ if (!CurFrame)
+ return;
+
+ if (!InEpilogCFI)
+ return getContext().reportError(Loc, "Stray .seh_unwindv2start in " +
+ CurFrame->Function->getName());
+
+ assert(CurrentEpilog);
+ if (CurFrame->EpilogMap[CurrentEpilog].UnwindV2Start)
+ return getContext().reportError(Loc, "Duplicate .seh_unwindv2start in " +
+ CurFrame->Function->getName());
+
+ MCSymbol *Label = emitCFILabel();
+ CurFrame->EpilogMap[CurrentEpilog].UnwindV2Start = Label;
+}
+
+void MCStreamer::emitWinCFIUnwindVersion(uint8_t Version, SMLoc Loc) {
+ WinEH::FrameInfo *CurFrame = EnsureValidWinFrameInfo(Loc);
+ if (!CurFrame)
+ return;
+
+ if (CurFrame->Version != WinEH::FrameInfo::DefaultVersion)
+ return getContext().reportError(Loc, "Duplicate .seh_unwindversion in " +
+ CurFrame->Function->getName());
+
+ if (Version != 2)
+ return getContext().reportError(
+ Loc, "Unsupported version specified in .seh_unwindversion in " +
+ CurFrame->Function->getName());
+
+ CurFrame->Version = Version;
+}
+
void MCStreamer::emitCOFFSafeSEH(MCSymbol const *Symbol) {}
void MCStreamer::emitCOFFSymbolIndex(MCSymbol const *Symbol) {}
diff --git a/llvm/lib/MC/MCWin64EH.cpp b/llvm/lib/MC/MCWin64EH.cpp
index bd5cf354659b6..253af0c048e5a 100644
--- a/llvm/lib/MC/MCWin64EH.cpp
+++ b/llvm/lib/MC/MCWin64EH.cpp
@@ -8,14 +8,61 @@
#include "llvm/MC/MCWin64EH.h"
#include "llvm/ADT/Twine.h"
+#include "llvm/MC/MCAssembler.h"
#include "llvm/MC/MCContext.h"
#include "llvm/MC/MCExpr.h"
#include "llvm/MC/MCObjectStreamer.h"
#include "llvm/MC/MCStreamer.h"
#include "llvm/MC/MCSymbol.h"
+#include "llvm/MC/MCValue.h"
#include "llvm/Support/Win64EH.h"
+
namespace llvm {
class MCSection;
+
+/// MCExpr that represents the epilog unwind code in an unwind table.
+class MCUnwindV2EpilogTargetExpr final : public MCTargetExpr {
+ const MCSymbol *FunctionEnd;
+ const MCSymbol *UnwindV2Start;
+ const MCSymbol *EpilogEnd;
+ uint8_t EpilogSize;
+ SMLoc Loc;
+
+ MCUnwindV2EpilogTargetExpr(const WinEH::FrameInfo &FrameInfo,
+ const WinEH::FrameInfo::Epilog &Epilog,
+ uint8_t EpilogSize_)
+ : FunctionEnd(FrameInfo.FuncletOrFuncEnd),
+ UnwindV2Start(Epilog.UnwindV2Start), EpilogEnd(Epilog.End),
+ EpilogSize(EpilogSize_), Loc(Epilog.Loc) {}
+
+public:
+ static MCUnwindV2EpilogTargetExpr *
+ create(const WinEH::FrameInfo &FrameInfo,
+ const WinEH::FrameInfo::Epilog &Epilog, uint8_t EpilogSize_,
+ MCContext &Ctx) {
+ return new (Ctx) MCUnwindV2EpilogTargetExpr(FrameInfo, Epilog, EpilogSize_);
+ }
+
+ void printImpl(raw_ostream &OS, const MCAsmInfo *MAI) const override {
+ OS << ":epilog:";
+ UnwindV2Start->print(OS, MAI);
+ }
+
+ bool evaluateAsRelocatableImpl(MCValue &Res, const MCAssembler *Asm,
+ const MCFixup *Fixup) const override;
+
+ void visitUsedExpr(MCStreamer &Streamer) const override {
+ // Contains no sub-expressions.
+ }
+
+ MCFragment *findAssociatedFragment() const override {
+ return UnwindV2Start->getFragment();
+ }
+
+ void fixELFSymbolsInTLSFixups(MCAssembler &) const override {
+ llvm_unreachable("Not supported for ELF");
+ }
+};
}
using namespace llvm;
@@ -163,20 +210,91 @@ static void EmitRuntimeFunction(MCStreamer &streamer,
context), 4);
}
+static std::optional<int64_t>
+GetOptionalAbsDifference(const MCAssembler &Assembler, const MCSymbol *LHS,
+ const MCSymbol *RHS) {
+ MCContext &Context = Assembler.getContext();
+ const MCExpr *Diff =
+ MCBinaryExpr::createSub(MCSymbolRefExpr::create(LHS, Context),
+ MCSymbolRefExpr::create(RHS, Context), Context);
+ // It should normally be possible to calculate the length of a function
+ // at this point, but it might not be possible in the presence of certain
+ // unusual constructs, like an inline asm with an alignment directive.
+ int64_t value;
+ if (!Diff->evaluateAsAbsolute(value, Assembler))
+ return std::nullopt;
+ return value;
+}
+
static void EmitUnwindInfo(MCStreamer &streamer, WinEH::FrameInfo *info) {
// If this UNWIND_INFO already has a symbol, it's already been emitted.
if (info->Symbol)
return;
MCContext &context = streamer.getContext();
+ MCObjectStreamer *OS = (MCObjectStreamer *)(&streamer);
MCSymbol *Label = context.createTempSymbol();
streamer.emitValueToAlignment(Align(4));
streamer.emitLabel(Label);
info->Symbol = Label;
- // Upper 3 bits are the version number (currently 1).
- uint8_t flags = 0x01;
+ uint8_t numCodes = CountOfUnwindCodes(info->Instructions);
+ bool LastEpilogIsAtEnd = false;
+ bool AddPaddingEpilogCode = false;
+ uint8_t EpilogSize = 0;
+ bool EnableUnwindV2 = (info->Version >= 2) && !info->EpilogMap.empty();
+ if (EnableUnwindV2) {
+ auto &LastEpilog = info->EpilogMap.rbegin()->second;
+
+ // Calculate the size of the epilogs. Note that we +1 to the size so that
+ // the terminator instruction is also included in the epilog (the Windows
+ // unwinder does a simple range check versus the current instruction pointer
+ // so, although there are terminators that are large than 1 byte, the
+ // starting address of the terminator instruction will always be considered
+ // inside the epilog).
+ auto MaybeSize = GetOptionalAbsDifference(
+ OS->getAssembler(), LastEpilog.End, LastEpilog.UnwindV2Start);
+ if (!MaybeSize) {
+ context.reportError(LastEpilog.Loc,
+ "Failed to evaluate epilog size for Unwind v2");
+ return;
+ }
+ assert(*MaybeSize >= 0);
+ if (*MaybeSize >= (int64_t)UINT8_MAX) {
+ context.reportError(LastEpilog.Loc,
+ "Epilog size is too large for Unwind v2");
+ return;
+ }
+ EpilogSize = *MaybeSize + 1;
+
+ // If the last epilog is at the end of the function, we can use a special
+ // encoding for it. Because of our +1 trick for the size, this will only
+ // work where that final terminator instruction is 1 byte long.
+ auto LastEpilogToFuncEnd = GetOptionalAbsDifference(
+ OS->getAssembler(), info->FuncletOrFuncEnd, LastEpilog.UnwindV2Start);
+ LastEpilogIsAtEnd = (LastEpilogToFuncEnd == EpilogSize);
+
+ // If we have an odd number of epilog codes, we need to add a padding code.
+ size_t numEpilogCodes =
+ info->EpilogMap.size() + (LastEpilogIsAtEnd ? 0 : 1);
+ if ((numEpilogCodes % 2) != 0) {
+ AddPaddingEpilogCode = true;
+ numEpilogCodes++;
+ }
+
+ // Too many epilogs to handle.
+ if ((size_t)numCodes + numEpilogCodes > UINT8_MAX) {
+ context.reportError(info->FunctionLoc,
+ "Too many unwind codes with Unwind v2 enabled");
+ return;
+ }
+
+ numCodes += numEpilogCodes;
+ }
+
+ // Upper 3 bits are the version number.
+ uint8_t flags = info->Version;
if (info->ChainedParent)
flags |= Win64EH::UNW_ChainInfo << 3;
else {
@@ -192,7 +310,6 @@ static void EmitUnwindInfo(MCStreamer &streamer, WinEH::FrameInfo *info) {
else
streamer.emitInt8(0);
- uint8_t numCodes = CountOfUnwindCodes(info->Instructions);
streamer.emitInt8(numCodes);
uint8_t frame = 0;
@@ -203,6 +320,35 @@ static void EmitUnwindInfo(MCStreamer &streamer, WinEH::FrameInfo *info) {
}
streamer.emitInt8(frame);
+ // Emit the epilog instructions.
+ if (EnableUnwindV2) {
+ MCDataFragment *DF = OS->getOrCreateDataFragment();
+
+ bool IsLast = true;
+ for (const auto &[_, Epilog] : llvm::reverse(info->EpilogMap)) {
+ if (IsLast) {
+ IsLast = false;
+ uint8_t Flags = LastEpilogIsAtEnd ? 0x01 : 0;
+ streamer.emitInt8(EpilogSize);
+ streamer.emitInt8((Flags << 4) | Win64EH::UOP_Epilog);
+
+ if (LastEpilogIsAtEnd)
+ continue;
+ }
+
+ // Each epilog is emitted as a fixup, since we can't measure the distance
+ // between the start of the epilog and the end of the function until
+ // layout has been completed.
+ auto *MCE = MCUnwindV2EpilogTargetExpr::create(*info, Epilog, EpilogSize,
+ context);
+ MCFixup Fixup = MCFixup::create(DF->getContents().size(), MCE, FK_Data_2);
+ DF->getFixups().push_back(Fixup);
+ DF->appendContents(2, 0);
+ }
+ }
+ if (AddPaddingEpilogCode)
+ streamer.emitInt16(Win64EH::UOP_Epilog << 8);
+
// Emit unwind instructions (in reverse order).
uint8_t numInst = info->Instructions.size();
for (uint8_t c = 0; c < numInst; ++c) {
@@ -234,6 +380,39 @@ static void EmitUnwindInfo(MCStr...
[truncated]
|
|
@llvm/pr-subscribers-clang Author: Daniel Paoliello (dpaoliello) ChangesAdds support for emitting Windows x64 Unwind V2 information, includes support Unwind v2 adds information about the epilogs in functions such that the unwinder can unwind even in the middle of an epilog, without having to disassembly the function to see what has or has not been cleaned up. Unwind v2 requires that all epilogs are in "canonical" form:
This change adds a pass to validate epilogs in modules that have Unwind v2 enabled and, if they pass, emits new pseudo instructions to MC that 1) note that the function is using unwind v2 and 2) mark the start of the epilog (this is either the first Note that the unwind v2 table only marks the size of the epilog in the "header" unwind code, but it's possible for epilogs to use different terminator instructions thus they are not all the same size. As a work around for this, MC will assume that all terminator instructions are 1-byte long - this still works correctly with the Windows unwinder as it is only using the size to do a range check to see if a thread is in an epilog or not, and since the instruction pointer will never be in the middle of an instruction and the terminator is always at the end of an epilog the range check will function correctly. This does mean, however, that the "at end" optimization (where an epilog unwind code can be elided if the last epilog is at the end of the function) can only be used if the terminator is 1-byte long. One other complication with the implementation is that the unwind table for a function is emitted during streaming, however we can't calculate the distance between an epilog and the end of the function at that time as layout hasn't been completed yet (thus some instructions may be relaxed). To work around this, epilog unwind codes are emitted via a fixup. This also means that we can't pre-emptively downgrade a function to Unwind v1 if one of these offsets is too large, so instead we raise an error (but I've passed through the location information, so the user will know which of their functions is problematic). Patch is 45.91 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/129142.diff 22 Files Affected:
diff --git a/clang/include/clang/Basic/CodeGenOptions.def b/clang/include/clang/Basic/CodeGenOptions.def
index a7f5f1abbb825..399bce726e08f 100644
--- a/clang/include/clang/Basic/CodeGenOptions.def
+++ b/clang/include/clang/Basic/CodeGenOptions.def
@@ -476,6 +476,9 @@ CODEGENOPT(ImportCallOptimization, 1, 0)
/// (BlocksRuntime) on Windows.
CODEGENOPT(StaticClosure, 1, 0)
+/// Enables unwind v2 (epilog) information for x64 Windows.
+CODEGENOPT(WinX64EHUnwindV2, 1, 0)
+
/// FIXME: Make DebugOptions its own top-level .def file.
#include "DebugOptions.def"
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 1cf62ab466134..4061d313970d3 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -7613,6 +7613,10 @@ def import_call_optimization : Flag<["-"], "import-call-optimization">,
"by the Windows kernel to enable import call optimization">,
MarshallingInfoFlag<CodeGenOpts<"ImportCallOptimization">>;
+def epilog_unwind : Flag<["-"], "winx64-eh-unwindv2">,
+ HelpText<"Enable unwind v2 (epilog) information for x64 Windows">,
+ MarshallingInfoFlag<CodeGenOpts<"WinX64EHUnwindV2">>;
+
} // let Visibility = [CC1Option]
//===----------------------------------------------------------------------===//
@@ -8725,6 +8729,8 @@ def _SLASH_M_Group : OptionGroup<"</M group>">, Group<cl_compile_Group>;
def _SLASH_volatile_Group : OptionGroup<"</volatile group>">,
Group<cl_compile_Group>;
+def _SLASH_d2epilogunwind : CLFlag<"d2epilogunwind">,
+ HelpText<"Enable unwind v2 (epilog) information for x64 Windows">;
def _SLASH_EH : CLJoined<"EH">, HelpText<"Set exception handling model">;
def _SLASH_EP : CLFlag<"EP">,
HelpText<"Disable linemarker output and preprocess to stdout">;
diff --git a/clang/lib/CodeGen/CodeGenModule.cpp b/clang/lib/CodeGen/CodeGenModule.cpp
index 7924c32fcf633..1f53a2d055516 100644
--- a/clang/lib/CodeGen/CodeGenModule.cpp
+++ b/clang/lib/CodeGen/CodeGenModule.cpp
@@ -1302,6 +1302,10 @@ void CodeGenModule::Release() {
getModule().addModuleFlag(llvm::Module::Warning, "import-call-optimization",
1);
+ // Enable unwind v2 (epilog).
+ if (CodeGenOpts.WinX64EHUnwindV2)
+ getModule().addModuleFlag(llvm::Module::Warning, "winx64-eh-unwindv2", 1);
+
// Indicate whether this Module was compiled with -fopenmp
if (getLangOpts().OpenMP && !getLangOpts().OpenMPSimd)
getModule().addModuleFlag(llvm::Module::Max, "openmp", LangOpts.OpenMP);
diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp
index 5deafa2ad0f4a..e96a271cb47d9 100644
--- a/clang/lib/Driver/ToolChains/Clang.cpp
+++ b/clang/lib/Driver/ToolChains/Clang.cpp
@@ -8532,6 +8532,9 @@ void Clang::AddClangCLArgs(const ArgList &Args, types::ID InputType,
if (Args.hasArg(options::OPT__SLASH_kernel))
CmdArgs.push_back("-fms-kernel");
+ if (Args.hasArg(options::OPT__SLASH_d2epilogunwind))
+ CmdArgs.push_back("-winx64-eh-unwindv2");
+
for (const Arg *A : Args.filtered(options::OPT__SLASH_guard)) {
StringRef GuardArgs = A->getValue();
// The only valid options are "cf", "cf,nochecks", "cf-", "ehcont" and
diff --git a/clang/test/CodeGen/epilog-unwind.c b/clang/test/CodeGen/epilog-unwind.c
new file mode 100644
index 0000000000000..73bba28f94b29
--- /dev/null
+++ b/clang/test/CodeGen/epilog-unwind.c
@@ -0,0 +1,5 @@
+// RUN: %clang_cc1 -winx64-eh-unwindv2 -emit-llvm %s -o - | FileCheck %s
+
+void f(void) {}
+
+// CHECK: !"winx64-eh-unwindv2", i32 1}
diff --git a/clang/test/Driver/cl-options.c b/clang/test/Driver/cl-options.c
index 9f9ca1bf1a8fd..c0031e9d96d09 100644
--- a/clang/test/Driver/cl-options.c
+++ b/clang/test/Driver/cl-options.c
@@ -817,4 +817,7 @@
// RUN: %clang_cl -vctoolsdir "" /arm64EC /c -target x86_64-pc-windows-msvc -### -- %s 2>&1 | FileCheck --check-prefix=ARM64EC_OVERRIDE %s
// ARM64EC_OVERRIDE: warning: /arm64EC has been overridden by specified target: x86_64-pc-windows-msvc; option ignored
+// RUN: %clang_cl /d2epilogunwind /c -### -- %s 2>&1 | FileCheck %s --check-prefix=EPILOGUNWIND
+// EPILOGUNWIND: -winx64-eh-unwindv2
+
void f(void) { }
diff --git a/llvm/include/llvm/MC/MCStreamer.h b/llvm/include/llvm/MC/MCStreamer.h
index 4fb2158f26e1e..1d6ed8e159e13 100644
--- a/llvm/include/llvm/MC/MCStreamer.h
+++ b/llvm/include/llvm/MC/MCStreamer.h
@@ -1066,6 +1066,8 @@ class MCStreamer {
virtual void emitWinCFIEndProlog(SMLoc Loc = SMLoc());
virtual void emitWinCFIBeginEpilogue(SMLoc Loc = SMLoc());
virtual void emitWinCFIEndEpilogue(SMLoc Loc = SMLoc());
+ virtual void emitWinCFIUnwindV2Start(SMLoc Loc = SMLoc());
+ virtual void emitWinCFIUnwindVersion(uint8_t Version, SMLoc Loc = SMLoc());
virtual void emitWinEHHandler(const MCSymbol *Sym, bool Unwind, bool Except,
SMLoc Loc = SMLoc());
virtual void emitWinEHHandlerData(SMLoc Loc = SMLoc());
diff --git a/llvm/include/llvm/MC/MCWinEH.h b/llvm/include/llvm/MC/MCWinEH.h
index fcce2dcd54837..c603aa896883d 100644
--- a/llvm/include/llvm/MC/MCWinEH.h
+++ b/llvm/include/llvm/MC/MCWinEH.h
@@ -10,6 +10,7 @@
#define LLVM_MC_MCWINEH_H
#include "llvm/ADT/MapVector.h"
+#include "llvm/Support/SMLoc.h"
#include <vector>
namespace llvm {
@@ -42,6 +43,7 @@ struct FrameInfo {
const MCSymbol *FuncletOrFuncEnd = nullptr;
const MCSymbol *ExceptionHandler = nullptr;
const MCSymbol *Function = nullptr;
+ SMLoc FunctionLoc;
const MCSymbol *PrologEnd = nullptr;
const MCSymbol *Symbol = nullptr;
MCSection *TextSection = nullptr;
@@ -52,6 +54,8 @@ struct FrameInfo {
bool HandlesExceptions = false;
bool EmitAttempted = false;
bool Fragment = false;
+ constexpr static uint8_t DefaultVersion = 1;
+ uint8_t Version = DefaultVersion;
int LastFrameInst = -1;
const FrameInfo *ChainedParent = nullptr;
@@ -59,7 +63,9 @@ struct FrameInfo {
struct Epilog {
std::vector<Instruction> Instructions;
unsigned Condition;
- MCSymbol *End;
+ const MCSymbol *End = nullptr;
+ const MCSymbol *UnwindV2Start = nullptr;
+ SMLoc Loc;
};
MapVector<MCSymbol *, Epilog> EpilogMap;
diff --git a/llvm/lib/MC/MCAsmStreamer.cpp b/llvm/lib/MC/MCAsmStreamer.cpp
index 4db405051f7f3..1b74b8d73165b 100644
--- a/llvm/lib/MC/MCAsmStreamer.cpp
+++ b/llvm/lib/MC/MCAsmStreamer.cpp
@@ -399,6 +399,8 @@ class MCAsmStreamer final : public MCStreamer {
void emitWinCFIEndProlog(SMLoc Loc) override;
void emitWinCFIBeginEpilogue(SMLoc Loc) override;
void emitWinCFIEndEpilogue(SMLoc Loc) override;
+ void emitWinCFIUnwindV2Start(SMLoc Loc) override;
+ void emitWinCFIUnwindVersion(uint8_t Version, SMLoc Loc) override;
void emitWinEHHandler(const MCSymbol *Sym, bool Unwind, bool Except,
SMLoc Loc) override;
@@ -2351,6 +2353,20 @@ void MCAsmStreamer::emitWinCFIEndEpilogue(SMLoc Loc) {
EmitEOL();
}
+void MCAsmStreamer::emitWinCFIUnwindV2Start(SMLoc Loc) {
+ MCStreamer::emitWinCFIUnwindV2Start(Loc);
+
+ OS << "\t.seh_unwindv2start";
+ EmitEOL();
+}
+
+void MCAsmStreamer::emitWinCFIUnwindVersion(uint8_t Version, SMLoc Loc) {
+ MCStreamer::emitWinCFIUnwindVersion(Version, Loc);
+
+ OS << "\t.seh_unwindversion " << (unsigned)Version;
+ EmitEOL();
+}
+
void MCAsmStreamer::emitCGProfileEntry(const MCSymbolRefExpr *From,
const MCSymbolRefExpr *To,
uint64_t Count) {
diff --git a/llvm/lib/MC/MCParser/COFFAsmParser.cpp b/llvm/lib/MC/MCParser/COFFAsmParser.cpp
index 4618e5675e47b..4632df9f32540 100644
--- a/llvm/lib/MC/MCParser/COFFAsmParser.cpp
+++ b/llvm/lib/MC/MCParser/COFFAsmParser.cpp
@@ -96,6 +96,10 @@ class COFFAsmParser : public MCAsmParserExtension {
".seh_startepilogue");
addDirectiveHandler<&COFFAsmParser::ParseSEHDirectiveEndEpilog>(
".seh_endepilogue");
+ addDirectiveHandler<&COFFAsmParser::ParseSEHDirectiveUnwindV2Start>(
+ ".seh_unwindv2start");
+ addDirectiveHandler<&COFFAsmParser::ParseSEHDirectiveUnwindVersion>(
+ ".seh_unwindversion");
}
bool parseSectionDirectiveText(StringRef, SMLoc) {
@@ -147,6 +151,8 @@ class COFFAsmParser : public MCAsmParserExtension {
bool parseSEHDirectiveEndProlog(StringRef, SMLoc);
bool ParseSEHDirectiveBeginEpilog(StringRef, SMLoc);
bool ParseSEHDirectiveEndEpilog(StringRef, SMLoc);
+ bool ParseSEHDirectiveUnwindV2Start(StringRef, SMLoc);
+ bool ParseSEHDirectiveUnwindVersion(StringRef, SMLoc);
bool parseAtUnwindOrAtExcept(bool &unwind, bool &except);
bool parseDirectiveSymbolAttribute(StringRef Directive, SMLoc);
@@ -767,6 +773,28 @@ bool COFFAsmParser::ParseSEHDirectiveEndEpilog(StringRef, SMLoc Loc) {
return false;
}
+bool COFFAsmParser::ParseSEHDirectiveUnwindV2Start(StringRef, SMLoc Loc) {
+ Lex();
+ getStreamer().emitWinCFIUnwindV2Start(Loc);
+ return false;
+}
+
+bool COFFAsmParser::ParseSEHDirectiveUnwindVersion(StringRef, SMLoc Loc) {
+ int64_t Version;
+ if (getParser().parseIntToken(Version, "expected unwind version number"))
+ return true;
+
+ if ((Version < 1) || (Version > UINT8_MAX))
+ return Error(Loc, "invalid unwind version");
+
+ if (getLexer().isNot(AsmToken::EndOfStatement))
+ return TokError("unexpected token in directive");
+
+ Lex();
+ getStreamer().emitWinCFIUnwindVersion(Version, Loc);
+ return false;
+}
+
bool COFFAsmParser::parseAtUnwindOrAtExcept(bool &unwind, bool &except) {
StringRef identifier;
if (getLexer().isNot(AsmToken::At) && getLexer().isNot(AsmToken::Percent))
diff --git a/llvm/lib/MC/MCStreamer.cpp b/llvm/lib/MC/MCStreamer.cpp
index dd04e1e30720a..dab98feac1cb1 100644
--- a/llvm/lib/MC/MCStreamer.cpp
+++ b/llvm/lib/MC/MCStreamer.cpp
@@ -500,7 +500,8 @@ MCSymbol *MCStreamer::emitLineTableLabel() {
MCSymbol *MCStreamer::emitCFILabel() {
// Return a dummy non-null value so that label fields appear filled in when
// generating textual assembly.
- return (MCSymbol *)1;
+ static size_t DummyLabelValue = 0;
+ return (MCSymbol *)(++DummyLabelValue);
}
void MCStreamer::emitCFIDefCfa(int64_t Register, int64_t Offset, SMLoc Loc) {
@@ -767,6 +768,7 @@ void MCStreamer::emitWinCFIStartProc(const MCSymbol *Symbol, SMLoc Loc) {
std::make_unique<WinEH::FrameInfo>(Symbol, StartProc));
CurrentWinFrameInfo = WinFrameInfos.back().get();
CurrentWinFrameInfo->TextSection = getCurrentSectionOnly();
+ CurrentWinFrameInfo->FunctionLoc = Loc;
}
void MCStreamer::emitWinCFIEndProc(SMLoc Loc) {
@@ -1026,6 +1028,7 @@ void MCStreamer::emitWinCFIBeginEpilogue(SMLoc Loc) {
InEpilogCFI = true;
CurrentEpilog = emitCFILabel();
+ CurFrame->EpilogMap[CurrentEpilog].Loc = Loc;
}
void MCStreamer::emitWinCFIEndEpilogue(SMLoc Loc) {
@@ -1037,12 +1040,53 @@ void MCStreamer::emitWinCFIEndEpilogue(SMLoc Loc) {
return getContext().reportError(Loc, "Stray .seh_endepilogue in " +
CurFrame->Function->getName());
+ assert(CurrentEpilog);
+ if ((CurFrame->Version >= 2) &&
+ !CurFrame->EpilogMap[CurrentEpilog].UnwindV2Start)
+ return getContext().reportError(Loc, "Missing .seh_unwindv2start in " +
+ CurFrame->Function->getName());
+
InEpilogCFI = false;
MCSymbol *Label = emitCFILabel();
CurFrame->EpilogMap[CurrentEpilog].End = Label;
CurrentEpilog = nullptr;
}
+void MCStreamer::emitWinCFIUnwindV2Start(SMLoc Loc) {
+ WinEH::FrameInfo *CurFrame = EnsureValidWinFrameInfo(Loc);
+ if (!CurFrame)
+ return;
+
+ if (!InEpilogCFI)
+ return getContext().reportError(Loc, "Stray .seh_unwindv2start in " +
+ CurFrame->Function->getName());
+
+ assert(CurrentEpilog);
+ if (CurFrame->EpilogMap[CurrentEpilog].UnwindV2Start)
+ return getContext().reportError(Loc, "Duplicate .seh_unwindv2start in " +
+ CurFrame->Function->getName());
+
+ MCSymbol *Label = emitCFILabel();
+ CurFrame->EpilogMap[CurrentEpilog].UnwindV2Start = Label;
+}
+
+void MCStreamer::emitWinCFIUnwindVersion(uint8_t Version, SMLoc Loc) {
+ WinEH::FrameInfo *CurFrame = EnsureValidWinFrameInfo(Loc);
+ if (!CurFrame)
+ return;
+
+ if (CurFrame->Version != WinEH::FrameInfo::DefaultVersion)
+ return getContext().reportError(Loc, "Duplicate .seh_unwindversion in " +
+ CurFrame->Function->getName());
+
+ if (Version != 2)
+ return getContext().reportError(
+ Loc, "Unsupported version specified in .seh_unwindversion in " +
+ CurFrame->Function->getName());
+
+ CurFrame->Version = Version;
+}
+
void MCStreamer::emitCOFFSafeSEH(MCSymbol const *Symbol) {}
void MCStreamer::emitCOFFSymbolIndex(MCSymbol const *Symbol) {}
diff --git a/llvm/lib/MC/MCWin64EH.cpp b/llvm/lib/MC/MCWin64EH.cpp
index bd5cf354659b6..253af0c048e5a 100644
--- a/llvm/lib/MC/MCWin64EH.cpp
+++ b/llvm/lib/MC/MCWin64EH.cpp
@@ -8,14 +8,61 @@
#include "llvm/MC/MCWin64EH.h"
#include "llvm/ADT/Twine.h"
+#include "llvm/MC/MCAssembler.h"
#include "llvm/MC/MCContext.h"
#include "llvm/MC/MCExpr.h"
#include "llvm/MC/MCObjectStreamer.h"
#include "llvm/MC/MCStreamer.h"
#include "llvm/MC/MCSymbol.h"
+#include "llvm/MC/MCValue.h"
#include "llvm/Support/Win64EH.h"
+
namespace llvm {
class MCSection;
+
+/// MCExpr that represents the epilog unwind code in an unwind table.
+class MCUnwindV2EpilogTargetExpr final : public MCTargetExpr {
+ const MCSymbol *FunctionEnd;
+ const MCSymbol *UnwindV2Start;
+ const MCSymbol *EpilogEnd;
+ uint8_t EpilogSize;
+ SMLoc Loc;
+
+ MCUnwindV2EpilogTargetExpr(const WinEH::FrameInfo &FrameInfo,
+ const WinEH::FrameInfo::Epilog &Epilog,
+ uint8_t EpilogSize_)
+ : FunctionEnd(FrameInfo.FuncletOrFuncEnd),
+ UnwindV2Start(Epilog.UnwindV2Start), EpilogEnd(Epilog.End),
+ EpilogSize(EpilogSize_), Loc(Epilog.Loc) {}
+
+public:
+ static MCUnwindV2EpilogTargetExpr *
+ create(const WinEH::FrameInfo &FrameInfo,
+ const WinEH::FrameInfo::Epilog &Epilog, uint8_t EpilogSize_,
+ MCContext &Ctx) {
+ return new (Ctx) MCUnwindV2EpilogTargetExpr(FrameInfo, Epilog, EpilogSize_);
+ }
+
+ void printImpl(raw_ostream &OS, const MCAsmInfo *MAI) const override {
+ OS << ":epilog:";
+ UnwindV2Start->print(OS, MAI);
+ }
+
+ bool evaluateAsRelocatableImpl(MCValue &Res, const MCAssembler *Asm,
+ const MCFixup *Fixup) const override;
+
+ void visitUsedExpr(MCStreamer &Streamer) const override {
+ // Contains no sub-expressions.
+ }
+
+ MCFragment *findAssociatedFragment() const override {
+ return UnwindV2Start->getFragment();
+ }
+
+ void fixELFSymbolsInTLSFixups(MCAssembler &) const override {
+ llvm_unreachable("Not supported for ELF");
+ }
+};
}
using namespace llvm;
@@ -163,20 +210,91 @@ static void EmitRuntimeFunction(MCStreamer &streamer,
context), 4);
}
+static std::optional<int64_t>
+GetOptionalAbsDifference(const MCAssembler &Assembler, const MCSymbol *LHS,
+ const MCSymbol *RHS) {
+ MCContext &Context = Assembler.getContext();
+ const MCExpr *Diff =
+ MCBinaryExpr::createSub(MCSymbolRefExpr::create(LHS, Context),
+ MCSymbolRefExpr::create(RHS, Context), Context);
+ // It should normally be possible to calculate the length of a function
+ // at this point, but it might not be possible in the presence of certain
+ // unusual constructs, like an inline asm with an alignment directive.
+ int64_t value;
+ if (!Diff->evaluateAsAbsolute(value, Assembler))
+ return std::nullopt;
+ return value;
+}
+
static void EmitUnwindInfo(MCStreamer &streamer, WinEH::FrameInfo *info) {
// If this UNWIND_INFO already has a symbol, it's already been emitted.
if (info->Symbol)
return;
MCContext &context = streamer.getContext();
+ MCObjectStreamer *OS = (MCObjectStreamer *)(&streamer);
MCSymbol *Label = context.createTempSymbol();
streamer.emitValueToAlignment(Align(4));
streamer.emitLabel(Label);
info->Symbol = Label;
- // Upper 3 bits are the version number (currently 1).
- uint8_t flags = 0x01;
+ uint8_t numCodes = CountOfUnwindCodes(info->Instructions);
+ bool LastEpilogIsAtEnd = false;
+ bool AddPaddingEpilogCode = false;
+ uint8_t EpilogSize = 0;
+ bool EnableUnwindV2 = (info->Version >= 2) && !info->EpilogMap.empty();
+ if (EnableUnwindV2) {
+ auto &LastEpilog = info->EpilogMap.rbegin()->second;
+
+ // Calculate the size of the epilogs. Note that we +1 to the size so that
+ // the terminator instruction is also included in the epilog (the Windows
+ // unwinder does a simple range check versus the current instruction pointer
+ // so, although there are terminators that are large than 1 byte, the
+ // starting address of the terminator instruction will always be considered
+ // inside the epilog).
+ auto MaybeSize = GetOptionalAbsDifference(
+ OS->getAssembler(), LastEpilog.End, LastEpilog.UnwindV2Start);
+ if (!MaybeSize) {
+ context.reportError(LastEpilog.Loc,
+ "Failed to evaluate epilog size for Unwind v2");
+ return;
+ }
+ assert(*MaybeSize >= 0);
+ if (*MaybeSize >= (int64_t)UINT8_MAX) {
+ context.reportError(LastEpilog.Loc,
+ "Epilog size is too large for Unwind v2");
+ return;
+ }
+ EpilogSize = *MaybeSize + 1;
+
+ // If the last epilog is at the end of the function, we can use a special
+ // encoding for it. Because of our +1 trick for the size, this will only
+ // work where that final terminator instruction is 1 byte long.
+ auto LastEpilogToFuncEnd = GetOptionalAbsDifference(
+ OS->getAssembler(), info->FuncletOrFuncEnd, LastEpilog.UnwindV2Start);
+ LastEpilogIsAtEnd = (LastEpilogToFuncEnd == EpilogSize);
+
+ // If we have an odd number of epilog codes, we need to add a padding code.
+ size_t numEpilogCodes =
+ info->EpilogMap.size() + (LastEpilogIsAtEnd ? 0 : 1);
+ if ((numEpilogCodes % 2) != 0) {
+ AddPaddingEpilogCode = true;
+ numEpilogCodes++;
+ }
+
+ // Too many epilogs to handle.
+ if ((size_t)numCodes + numEpilogCodes > UINT8_MAX) {
+ context.reportError(info->FunctionLoc,
+ "Too many unwind codes with Unwind v2 enabled");
+ return;
+ }
+
+ numCodes += numEpilogCodes;
+ }
+
+ // Upper 3 bits are the version number.
+ uint8_t flags = info->Version;
if (info->ChainedParent)
flags |= Win64EH::UNW_ChainInfo << 3;
else {
@@ -192,7 +310,6 @@ static void EmitUnwindInfo(MCStreamer &streamer, WinEH::FrameInfo *info) {
else
streamer.emitInt8(0);
- uint8_t numCodes = CountOfUnwindCodes(info->Instructions);
streamer.emitInt8(numCodes);
uint8_t frame = 0;
@@ -203,6 +320,35 @@ static void EmitUnwindInfo(MCStreamer &streamer, WinEH::FrameInfo *info) {
}
streamer.emitInt8(frame);
+ // Emit the epilog instructions.
+ if (EnableUnwindV2) {
+ MCDataFragment *DF = OS->getOrCreateDataFragment();
+
+ bool IsLast = true;
+ for (const auto &[_, Epilog] : llvm::reverse(info->EpilogMap)) {
+ if (IsLast) {
+ IsLast = false;
+ uint8_t Flags = LastEpilogIsAtEnd ? 0x01 : 0;
+ streamer.emitInt8(EpilogSize);
+ streamer.emitInt8((Flags << 4) | Win64EH::UOP_Epilog);
+
+ if (LastEpilogIsAtEnd)
+ continue;
+ }
+
+ // Each epilog is emitted as a fixup, since we can't measure the distance
+ // between the start of the epilog and the end of the function until
+ // layout has been completed.
+ auto *MCE = MCUnwindV2EpilogTargetExpr::create(*info, Epilog, EpilogSize,
+ context);
+ MCFixup Fixup = MCFixup::create(DF->getContents().size(), MCE, FK_Data_2);
+ DF->getFixups().push_back(Fixup);
+ DF->appendContents(2, 0);
+ }
+ }
+ if (AddPaddingEpilogCode)
+ streamer.emitInt16(Win64EH::UOP_Epilog << 8);
+
// Emit unwind instructions (in reverse order).
uint8_t numInst = info->Instructions.size();
for (uint8_t c = 0; c < numInst; ++c) {
@@ -234,6 +380,39 @@ static void EmitUnwindInfo(MCStr...
[truncated]
|
There was a problem hiding this comment.
Any further thoughts on splitting unwind info, as we discussed in #110338?
Have you tried to figure out how frequently LLVM generates non-compliant epilogues? Are you planning changes to prologue/epilogue emission to avoid those cases?
llvm/lib/MC/MCStreamer.cpp
Outdated
| @@ -500,7 +500,8 @@ MCSymbol *MCStreamer::emitLineTableLabel() { | |||
| MCSymbol *MCStreamer::emitCFILabel() { | |||
| // Return a dummy non-null value so that label fields appear filled in when | |||
| // generating textual assembly. | |||
| return (MCSymbol *)1; | |||
| static size_t DummyLabelValue = 0; | |||
There was a problem hiding this comment.
What is this supposed to do?
There was a problem hiding this comment.
The original code here is weird, not sure if there is a better way to handle it (create a label that doesn't appear in the text output)?
I made this change since we're now using these labels to index into a map, so I need each label to be unique.
There was a problem hiding this comment.
emitCFILabel() is overridden for binary streamers, so this implementation is only used if your outputting text. If you're outputting text, you shouldn't need to construct the unwind maps in the first place.
If you want to construct the maps anyway, for some reason, I'd suggest identifying entries in the EpilogMap using an integer ID, instead of the label.
There was a problem hiding this comment.
(Also, non-const static variables are basically banned in LLVM code: in contexts like LTO, we run multiple instances of the compiler in parallel, and a static variable will explode if you do that.)
There was a problem hiding this comment.
Ok, I removed the map and replaced with a vector, then changed CurrentEpilog to point to the Epilog object itself (since it was primarily used to index into the map).
Unfortunately, this isn't going to work. @pmsjt informs me that we can't have epilogs in chained unwind tables, since they are only for additional savereg codes. Using the "assume terminator is 1-byte" trick allows mixed terminators in a single function, so the only other place we'd need it is for functions with more than 256 unwind codes, where the dev will have to split their functions manually. Not sure if I should add a heuristic into the pass to downgrade to v1 if there might be too many codes?
From there existing tests, there was only one non-compliant epilog: llvm-project/llvm/test/CodeGen/X86/avx512-regcall-Mask.ll Lines 952 to 958 in 818bca8 But yes, my current plans are:
If I observe cases where the offset is too large, my plan is to modify the pass to move blocks closer to the end of the function if they are beyond some heuristic (basically guessing what the offset would be). |
How do epilogs work in chained unwind tables? Do the epilog opcodes from the original table get ignored? If they do get ignored, can you specify different opcodes in the chained table? If they don't get ignored, are the specified offsets relative to the chained unwind table? Also, you don't actually have to chain the unwind tables; you can just specify the prolog is size zero, I think?
I suspect you can hit the 4KB limit before you hit the opcode limit. |
They are additive. When execution is on a fragment, it is assumed that all unwind opcodes from the parent fragment(s), all the way to the principal fragment, are already applied. |
|
So if an unwind table has an epilog opcode, and you chain another unwind table to it, the chained unwind table inherits the epilog opcode? You could use that. The primary unwind table ends at the first epilog, you mark it with an "atend" epilog opcode. The next region starts immediately after that, and continues until the next epilog. The subsequent region uses a chained unwind table. The chained unwind table inherits the atend marking, so it has a correct epilog marking. Repeat for each epilog, and you can cover a function with an arbitrary number of epilogs without repeating the opcodes. That's not great for codesize, and it excludes any other use of chaining... but it's better than just giving up. Bailing out is a hazard: minor code changes can trigger the error, and the connection between the user's code and the error is hard to understand. |
|
I don't know what an "atend" is, but what you are describing doesn't seem to be a possible solution. Epilogs must be present in the main function entry. Only the main function entry can contain unwind opcodes which make changes to the stack pointer and return (or tail-call). Fragments can save and restore registers with mov but not push/pop. Think of fragments as extensions of the body, which can save and restore aditional registers, but they won't have prologs or epilogs of their own. |
|
You're saying if you have a chained unwind, epilog opcodes from the parent table and the child table are ignored? Any thoughts on not using chained unwind, setting the prologue size to zero? |
|
No, I said the exact oposite: They are additive. When execution is on a fragment, it is assumed that all unwind opcodes from the parent fragment(s), all the way to the principal fragment, are already applied. |
|
Okay, so the chained table includes the opcodes from the parent, including any epilogue markings. Those epilogue markings include offsets relative to the end of the function. Is that always relative to the end of the parent function? If so, I guess you can't usefully use chained unwind tables in this context. |
|
Not quite. Chained entries contain a link to the parent entry, not a copy of the unwind opcodes. Chained entries have only the opcodes for their own fragment spills and fills. |
|
But is the And is the first epilog code in a chained entry assumed to be a "header" epilog code (i.e., has |
|
The way chaining works with Epilog V2 is a bit contrived. Let me start over and I hope this might help. Prolog entries chain up to form a sequence of opcodes which are additive. When execution is in the body or prolog of a fragment, the unwinder will process (in reverse) all the prolog opcodes that apply to the current fragment (may not be all, if execution is in within the prolog) plus all the prolog opcodes for the parent fragments. x86-64 Epilogs are strange. Unlike Arm, x86-64 EpilogV2 don't have their own op list - they borrow the Prolog's list. If execution is in the Epilog of a fragment, the unwinder will collect all the unwinder opcodes for the current fragment in addition to the paren't fragments and apply them. The rules for UWOP_EPILOG in a fragment are the same as the main function entry. The catch is that all the epilog information is local to the fragment. It'll have its own header and epilog entries. The UWOP_EPILOG entries from the parent fragments are not relevant because UWOP_EPILOG's only identify location, not the operations to perform. The determination of the Epilog start and length is completely local. What is not local is the list of operations to perform - since those are the Prolog's list, then the unwinder will run up the chain list to collect all UWOPs to revert their actions. In practive, it really only cares about the main fragment, since all the remaining fragments can only have SAVE_NONVOL which Epilog processing assumes to be restored by the time the Epilog starts. This is a real-world example: This function has 3 segments (it has more, really, but I'll use just these 3). The main fragment doesn't even have an Epilog and that's OK. The main fragment's prolog looks like this: In its Prolog, the second fragment does and in its Epilog, it does which is not relevant because the uwnider only cares about when POPs start and fragments can't have them. The third fragment doesn't have any prolog. Could have it (would still be restricted to SAVE_NONVOL). It does, however, have two Epilogs. Both look the same and So, yes (sorry if I misled you otherwise) @dpaoliello , it is a completely valid option to have two different fragments, each with its own Epilog header. You can have no Epilog on the main entry and then fragments for each Epilog. Or have one Epilog in the main entry and then have additional fragments for Epilogs. You can have one fragment for the RET Epilogs and one fragment for the tail-call Epilogs. And so on... |
|
Thanks for the info! For the different terminators, I'd prefer to keep to my current "assume 1-byte" trick: the implementation is simple, and it will reduce the amount of unwind data we need to emit. But for the "too many unwind codes" and "too far from the end" scenarios, it sounds like we'll be able to use chaining to work around them. |
| @@ -7788,6 +7788,10 @@ def import_call_optimization : Flag<["-"], "import-call-optimization">, | |||
| "by the Windows kernel to enable import call optimization">, | |||
| MarshallingInfoFlag<CodeGenOpts<"ImportCallOptimization">>; | |||
|
|
|||
| def epilog_unwind : Flag<["-"], "winx64-eh-unwindv2">, | |||
| HelpText<"Enable unwind v2 (epilog) information for x64 Windows">, | |||
| MarshallingInfoFlag<CodeGenOpts<"WinX64EHUnwindV2">>; | |||
There was a problem hiding this comment.
We might want a regular clang driver flag for this, so it isn't only accessible through clang-cl.
There was a problem hiding this comment.
Took the opportunity to switch it to a bool flag as well, so that we can enable it by default at some point
| // layout has been completed. | ||
| auto *MCE = MCUnwindV2EpilogTargetExpr::create(*info, Epilog, EpilogSize, | ||
| context); | ||
| MCFixup Fixup = MCFixup::create(DF->getContents().size(), MCE, FK_Data_2); |
There was a problem hiding this comment.
You might run into trouble trying to measure the size of the function here, if you need to in the future. This doesn't block the patch in its current form. But if you need to check whether an epilogue is close to the end of the function, you might need to defer some of epilogue emission to an MCFragment, so it can interact with relaxation.
There was a problem hiding this comment.
you might need to defer some of epilogue emission to an MCFragment, so it can interact with relaxation
That's what I'm currently doing: I emit a custom fixup here that is resolved after relaxation so that I can get the distance between the epilog and end of the function.
For future work to chain unwind infos if the function is too large, I'll probably have to use a heuristic that guesses when the distance is too large, and err on the side of splitting.
There was a problem hiding this comment.
I guess you could conservatively estimate the size of a fragment, maybe? We currently don't have any code that tries to do that, but I guess it's theoretically possible.
I was thinking more that you could try to defer the decision of whether to split to relaxation time, but that's more complicated to implement, I guess.
| MCSymbolRefExpr::create(RHS, Context), Context); | ||
| // It should normally be possible to calculate the length of a function | ||
| // at this point, but it might not be possible in the presence of certain | ||
| // unusual constructs, like an inline asm with an alignment directive. |
There was a problem hiding this comment.
This function will fail a lot more frequently on x86 compared to other targets. Because wide branch instructions have a 32-bit offset, the compiler doesn't do relaxation itself; it emits branches with an 8-bit offset, and lets the assembler relax them if necessary. Which is usually convenient, but it makes things harder in this context.
I guess this doesn't necessarily block the patch, but I'm surprised this hasn't already caused problems for you.
There was a problem hiding this comment.
This is actually pre-existing code that I moved earlier since I needed to call it in a different function.
It's definitely caused issues for me: you'll note that I only use this to measure the size of an epilog, so it's always measuring within the same block across a known set of instructions.
For the distance between an epilog and the end of the function I had to create a custom fixup for exactly the reason that you mention.
llvm/include/llvm/MC/MCWinEH.h
Outdated
|
|
||
| auto findEpilog(const MCSymbol *Start) { | ||
| return llvm::find_if(Epilogs, | ||
| [Start](Epilog &E) { return E.Start == Start; }); |
There was a problem hiding this comment.
This is linear, so if you do it for every epilogue, it's quadratic in the number of epilogues. Which is probably fine in most cases, but you can get a lot of epilogues in some weird cases.
There was a problem hiding this comment.
Ok, I've found a solution that solves my original problem but also keeps the maps: I've switched back to a MapVector but continue to use CurrentWinEpilog in the streamer AND use insert_or_assign whenever we begin a new epilog so that the current epilog is reset when only emitting text.
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
…ion (equivalent to MSVC /d2epilogunwind)
| MCValue &Res, const MCAssembler *Asm) const { | ||
| // Calculate the offset to this epilog, and validate it's within the allowed | ||
| // range. | ||
| auto Offset = GetOptionalAbsDifference(*Asm, FunctionEnd, UnwindV2Start); |
There was a problem hiding this comment.
This looks like it could fail, at first glance... but I guess this runs late enough that it succeeds.
Adds support for emitting Windows x64 Unwind V2 information, includes support
/d2epilogunwindin clang-cl.Unwind v2 adds information about the epilogs in functions such that the unwinder can unwind even in the middle of an epilog, without having to disassembly the function to see what has or has not been cleaned up.
Unwind v2 requires that all epilogs are in "canonical" form:
PUSHin the prolog there must be a correspondingPOPinstruction in exact reverse order.This change adds a pass to validate epilogs in modules that have Unwind v2 enabled and, if they pass, emits new pseudo instructions to MC that 1) note that the function is using unwind v2 and 2) mark the start of the epilog (this is either the first
POPif there is one, otherwise the terminator instruction). If a function does not meet these requirements, it is downgraded to Unwind v1 (i.e., these new pseudo instructions are not emitted).Note that the unwind v2 table only marks the size of the epilog in the "header" unwind code, but it's possible for epilogs to use different terminator instructions thus they are not all the same size. As a work around for this, MC will assume that all terminator instructions are 1-byte long - this still works correctly with the Windows unwinder as it is only using the size to do a range check to see if a thread is in an epilog or not, and since the instruction pointer will never be in the middle of an instruction and the terminator is always at the end of an epilog the range check will function correctly. This does mean, however, that the "at end" optimization (where an epilog unwind code can be elided if the last epilog is at the end of the function) can only be used if the terminator is 1-byte long.
One other complication with the implementation is that the unwind table for a function is emitted during streaming, however we can't calculate the distance between an epilog and the end of the function at that time as layout hasn't been completed yet (thus some instructions may be relaxed). To work around this, epilog unwind codes are emitted via a fixup. This also means that we can't pre-emptively downgrade a function to Unwind v1 if one of these offsets is too large, so instead we raise an error (but I've passed through the location information, so the user will know which of their functions is problematic).