-
Notifications
You must be signed in to change notification settings - Fork 493
ORC-445: [C++] Code improvements in RLEV2Util. #346
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
c++/src/RLEV2Util.hh
Outdated
} else { | ||
return 64; | ||
} | ||
return ClosestFixedBitsMap[n]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what if n > 64?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@xndai Hi Xiening, thanks for the catch. When computing patch width, it is possible for the input n to be larger than 64. I have added boundary check in this function and getClosestAlignedFixedBits().
Can you please provide some perf data to confirm the refactoring is effective? |
@@ -23,85 +23,32 @@ | |||
|
|||
namespace orc { | |||
extern const uint32_t FBSToBitWidthMap[FixedBitSizes::SIZE]; | |||
extern const uint32_t ClosestFixedBitsMap[65]; | |||
extern const uint32_t ClosestAlignedFixedBitsMap[65]; | |||
extern const uint32_t BitWidthToFBSMap[65]; | |||
|
|||
inline uint32_t decodeBitWidth(uint32_t n) { | |||
return FBSToBitWidthMap[n]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
make sure this won't be out of bound.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi Gang, I thought this over and don't feel we need to add boundary check here for two reasons:
- in current RLEv2 encoder and decoder code, all the callers of decodeBitWidth() pass in a integer that only has the lowest 5 bits set, so the input n is always less than FixedBitSizes::SIZE (32).
- Since this function is used to decode the 5-bit length code, it would be a programming error for a caller to pass in any value that is >= 32. In this case, returning 64 (as the original implementation does) is not helping.
@wgtmac Hi Gang, thanks for the comments. I'll add performance measurements later this week. |
…ry check; 3) add unit test.
@wgtmac Hi Gang, The new functions are 20 to 40% faster than the original ones. I've also committed some new changes:
Please let me know if you have further suggestions. Thanks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 LGTM. Thanks @fangzheng for the detail perf data!
No description provided.