Description
We've been encountering issues getting the handshake to work on iOS devices. After a lot of work, we've verified that SHA1 seems to be operating correctly, but the problem is how the pack procedure is configured.
When the variables h0-h4 are passed in to pack as unsigned integers, we see good values being generated on OSX. This includes the iOS Simulator.
Using the latest XCode and running on a device (you can't duplicate this issue using the Simulator), if the variable is > 2^31, it's treated as a negative number and is turned into a zero on iOS.
By converting the pack routine to using signed integers in src/websocket//tools.lua,
We changed the final line in the sha1 function
return spack('>I>I>I>I>I',h0,h1,h2,h3,h4)
to the following:
return spack('>i>i>i>i>i',h0,h1,h2,h3,h4)
It does seem counter-intuitive, but using unsigned integers causes lpack to fall into the C++ standard (4.9) of undefined conversion behaviour, and using signed integers gives the desired behaviour in lpack.