Msgpack truncates data member

[paddybyers]

We are seeing truncation of the data member when a ProtocolMessage is encoded.

The problem has been confirmed on Android 4.4.2 with the ably-java 0.8.8 library, which uses msgpack-core 0.8.11.

The problem is not seen on ably-java 0.8.7 (using msgpack-core 0.8.2).

The following platforms seem to have no issue:

Any Oracle JRE
Android 5.0.1, 6.0, 7.0.

The suspicion is a incompatibility between the msgpack lib and the version of java.nio.charset.CharsetEncoderICU in earlier Android versions. There have been several issues with msgpack, and there are still open issues on 0.8.11: msgpack/msgpack-java#405.

Plan is to:

• narrow down the specific string or struct that causes the issue so we can file an issue with msgpack;

• in the ably-java lib, force use of the JSON encoding on android versions < 5.0.

[mattheworiordan]

Given the bump from 0.8.2 up to 0.8.11, could we try a version in between that doesn't exhibit these issues?

[paddybyers]

Given the bump from 0.8.2 up to 0.8.11, could we try a version in between that doesn't exhibit these issues?

Yes we could, but 0.8.11 was made to fix other similar issues, so we would just be creating unknown situations in which we fail.

[paddybyers]

This has been isolated to a simple case where a map has a single string member of 512 bytes or greater:

msgpack/msgpack-java#406

[mattheworiordan]

Another reported incident of this I am afraid in support ticket 1471. Unfortunately this time it's Android 5:

Samsung Galaxy S5 (Model SM-G900F)
Android 5.0
I've try upgraded to 0.8.9 and issue is still on but setting useBinaryProtocol to false is fix the issue.

[paddybyers]

It would be useful if we could confirm that this really is the same underlying issue: would it be possible to run the attached apk on the device and see if it confirms the presence of the msgpack issue? (Run the app, and a Toast will show with "success" or "fail").
MsgpackTest.zip

[paddybyers]

I've implemented a different fix, based on the workaround suggested in the upstream msgpack bug.

[paddybyers]

The bug is now fixed upstream: msgpack/msgpack-java#409

We will migrate once that gets released.