Export to GitHub

rapidjson - issue #104

SIMD instruction can cause seg-fault/UMR


Posted on Mar 18, 2014 by Helpful Hippo

The usage of rapidjson::SkipWhitespace_SIMD(char const*) reads 16 bytes (correct me if I'm wrong) at a time. It can cause uninitialized memory read on the following condition: If rapidjson::SkipWhitespace_SIMD(char const*) is called at close to the end of string buffer which has less than 16 bytes of allocated space, the function will read beyond the memory it owns.

What steps will reproduce the problem? Non-deterministic. However, you can get the error message just by running the simple test driver with valgrind.

What is the expected output? What do you see instead? Can cause seg-fault.

What version of the product are you using? On what operating system? rhel5_x86-64, rhel6_x86-64

Comment #1

Posted on Jun 20, 2014 by Happy Dog

(No comment was entered for this change.)

Comment #2

Posted on Jun 24, 2014 by Happy Dog

I have inspected the code and tried to find solution. However, I cannot find a possible workaround within the library. This is because it will need the length of buffer in order to prevent reading out of boundary. I think the easy and efficient way is to allocate extra 15 bytes in the source string buffer at user's code. I can add some warning text in the document. What do you think?

Comment #3

Posted on Jun 24, 2014 by Helpful Hippo

That is the same workaround we use in our code. We had some ideas to fix it (like take an extra string length argument) and conclude that it is too much work comparing to just allocate 15 bytes more. I think it is fair to do the workaround with proper warnings. In our use case, we parse around 50 million JSON files/buffers per day and we got hit by the bug around 100 times per day on average before the workaround.

Comment #4

Posted on Jul 29, 2014 by Happy Dog

https://github.com/miloyip/rapidjson/issues/85 This bug has been fixed. You can just copy the two SkipWhitespace_SIMD() functions to your version if you do not want to upgrade RapidJSON. Please help verifying the bug if possible. Thank you.

Comment #5

Posted on Jul 29, 2014 by Helpful Hippo

Thanks. We are merging with your code and will profile with valgrind.

Comment #6

Posted on Jul 30, 2014 by Helpful Hippo

valgrind still complains for uninitialized memory read. I will send more detailed analysis later.

Status: Fixed

Labels:
Type-Defect Priority-Medium