Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

decoding: decoder max should match MAX_INSTR_LENGTH #9

Open
derekbruening opened this issue Nov 27, 2014 · 0 comments
Open

decoding: decoder max should match MAX_INSTR_LENGTH #9

derekbruening opened this issue Nov 27, 2014 · 0 comments

Comments

@derekbruening
Copy link
Contributor

From derek.br...@gmail.com on February 14, 2009 10:10:53

Today we use a MAX_INSTR_LENGTH define of 17 in various places yet our own
decoder does not stop at 17. The two should match so we can allocate a
buffer and decode from it. Since it is a conservative estimate the maximum
should probably be higher since some processors (at list historically) have
allowed 18, 20, and even (so I've heard, never seen) 24-byte instructions.
The AMD manual claims 15 is the limit, and 17 used to be the typically
assumed limit, but the actual limit at which a processor raises #GP is
variable. We shouldn't make it too high though since we don't want to read
off onto the next page when not necessary.

Example of our decoder on a 23-byte instr (this raises #GP on my Q9300):

0x080494a8 66 66 66 66 66 66 66 data16 nop 0x00000000(%eax,%eax,1)
66 66 66 66 66 66 66 66 0f 1f 84 00 00 00 00 00

Original issue: http://code.google.com/p/dynamorio/issues/detail?id=9

This was referenced Nov 27, 2014
derekbruening added a commit that referenced this issue Jun 14, 2020
Adds new options and interfaces to specify alternate-bitwidth client
libraries for use when the application creates a child process of the
other bitwidth.

For DR, adds -client_lib32 and -client_lib64 options.  Switches main
usage to use the appropriate option, with its contents then copied
into -client_lib (to avoid the pain of removing that options).

For drconfiglib, adds dr_register_client_ex() with
dr_client_iterator_next_ex() to support querying other-bitwidth client
registrations.

Adds a new libutil.drconfig_test for drconfiglib.  Fixes a bug found
by the test: existing client queries were cutting off the last
character of the path and options.

For drrun and drconfig, adds "-c32" and "-c64" options, with an
additional "--" separating the client options between them.

For tool files, adds CLIENT{32,64}_{REL,ABS}.  Updates drcov,
drcpusim, and drcachesim to use the new syntax and drcachesim's
launcher to process it.  Tested these manually:
    ===========================================================================
    $ ninja install
    $ rm *.log
    $ ../exports/bin64/drrun -t drcov -- ~/dr/test/execve64 ~/dr/test/hello32
    $ l -t *.log
    20K drcov.execve64.109583.0000.proc.log  20K drcov.execve64.109585.0000.proc.log
    20K drcov.hello32.109585.0000.proc.log
    $ rm -rf drm*.dir
    $ ../exports/bin64/drrun -verbose -t drcachesim -verbose 1 -offline -- ~/dr/test/execve64 ~/dr/test/hello32
    $ l -td *.dir
    4.0K drmemtrace.hello32.117714.7095.dir/  4.0K drmemtrace.execve64.117714.6260.dir/
    4.0K drmemtrace.execve64.117713.9314.dir/
    ===========================================================================

Adds tests of -c32/-c64 to the existing cross-arch linux.execve{32,64}
tests (Windows won't work until #803 is addressed).
The tests look like this:
    ===========================================================================
    $ cmake . && ctest -V -R 'linux.execve(32|64)'

    8: Running test command: "/home/bruening/dr/git/build_x64_dbg_tests/bin64/drrun" "-32" "-dr_home" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/32bit" "-stderr_mask" "0xC" "-dumpcore_mask" "0" "-c32" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/32bit/suite/tests/bin/libclient.large_options.dll.so" "-paramA" "foo" "-paramB" "bar" "--" "-c64" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/bin/libclient.large_options.dll.so" "-paramA" "foo" "-paramB" "bar" "--" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/32bit/suite/tests/bin/linux.execve32" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/bin/linux.execve-sub64"
    8: large_options passed: -paramA foo -paramB bar
    8: parent is running under DynamoRIO
    8: parent waiting for child
    8: child is running under DynamoRIO
    8: large_options passed: -paramA foo -paramB bar
    8: it_worked
    8: running under DynamoRIO
    8: large_options exiting
    8: child has exited
    8: large_options exiting
    8:
    1/2 Test #8: linux.execve32 ...................   Passed    3.93 sec

    9: Test command: /home/bruening/dr/git/build_x64_dbg_tests/bin64/drrun "-stderr_mask" "0xC" "-dumpcore_mask" "0" "-c32" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/32bit/suite/tests/bin/libclient.large_options.dll.so" "-paramA" "foo" "-paramB" "bar" "--" "-c64" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/bin/libclient.large_options.dll.so" "-paramA" "foo" "-paramB" "bar" "--" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/bin/linux.execve64" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/32bit/suite/tests/bin/linux.execve-sub32"
    9: Test timeout computed to be: 1500
    9: large_options passed: -paramA foo -paramB bar
    9: parent is running under DynamoRIO
    9: parent waiting for child
    9: child is running under DynamoRIO
    9: large_options passed: -paramA foo -paramB bar
    9: it_worked
    9: running under DynamoRIO
    9: large_options exiting
    9: child has exited
    9: large_options exiting
    2/2 Test #9: linux.execve64 ...................   Passed    0.86 sec
    ===========================================================================

Issue: #147, #803
Fixes: #147
derekbruening added a commit that referenced this issue Jun 15, 2020
Adds new options and interfaces to specify alternate-bitwidth client
libraries for use when the application creates a child process of the
other bitwidth.

For DR, adds -client_lib32 and -client_lib64 options.  Switches main
usage to use the appropriate option, with its contents then copied
into -client_lib (to avoid the pain of removing that options).

For drconfiglib, adds dr_register_client_ex() with
dr_client_iterator_next_ex() to support querying other-bitwidth client
registrations.

Adds a new libutil.drconfig_test for drconfiglib.  Fixes a UNIX bug
found by the test: existing client queries were cutting off the last
character of the path and options due to differing snprintf semantics.
Also fixes a Windows drconfig handle leak found by the test that
was preventing unregistration from deleting config files.

For drrun and drconfig, adds "-c32" and "-c64" options, with an
additional "--" separating the client options between them.

For tool files, adds CLIENT{32,64}_{REL,ABS}.  Updates drcov,
drcpusim, and drcachesim to use the new syntax and drcachesim's
launcher to process it.  Tested these manually:
    ===========================================================================
    $ ninja install
    $ rm *.log
    $ ../exports/bin64/drrun -t drcov -- ~/dr/test/execve64 ~/dr/test/hello32
    $ l -t *.log
    20K drcov.execve64.109583.0000.proc.log  20K drcov.execve64.109585.0000.proc.log
    20K drcov.hello32.109585.0000.proc.log
    $ rm -rf drm*.dir
    $ ../exports/bin64/drrun -verbose -t drcachesim -verbose 1 -offline -- ~/dr/test/execve64 ~/dr/test/hello32
    $ l -td *.dir
    4.0K drmemtrace.hello32.117714.7095.dir/  4.0K drmemtrace.execve64.117714.6260.dir/
    4.0K drmemtrace.execve64.117713.9314.dir/
    ===========================================================================

Adds tests of -c32/-c64 to the existing cross-arch linux.execve{32,64}
tests (Windows won't work until #803 is addressed).
The tests look like this:
    ===========================================================================
    $ cmake . && ctest -V -R 'linux.execve(32|64)'

    8: Running test command: "/home/bruening/dr/git/build_x64_dbg_tests/bin64/drrun" "-32" "-dr_home" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/32bit" "-stderr_mask" "0xC" "-dumpcore_mask" "0" "-c32" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/32bit/suite/tests/bin/libclient.large_options.dll.so" "-paramA" "foo" "-paramB" "bar" "--" "-c64" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/bin/libclient.large_options.dll.so" "-paramA" "foo" "-paramB" "bar" "--" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/32bit/suite/tests/bin/linux.execve32" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/bin/linux.execve-sub64"
    8: large_options passed: -paramA foo -paramB bar
    8: parent is running under DynamoRIO
    8: parent waiting for child
    8: child is running under DynamoRIO
    8: large_options passed: -paramA foo -paramB bar
    8: it_worked
    8: running under DynamoRIO
    8: large_options exiting
    8: child has exited
    8: large_options exiting
    8:
    1/2 Test #8: linux.execve32 ...................   Passed    3.93 sec

    9: Test command: /home/bruening/dr/git/build_x64_dbg_tests/bin64/drrun "-stderr_mask" "0xC" "-dumpcore_mask" "0" "-c32" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/32bit/suite/tests/bin/libclient.large_options.dll.so" "-paramA" "foo" "-paramB" "bar" "--" "-c64" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/bin/libclient.large_options.dll.so" "-paramA" "foo" "-paramB" "bar" "--" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/bin/linux.execve64" "/home/bruening/dr/git/build_x64_dbg_tests/suite/tests/32bit/suite/tests/bin/linux.execve-sub32"
    9: Test timeout computed to be: 1500
    9: large_options passed: -paramA foo -paramB bar
    9: parent is running under DynamoRIO
    9: parent waiting for child
    9: child is running under DynamoRIO
    9: large_options passed: -paramA foo -paramB bar
    9: it_worked
    9: running under DynamoRIO
    9: large_options exiting
    9: child has exited
    9: large_options exiting
    2/2 Test #9: linux.execve64 ...................   Passed    0.86 sec
    ===========================================================================

Issue: #147, #803
Fixes: #147
derekbruening added a commit that referenced this issue Jun 24, 2022
The view tool was not printing the virtual address for
physical-to-virtual translations, nor the address for software
prefetches, which made it hard to diagnose physical address failures
in general and in particular when the culprit virtual address was from
a prefetch as it did not appear in the text output.  We add that here.

Since some prefetches have long names, we abbreviate them, but still
need to widen the name field to align them all.

While at it, we add the function marker values to the view tool as
well.

Example output:
       10: T462774 ifetch        7 byte(s) @ 0x00007f6cae908a54 non-branch
       11: T462774 read          8 byte(s) @ 0x00007f6cae90ca20 by PC 0x00007f6cae908a54
       12: T462774 ifetch        4 byte(s) @ 0x00007f6cae908a5c non-branch
       13: T462774 pref-w-L3-NT  8 byte(s) @ 0x00007f6c389xd238 by PC 0x00007f6cae908a5c
      ...
   441522: T48816 <marker: function #9>
   441523: T48816 <marker: function return address 0xaaaad5de4944>
   441524: T48816 <marker: function argument 0x15187bd4ae1d>
   441526: T48816 <marker: function argument 0x1>
      ...
   441546: T48816 <marker: function #9>
   441547: T48816 <marker: function return value 0x15187bd4ae1d>

Issue: #4014
derekbruening added a commit that referenced this issue Jun 28, 2022
…5544)

The view tool was not printing the virtual address for
virtual-to-physical translations, nor the address for software
prefetches, which made it hard to diagnose physical address failures
in general and in particular when the culprit virtual address was from
a prefetch as it did not appear in the text output.  We add that here.

Since some prefetches have long names, we abbreviate them, but still
need to widen the name field to align them all.

While at it, we add the function marker values to the view tool as
well.

Example output:
       10: T462774 ifetch        7 byte(s) @ 0x00007f6cae908a54 non-branch
       11: T462774 read          8 byte(s) @ 0x00007f6cae90ca20 by PC 0x00007f6cae908a54
       12: T462774 ifetch        4 byte(s) @ 0x00007f6cae908a5c non-branch
       13: T462774 pref-w-L3-NT  8 byte(s) @ 0x00007f6c389xd238 by PC 0x00007f6cae908a5c
      ...
   441522: T48816 <marker: function #9>
   441523: T48816 <marker: function return address 0xaaaad5de4944>
   441524: T48816 <marker: function argument 0x15187bd4ae1d>
   441526: T48816 <marker: function argument 0x1>
      ...
   441546: T48816 <marker: function #9>
   441547: T48816 <marker: function return value 0x15187bd4ae1d>

Issue: #4014
derekbruening added a commit that referenced this issue Jul 5, 2023
8 of the 13 tests on Mac AArch64 labeled "OSX" fail prior to this PR.
Here we fix the following:

+ Syscall success is indicated by the carry flag just like x86 Mac
+ Handle sigreturn with its extra parameters just like x86 Mac
+ Fix signal handler parameters
+ Fix stolen register support in signal contexts
+ Use MAP_JIT and pthread_jit_write_protect_np for +rwx gencode in tests
+ Use DYLD_LIBRARY_PATH on Mac in tests

Now all 13 tests pass:
---------------------------------------------------------------------------------------
ctest -j 5 -L OSX
 1/13 Test  #13: code_api|common.fib ................................  Passed  0.59 sec
 2/13 Test #243: code_api|libutil.frontend_test .....................  Passed  0.63 sec
 3/13 Test #231: code_api|api.ir ....................................  Passed  0.67 sec
 4/13 Test   #9: code_api|linux.sigaction.native ....................  Passed  0.25 sec
 5/13 Test  #31: code_api|linux.signal0000 ..........................  Passed  0.10 sec
 6/13 Test #240: code_api|api.ir-static .............................  Passed  0.34 sec
 7/13 Test #241: code_api|api.drdecode ..............................  Passed  0.38 sec
 8/13 Test #245: code_api|api.dis-a64 ...............................  Passed  1.15 sec
 9/13 Test #264: no_code_api,no_intercept_all_signals|linux.sigaction  Passed  0.08 sec
10/13 Test  #33: code_api|linux.signal0010 ..........................  Passed  0.34 sec
11/13 Test  #35: code_api|linux.signal0100 ..........................  Passed  0.42 sec
12/13 Test  #37: code_api|linux.signal0110 ..........................  Passed  0.45 sec
13/13 Test   #7: samples_proj .......................................  Passed  1.89 sec
100% tests passed, 0 tests failed out of 13
---------------------------------------------------------------------------------------

Issue: #5383
github-merge-queue bot pushed a commit that referenced this issue Jul 7, 2023
8 of the 13 tests on Mac AArch64 labeled "OSX" fail prior to this PR.
Here we fix the following:

+ Syscall success is indicated by the carry flag just like x86 Mac
+ Handle sigreturn with its extra parameters just like x86 Mac
+ Fix signal handler parameters
+ Fix stolen register support in signal contexts
+ Use MAP_JIT and pthread_jit_write_protect_np for +rwx gencode in tests
+ Use DYLD_LIBRARY_PATH on Mac in tests

Now all 13 tests pass:
```
---------------------------------------------------------------------------------------
$ ctest -j 5 -L OSX
 1/13 Test  #13: code_api|common.fib ................................  Passed  0.59 sec
 2/13 Test #243: code_api|libutil.frontend_test .....................  Passed  0.63 sec
 3/13 Test #231: code_api|api.ir ....................................  Passed  0.67 sec
 4/13 Test   #9: code_api|linux.sigaction.native ....................  Passed  0.25 sec
 5/13 Test  #31: code_api|linux.signal0000 ..........................  Passed  0.10 sec
 6/13 Test #240: code_api|api.ir-static .............................  Passed  0.34 sec
 7/13 Test #241: code_api|api.drdecode ..............................  Passed  0.38 sec
 8/13 Test #245: code_api|api.dis-a64 ...............................  Passed  1.15 sec
 9/13 Test #264: no_code_api,no_intercept_all_signals|linux.sigaction  Passed  0.08 sec
10/13 Test  #33: code_api|linux.signal0010 ..........................  Passed  0.34 sec
11/13 Test  #35: code_api|linux.signal0100 ..........................  Passed  0.42 sec
12/13 Test  #37: code_api|linux.signal0110 ..........................  Passed  0.45 sec
13/13 Test   #7: samples_proj .......................................  Passed  1.89 sec
100% tests passed, 0 tests failed out of 13
---------------------------------------------------------------------------------------
```
Issue: #5383
derekbruening added a commit that referenced this issue Oct 5, 2023
Adds a new option -record_sysall to drmemtrace which records the
parameter and success values for the given system call numbers.  Just
like with -record_function, the user must specify the parameter count.

SYS_futex is left as traced by default, but it can be disabled.

Adds documentation and a test.

Further manual testing:

  ---------------------------------------------------------------------------
  $ rm -rf drmemtrace.*.dir; bin64/drrun -t drcachesim -offline -record_syscall '1|-3&12|4&9|2' -record_syscall '12|2&158|4' -- suite/tests/bin/simple_app && bin64/drrun -t drcachesim -indir drmemtrace.*.dir -simulator_type view 2>&1 | egrep 'system call |function'
  Error: -record_syscall invalid parameter count -3
  ---------------------------------------------------------------------------

  ---------------------------------------------------------------------------
  $ rm -rf drmemtrace.*.dir; bin64/drrun -t drcachesim -offline -record_syscall '1|-3&12|4&9|2' -record_syscall '12|2&158|4' -- suite/tests/bin/simple_app && bin64/drrun -t drcachesim -indir drmemtrace.*.dir -simulator_type view 2>&1 | egrep 'system call |function'
  <Application simple_app (484125) DynamoRIO usage error : invalid system call parameter number>
  ---------------------------------------------------------------------------

  ---------------------------------------------------------------------------
  $ rm -rf drmemtrace.*.dir; bin64/drrun -t drcachesim -offline -record_syscall '1|3&12|4&9|2' -record_syscall '12|2&158|4' -- suite/tests/bin/simple_app && bin64/drrun -t drcachesim -indir drmemtrace.*.dir -simulator_type view 2>&1 | egrep 'system call |function'
  ...
  Hello, world!
  <Stopping application simple_app (484049)>
         32411       26808:      484049 <marker: system call 12>
         32412       26808:      484049 <marker: function==syscall #12>
         32413       26808:      484049 <marker: function argument 0x0>
         32414       26808:      484049 <marker: function argument 0x7ffc87c52d4c>
         32415       26808:      484049 <marker: function==syscall #12>
         32416       26808:      484049 <marker: function return value 0x1>
         50436       41126:      484049 <marker: system call 9>
         50437       41126:      484049 <marker: function==syscall #9>
         50438       41126:      484049 <marker: function argument 0x0>
         50439       41126:      484049 <marker: function argument 0x2000>
         50440       41126:      484049 <marker: function==syscall #9>
         50441       41126:      484049 <marker: function return value 0x1>
         50980       41485:      484049 <marker: system call 21>
         52193       42391:      484049 <marker: system call 257>
         52223       42409:      484049 <marker: system call 262>
         52253       42430:      484049 <marker: system call 9>
         52254       42430:      484049 <marker: function==syscall #9>
         52255       42430:      484049 <marker: function argument 0x0>
         52256       42430:      484049 <marker: function argument 0x1b5c7>
         52257       42430:      484049 <marker: function==syscall #9>
         52258       42430:      484049 <marker: function return value 0x1>
         52274       42439:      484049 <marker: system call 3>
         54224       44056:      484049 <marker: system call 257>
         54252       44074:      484049 <marker: system call 0>
         54329       44130:      484049 <marker: system call 17>
         54455       44208:      484049 <marker: system call 262>
         54988       44591:      484049 <marker: system call 17>
         55653       45105:      484049 <marker: system call 9>
         55654       45105:      484049 <marker: function==syscall #9>
         55655       45105:      484049 <marker: function argument 0x0>
         55656       45105:      484049 <marker: function argument 0x1e1f50>
         55657       45105:      484049 <marker: function==syscall #9>
         55658       45105:      484049 <marker: function return value 0x1>
         ...
  ---------------------------------------------------------------------------

Fixes #6344
derekbruening added a commit that referenced this issue Oct 5, 2023
Adds a new option -record_sysall to drmemtrace which records the
parameter and success values for the given system call numbers. Just
like with -record_function, the user must specify the parameter count.

SYS_futex is left as traced by default, but it can be disabled.

Adds documentation and a test.

Further manual testing:
```
  ---------------------------------------------------------------------------
  $ rm -rf drmemtrace.*.dir; bin64/drrun -t drcachesim -offline -record_syscall '1|-3&12|4&9|2' -record_syscall '12|2&158|4' -- suite/tests/bin/simple_app && bin64/drrun -t drcachesim -indir drmemtrace.*.dir -simulator_type view 2>&1 | egrep 'system call |function'
  Error: -record_syscall invalid parameter count -3
  ---------------------------------------------------------------------------

  ---------------------------------------------------------------------------
  $ rm -rf drmemtrace.*.dir; bin64/drrun -t drcachesim -offline -record_syscall '1|-3&12|4&9|2' -record_syscall '12|2&158|4' -- suite/tests/bin/simple_app && bin64/drrun -t drcachesim -indir drmemtrace.*.dir -simulator_type view 2>&1 | egrep 'system call |function'
  <Application simple_app (484125) DynamoRIO usage error : invalid system call parameter number>
  ---------------------------------------------------------------------------

  ---------------------------------------------------------------------------
  $ rm -rf drmemtrace.*.dir; bin64/drrun -t drcachesim -offline -record_syscall '1|3&12|4&9|2' -record_syscall '12|2&158|4' -- suite/tests/bin/simple_app && bin64/drrun -t drcachesim -indir drmemtrace.*.dir -simulator_type view 2>&1 | egrep 'system call |function'
  ...
  Hello, world!
  <Stopping application simple_app (484049)>
         32411       26808:      484049 <marker: system call 12>
         32412       26808:      484049 <marker: function==syscall #12>
         32413       26808:      484049 <marker: function argument 0x0>
         32414       26808:      484049 <marker: function argument 0x7ffc87c52d4c>
         32415       26808:      484049 <marker: function==syscall #12>
         32416       26808:      484049 <marker: function return value 0x1>
         50436       41126:      484049 <marker: system call 9>
         50437       41126:      484049 <marker: function==syscall #9>
         50438       41126:      484049 <marker: function argument 0x0>
         50439       41126:      484049 <marker: function argument 0x2000>
         50440       41126:      484049 <marker: function==syscall #9>
         50441       41126:      484049 <marker: function return value 0x1>
         50980       41485:      484049 <marker: system call 21>
         52193       42391:      484049 <marker: system call 257>
         52223       42409:      484049 <marker: system call 262>
         52253       42430:      484049 <marker: system call 9>
         52254       42430:      484049 <marker: function==syscall #9>
         52255       42430:      484049 <marker: function argument 0x0>
         52256       42430:      484049 <marker: function argument 0x1b5c7>
         52257       42430:      484049 <marker: function==syscall #9>
         52258       42430:      484049 <marker: function return value 0x1>
         52274       42439:      484049 <marker: system call 3>
         54224       44056:      484049 <marker: system call 257>
         54252       44074:      484049 <marker: system call 0>
         54329       44130:      484049 <marker: system call 17>
         54455       44208:      484049 <marker: system call 262>
         54988       44591:      484049 <marker: system call 17>
         55653       45105:      484049 <marker: system call 9>
         55654       45105:      484049 <marker: function==syscall #9>
         55655       45105:      484049 <marker: function argument 0x0>
         55656       45105:      484049 <marker: function argument 0x1e1f50>
         55657       45105:      484049 <marker: function==syscall #9>
         55658       45105:      484049 <marker: function return value 0x1>
         ...
  ---------------------------------------------------------------------------
```
Fixes #6344
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant