release.dox - dynamorio - Dynamic Instrumentation Tool Platform

Source path: svn/ trunk/ api/ docs/ release.dox

‹r1363

r1379

/* ******************************************************************************
 * Copyright (c) 2010-2012 Google, Inc.  All rights reserved.
 * Copyright (c) 2011 Massachusetts Institute of Technology  All rights reserved.
 * Copyright (c) 2008-2010 VMware, Inc.  All rights reserved.
 * ******************************************************************************/

/*
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 * 
 * * Redistributions of source code must retain the above copyright notice,
 *   this list of conditions and the following disclaimer.
 * 
 * * Redistributions in binary form must reproduce the above copyright notice,
 *   this list of conditions and the following disclaimer in the documentation
 *   and/or other materials provided with the distribution.
 * 
 * * Neither the name of VMware, Inc. nor the names of its contributors may be
 *   used to endorse or promote products derived from this software without
 *   specific prior written permission.
 * 
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
 * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED. IN NO EVENT SHALL VMWARE, INC. OR CONTRIBUTORS BE LIABLE
 * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
 * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
 * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
 * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
 * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
 * DAMAGE.
 */

/**
***************************************************************************
***************************************************************************
***************************************************************************

\page release_notes Release Notes for Version \DR_VERSION

This section is divided into the following subsections:

 - \ref sec_package
\ifnot vmsafe
 - \ref sec_changes
\endif
 - \ref sec_limits
\ifnot vmsafe 
 - \ref sec_future
\endif

***************************************************************************
\htmlonly
<table width=100% bgcolor="#000000" cellspacing=0 cellpadding=2 border=0>
  <tr><td><table width=100% bgcolor="#0000C0" cellspacing=0 cellpadding=1 border=0>
  <tr><td><table width=100% bgcolor="#0000C0" cellspacing=0 cellpadding=1 border=0>
  <tr><td></td></tr></table></td></tr></table></td></tr></table>
\endhtmlonly
\section sec_package Distribution Contents

The following are part of the \DynamoRIO release distribution:

\if profiling
  FIXME PR 225255: profile library too
\endif

 - Four different \DynamoRIO libraries: debug and release for each of
   32-bit and 64-bit. 
   The debug library enables assertion messages to more easily diagnose
   API usage errors.
 - Four different IA-32/AMD64 decoding static libraries: debug and release
   for each of 32-bit and 64-bit. 
   The debug library enables assertion messages to more easily diagnose
   API usage errors.
 - The \DynamoRIO configuration and execution libraries and command-line
   utilities \c drconfiglib.dll, \c drinjectlib.dll, \c drconfig.exe,
   \c drrun.exe, and \c drinject.exe.
   \if linux Windows package only. \endif
 - A utility \c drview.exe for viewing which processes
   are running under \DynamoRIO control.
   \if linux Windows package only. \endif
\if linux
 - The \c drconfig, \c drrun, and \c drinject scripts for configuring and
   running applications under \DynamoRIO on Linux.  Linux package only.
\endif
 - Header files for the \DynamoRIO APIs.
 - This documentation.
 - \ref API_samples "Sample \clients".
\ifnot vmsafe
 - A graphical statistics viewer \c DRgui.exe that displays internal
   \DynamoRIO statistics as well as custom statistics defined
   by a \client (see \ref sec_ex7). \if linux Windows package only. \endif
   \DynamoRIO exports a large number of statistics in its debug build, and
   a handful in release builds.
\endif
\ifnot vmsafe
 - A binary tracedump reader, which also functions as a sample
   \client using \DynamoRIO as a standalone library (see \ref
   sec_standalone).
\endif

When unpacking the release package on Windows, be aware that the Cygwin
unzip utility does not preserve executable permissions properly.  After
unzipping, add ugo+rx permissions to all of the .dll and .exe files in the
resulting directory tree:
\code
find . -name \*.dll -exec chmod ugo+rx {} \; -o -name \*.exe -exec chmod ugo+rx {} \;
\endcode

\ifnot vmsafe
***************************************************************************
\htmlonly
<table width=100% bgcolor="#000000" cellspacing=0 cellpadding=2 border=0>
  <tr><td><table width=100% bgcolor="#0000C0" cellspacing=0 cellpadding=1 border=0>
  <tr><td><table width=100% bgcolor="#0000C0" cellspacing=0 cellpadding=1 border=0>
  <tr><td></td></tr></table></td></tr></table></td></tr></table>
\endhtmlonly
\section sec_changes Changes Since Prior Releases

The current version is \DR_VERSION.  

The changes between version \DR_VERSION and 3.2.0 are:

 - Added persistent cache support for \clients via
   dr_register_persist_ro(), dr_register_persist_rx(), 
   dr_register_persist_rw(), dr_register_persist_patch(), and
   #DR_EMIT_PERSISTABLE.  However, dr_register_persist_patch() is
   still experimental, is in flux, and is subject to change in
   the next release.
 - Added hashtable persistence support via hashtable_persist_size(),
   hashtable_persist(), and hashtable_resurrect()
 - Added dr_exit_process() to cleanly shut down with a specified
   exit code
 - Added dr_convert_handle_to_pid()
 - Added dr_nudge_client_ex() to enable nudging other processes
   from within a \client
 - Added reg_resize_to_opsz()
 - Added dr_save_arith_flags_to_xax() and dr_restore_arith_flags_from_xax()
 - Added dr_snwprintf() and dr_vsnwprintf()
 - Added drwrap_replace_native(), drwrap_is_replaced(),
   drwrap_is_replaced_native()
 - Added dr_clobber_retaddr_after_read()
 - Deprecated dr_save_arith_flags() and dr_restore_arith_flags()
 - Fixed performance regression involving inlined clean calls
 - Fixed bug involving 32-bit Linux support on a 64-bit Linux
   kernel on an AVX-enabled processor (Issue 754)
 - Fixed bug involving multimedia state on a 32-bit Windows kernel
   (Issue 139)
 - Fixed bugs building and running on Ubuntu 11.10 (Issue 718, Issue 720)
 - Several other bug fixes

The changes between version 3.2.0 and 3.1.0 are:

 - Added support for PECOFF + DWARF2 symbols on Windows in the
   drsyms Extension
 - Added support for building extensions as static libraries (subject to
   licensing constraints) via DR_EXT_{DRWRAP,DRUTIL,DRMGR,DRSYMS}_STATIC
 - Added support for linking C \clients with libc via DynamoRIO_USE_LIBC
 - Added dr_insert_clean_call_ex()
 - Added a no-frills drwrap mode for faster but constrained wrapping
 - Added drwrap_get_drcontext() for performance
 - Added drwrap notification on exceptions bypassing post-hooks
 - Added drwrap_wrap_ex() to pass initial constant user data
 - Added drwrap_is_wrapped() and drwrap_is_post_wrap()
 - Added drwrap_set_global_flags() to control safety of
   application memory accesses
 - Added drwrap_get_mcontext_ex() to avoid the cost of copying
   multimedia register values when not necessary
 - Added drwrap interface for caching post-call addresses
 - Added drmgr_decode_sysnum_from_wrapper()
 - Added drutil_expand_rep_string_ex() that returns additional
   information about string loop expansion
 - Added improved instrlist disassembly that includes labels and
   instruction targets
 - Added instr_compute_address_ex_pos() for instrs with multiple memory
   operands.
 - Added dr_get_client_base()
 - Added dr_vsnprintf()
 - Added service pack version to dr_get_os_version()
 - Added mediation of note fields to drmgr
 - Added custom storage in label instructions via instr_get_label_data_area()
 - Added support for multiple non-meta control transfer instructions
   with intra-block targets in one basic block to drmgr
 - Added user data passing support among all four passes of drmgr
 - Several bug fixes

Version 3.0.0 was a development version.  3.1.0 is the first official
released version of the 3.x series.

The changes between version 3.0.0 and 2.2.0 include the following major
features:

 - Added a private loader on Linux for better support for C++ \clients and
   third-party library usage by \clients
 - Added Linux support for \p drsyms to enable symbol processing in 
   Linux \clients
 - Added \p drutil Extension which provides memory address retrieval
   and string loop expansion (note: LGPL license)
 - Added a static library for decoding and encoding

\b IMPORTANT: The 3.0.0 and onward \client API is mostly backward
compatible with releases from 1.0.0 (originally called 0.9.6: see below)
onward with the exception of functions that involve dr_mcontext_t and
several other source and binary compatibility changes since version 2.2.0
described below.  The dr_mcontext_t struct and all structs that contain it
have changed and are NOT backward compatible with releases prior to 3.0.0.

A sample script for updating \client sources to the 3.0.0 API's version of
dr_mcontext_t and related functions is as follows:

\code
perl -pi -e '\
s|dr_mcontext_t (\w+);|dr_mcontext_t \1 = {sizeof(\1),DR_MC_ALL,};|;\
s|(dr_[gs]et_mcontext\(\S+,\s+\S+),\s+[^\s\)]+\)|\1)|;\
s|(dr_redirect_execution\(\S+),\s+\S+\)|\1)|;\
s|^\s+int app_errno;\s*\n||ms;\
s|raw_mcontext\.|raw_mcontext->|g;\
s|info->mcontext\.|info->mcontext->|g;\
s|excpt->mcontext\.|excpt->mcontext->|g;' *.c
\endcode

The script makes 3 main changes.  First, any dr_mcontext_t allocated by the
\client must have its \p size and \p flags fields set.  Second, the \p app_errno
parameter was removed from several functions; it required a local variable,
so any local named \p app_errno is removed.  Third, the dr_mcontext_t
fields in the fault, signal, and exception structs all became
pointers.

This script may not catch all cases.  Use your version control system to
look at the diff after applying it to ensure it did not change anything it
shouldn't have.  Run with debug build to catch other instances where
dr_mcontext_t.size is not set.  Also note that using the dr_mcontext_t
initialization syntax in the script will end up calling memset; for
performance-critical situations, instead initialize only the size and flags
fields separately.  Also note that if the xmm or ymm registers are not
needed, asking for DR_MC_CONTROL and/or DR_MC_INTEGER is more performant
than DR_MC_ALL.

\b IMPORTANT: Further changes between version 3.0.0 and 2.2.0 include the following that
affect source and/or binary compatibilty:

 - Changed the #dr_mcontext_t structure field layout.  This is a binary
   compatibility change with respect to versions prior to 3.0.0.
 - Added a dr_mcontext_t.size field which must be set by the \client prior
   to calling dr_get_mcontext(), dr_set_mcontext(), or
   dr_redirect_execution.  This is a source compatibility change with
   respect to versions prior to 3.0.0.
 - Added a dr_mcontext_t.flags field which must be set by the \client prior
   to calling dr_get_mcontext(), dr_set_mcontext(), or
   dr_redirect_execution.  This is a source compatibility change with
   respect to versions prior to 3.0.0.
 - Removed the app_errno parameter from dr_get_mcontext(),
   dr_set_mcontext(), and dr_redirect_execution().  This is a source
   compatibility change with respect to versions prior to 3.0.0.
 - Changed all dr_mcontext_t fields in the dr_restore_state_info_t,
   dr_exception_t, and dr_siginfo_t structs to be pointers.  This is a
   source compatibility change with respect to versions prior to 3.0.0.
 - Changed the bool typedef from int to char for C++ compatibility.
   This is a binary compatibility change with respect to versions
   prior to 3.0.0.
 - Changed the signature of drwrap_unwrap(), in order to allow one of the
   pre or post hooks to be optional (Issue 562).  This is a source
   compatibility change with respect to versions prior to 3.0.0.
 - Moved console printing support from the drsyms Extension to core DR.  The
   drsym_write_to_console() and drsym_using_console() are no longer
   supported.  Instead, call dr_enable_console_printing() in dr_init(),
   which then enables dr_printf() and dr_fprintf() to print to the console
   (with some limitations: see dr_enable_console_printing() documentation).
   This is a source compatibility change with respect to versions
   prior to 3.0.0.
 - Added a \p flags argument to most of the routines in the \p drsyms extension
   to control demangling, and added drsym_demangle_symbol().  This is a source
   compatibility change with respect to versions prior to 3.0.0.
 - Added drsym_get_module_debug_kind() and a \p debug_kind field to the \p
   drsym_info_t struct written by drsym_lookup_address().  These additions allow
   drsyms users to determine what kind of debug info is available for a module.
   The \p debug_kind field creates a binary compatibility change for users of \p
   drsym_info_t with respect to versions prior to 3.0.0.

Additional changes between version 3.0.0 and 2.2.0 include the following:

 - Added \p drvector to drcontainers Extension: simple resizable vector
 - Added a windbg script for auto-locating libraries for easier
   debugging
 - Added dr_mutex_self_owns() and recursive lock support (dr_recurlock_*)
   (Issue 219)
 - Added dr_map_file(), dr_unmap_file(), and dr_file_size() (Issue 542)
 - Added dr_rename_file() and dr_delete_file().
 - Added routines to disassemble to a buffer rather than a file (Issue 524)
 - Added support for the AVX and FMA ISA extensions
 - Added dr_insert_get_seg_base()
 - Added return value to dr_redirect_execution() and dr_set_mcontext()
 - Increased maximum option string from 512 to 2048 (Issue 363)
 - Increased default stack size from 20KB to 56KB to make it easier to use
   C++ and external libraries with larger stack usage
 - Added dr_get_os_version() (Issue 304)
 - Deprecated the "meta-instruction that can fault" property and
   instr_is_meta_may_fault(), instr_set_meta_may_fault(),
   instrlist_meta_fault_preinsert(), instrlist_meta_fault_postinsert(),
   and instrlist_meta_fault_append().
 - Added dr_using_app_state()
 - Added instr_encode_to_copy() and instrlist_encode_to_copy()
 - Added disassemble_set_syntax() for -syntax_intel control without
   runtime options and proc_set_vendor() to control vendor-specific
   ISA details when decoding or encoding
 - Added instrlist_set_fall_through_target() and
   instrlist_set_return_target()
 - Added hashtable_clear() to the drcontainers Extension
 - Several bug fixes

The changes between version 2.2.0 and 2.1.0 are:

 - Added \p drwrap Extension which provides function wrapping
   and replacing (note: LGPL license)
 - Added \p drmgr Extension: the \DynamoRIO Multi-Instrumentation Manager
   Extension, a mediator for combining and coordinating multiple
   instrumentation passes
 - Added read-write locks (Issue 406)
 - Added isolation of \client-opened files from the application (Issue 357)
 - Added dr_mcontext_t.xip for syscall events (Issue 442)
 - Several bug fixes

The changes between version 2.1.0 and 2.0.0 are:

 - Added Windows 7 support
 - Added clean call sequence optimization and auto-inlining.
 - Added Windows child process following support for \clients: -follow_children
   is now on by default for both Windows and Linux.
 - Added DR_TRY_EXCEPT() (Issue 51)
 - Added dynamic \client auxiliary library loading support
   via dr_load_aux_library(), dr_lookup_aux_library_routine(), and
   dr_unload_aux_library()
 - Added dr_switch_to_app_state() and dr_switch_to_dr_state()
 - Added dr_client_thread_set_suspendable()
 - Added dr_get_random_value(), dr_set_random_seed(), and dr_get_random_seed()
 - Added dr_file_exists() and dr_directory_exists() for Linux
 - Added support for dr_get_mcontext() from secondary thread init events,
   and changed its return type to bool
 - Added dynamic hashtable resizing to the drcontainers hashtable
 - Added dr_app_pc_from_cache_pc()
 - Added a segment list to module_data_t for Linux and internal
   support for non-contiguously-mapped modules (Issue 160)
 - Added PEB isolation (Issue 249) and dr_get_app_PEB()
 - Added drsym_enumerate_symbols() to the \p drsyms Extension
 - Added limited support for printing to the cmd window (Issue 261) via the
   \p drsyms Extension: drsym_write_to_console() and drsym_using_console() 
 - Renamed the REG_ constants to DR_REG_ to avoid conflicts with system
   headers (Issue 34).  \Clients should set(DynamoRIO_REG_COMPATIBILITY ON)
   prior to configure_DynamoRIO_client() to use the old constants and avoid
   any source changes; this will happen automatically if the \client
   targets version 2.0 or earlier.  Binary compatibility is unaffected.
 - Deprecated dr_request_synchronized_exit() and replaced it with
   dr_set_process_exit_behavior().  Now a full thread synch is performed
   at exit time in release build if a process exit event or thread exit
   event is registered.  dr_set_process_exit_behavior() can provide
   more performant exit performance for \clients that have flexible
   exit event requirements.
 - Switched debug build to also be an INTERNAL=ON build
 - Fixed bug in handling single-byte-bb selfmod code
 - Fixed bugs in handling alarm signals
 - Fixed 64-bit Windows stack alignment bug (Issue 331)
 - Fixed handling of "data32 rex.w call"
 - Fixed Issue 320: a problem with thread-private cache resizing
 - Fixed Issue 319: decode movlhps and movhlps properly
 - Fixed Issue 139: add xmm0-7 preservation for 32-bit Linux applications, 
   which may have noticeable impacts on \clients calling clean calls:
   e.g., pushing bbs over the max size limit or having a noticeable
   performance hit.
 - Support building sources using Visual Studio

In version 2.0.0, the configuration and deployment API and tools changed and
are not backward compatible with earlier versions: see below for details.
The changes between version 2.0.0 and 1.5.0 are:

 - Changed the configuration and deployment model for both Linux and
   Windows to use a configuration file based approach on both platforms,
   which adds control over child processes on Linux and supports local
   configuration on Windows for un-privileged and parallel execution
   (Issue 265).  The registry is no longer used for individual application
   configuration on Windows other than to point at the location for
   global configuration files, when used.<br>
   \b IMPORTANT: On Windows the following non-backward-compatible changes
   have been made:
   - drdeploy.exe no longer exists (replaced by drconfig.exe and drrun.exe)
   - drconfig.dll is now drconfiglib.dll
   - drconfiglib.dll's API routines now take in a process id to support
     one-time targeted-process configuration (to support parallel execution)
   - configuration is either per-user or global, with per-user taking
     precedence when both exist
   - configuration does NOT enable systemwide injection by default:
     use the -syswide_on parameter to drconfig or drrun for that
     (it requires administrative privileges)
   .
   \b IMPORTANT: On Linux, if you're using custom injection via
   raw environment variables rather than using the \p drdeploy script,
   you must also set DYNAMORIO_RUNUNDER to 1 for injection to work with
   this release.
 - Added drinjectlib.dll and dr_inject.h, making custom injection tools
   easier to build (Issue 246)
 - Added \DynamoRIO Extension support for auxiliary libraries that extend the
   \DynamoRIO API (Issue 277)
 - Added symbol lookup support via Extension (Windows only for now) (Issue 44)
 - Added a "drcontainers" Extension that initially contains a hashtable
 - Added thread creation support: dr_create_client_thread() (Issue 41)
 - Added dr_sleep()
 - Added dr_set_itimer() and dr_get_itimer() (Linux-only) (Issue 283)
 - Added dr_app_pc_for_decoding()
 - Added -synch_at_exit option and dr_request_synchronized_exit() to
   provide guarantees over thread exit event timing in release build
 - Added instr_cmovcc_triggered() and instr_cmovcc_to_jcc()
 - Renamed OP_fcmovene to OP_fcmovne
 - Implemented instr_invert_cbr() for OP_jcc_short
 - Added the full path to modules in module_data_t
 - Added dr_get_proc_address_ex() to support indirect code objects
 - Added dr_get_milliseconds() and dr_get_time() impl for Linux
 - Added instr_is_undefined()

The changes between version 1.5.0 and 1.4.0 are:

 - Added a private loader on Windows for better support for library usage
   by \clients
 - Added nudge support on Linux
 - Added dr_suspend_all_other_threads() and dr_resume_all_other_threads()
 - Made it easier for \clients to use faults to push rare events out of
   instrumentation paths:
   - Added access to the pre-translated context and the code fragment
     information for both 
     dr_register_signal_event() and dr_register_exception_event()
     This changed the return type for exception event callbacks.
   - Added a signal/exception event on a fault in non-code-cache
     \DynamoRIO code, such as \client-generated code.
   - Added the "meta-instruction that can fault" property via
     instr_is_meta_may_fault(), instr_set_meta_may_fault(),
     instrlist_meta_fault_preinsert(), instrlist_meta_fault_postinsert(),
     and instrlist_meta_fault_append().
   - Added a new event dr_register_restore_state_ex_event() that provides
     the pre-translated context and code fragment information, and allows
     for translation failure for non-fault translations.
 - Added dr_dup_file_handle()
 - Added dr_memory_is_dr_internal() and dr_memory_is_in_client()
 - Added dr_get_parent_id()
 - Added decode_opcode_name()
 - Removed the deprecated snprintf() as it causes symbol pre-emption
   problems on Linux.  Older \clients should switch to dr_snprintf().
 - Fixed bug in cross-architecture execve (Issue 146)
 - Clone record is now passed via dstack instead of ebp (Issue 149)
 - Fixed close() syscall handling, !HAVE_TLS assert & minor issues (Issue 151)

The changes between version 1.4.0 and 1.3.2 are:

 - Added directly-addressable thread-local storage slots for exclusive
   \client use:
   - dr_raw_tls_calloc()
   - dr_raw_tls_free()
 - Provide 64-bit versions of the drdeploy.exe and drview.exe tools
 - Provide dr_get_proc_address() on Linux
 - Added dr_query_memory_ex() to allow address space walking on Linux
 - Added -msgbox_mask on Linux: waits for a keypress
   - Added STDIN and dr_get_stdin_file()
 - Added shared library versioning on Linux
 - Support calling dr_get_mcontext() from bb and trace callbacks
 - Provide support for building \clients using CMake (see cmake/*, and for
   an example of usage see samples/CMakeLists.txt)
 - Provide support for \clients to use -fvisibility by setting
   the define USE_VISIBILITY_ATTRIBUTES for dr_defines.h
 - Added instr_compute_address_ex() for instrs with multiple memory operands
 - Provide dr_snprintf() (identical to snprintf()) for consistent naming
   and to avoid gcc warnings about using pointers with %x (which we're
   using because there is no portable way to precisely control %p)
 - The statistics viewer \c DRgui.exe is no longer supported on Windows
   NT.  Statistics still work, but the graphical application itself will
   not run on NT.
 - Changed the top-level registry key to "\DynamoRIO"
 - Re-arranged layout of bin and samples directories
 - Symbols for all binaries are now included

The changes between version 1.3.2 and 1.3.1 are:

 - Added support for Linux execve of cross-architectural executables
   (e.g., 32-bit process performing execve of 64-bit executable)
   - Also, libdrpreload.so is now kept in the same libXX/{debug,release} 
     directory as libdynamorio.so
 - instr_convert_short_meta_jmp_to_long() now returns the longer version of
   the taken jump, to use when setting the target of a jecxz or loop*
   branch.
 - Various bug fixes including in these areas:
   - dr_syscall_set_result() and dr_syscall_invoke_another()
   - 64-bit drinject stack alignment
   - 64-bit erroneous assert in dr_get_process_id()
   - 64-bit dr_file_{tell,seek} worked but returned failure
   - -opt_memory bugs resulting in asserts
   - sigprocmask() corner case bug
   - signal handler sharing for NPTL threads
   - decoding across page boundaries on Linux

Version 1.3.1 is identical to 1.3.0 but is under a BSD license (see \ref
page_license).

We re-numbered the previous \DynamoRIO versions as follows:

 - 1.0.0 = 0.9.6 build 9600
 - 1.1.0 = 0.9.6 build 9601
 - 1.2.0 = 0.9.6 build 9602

The changes between version 1.3.0 and version 1.2.0 (0.9.6 9602) are:

 - Version numbering shift to 1.x.y instead of 0.9.6 960x
 - New system call pre, post, and filter events, and new system call
   parameter and result access, along with a feature to chain system calls:
   - dr_register_pre_syscall_event()
   - dr_register_post_syscall_event()
   - dr_register_filter_syscall_event()
   - dr_syscall_get_param()
   - dr_syscall_set_param()
   - dr_syscall_set_sysnum()
   - dr_syscall_get_result()
   - dr_syscall_set_result()
   - dr_syscall_invoke_another()
   - dr_is_wow64()
 - New signal event for Linux
   - dr_register_signal_event()
 - New option \ref op_pause "-pause_on_error", and error messages to
   stdout by default for release builds, to improve Linux debugging
 - New routines for memory allocation and memory manipulation:
   - dr_nonheap_alloc()
   - dr_nonheap_free()
   - dr_memory_protect()
 - New option \ref op_syntax_intel "-syntax_intel" for Intel-style disassembly
 - New option \ref op_sysenter "-sysenter_is_int80"
 - The parameter to an application's system call (normally kept in the eax
   register) can now be freely changed in basic blocks on all platforms
 - Added support for 64-bit -thread_private
 - Added corner-case undocumented IA-32 opcode support
 - Fixed bug running multi-threaded 64-bit Linux apps
 - Fixed bugs in 64-bit Linux signal handling
 - Fixed bug running -thread_private debug build
 - Fixed bug running 32-bit apps on 64-bit Linux on AMD processors
 - Fixed bug where OS_OPEN_APPEND overwrote instead of appending on Windows

The changes between the 0.9.6 release builds 9602 and 9601 are:

 - Performance improvements for both the base \DynamoRIO system and for
   \client instrumentation when running on Pentium M, Core, and Core 2
   processors.
 - 64-bit bug fixes
 - Added several convenience routines:
   - get_register_name() 
   - reg_to_pointer_sized()
   - reg_is_gpr()
   - reg_is_segment()
   - reg_32_to_8()
 - Disassembly now expands immed sizes to match operands
 - Fixed bug in instr_is_nop()

The changes between the 0.9.6 release builds 9601 and 9600 are:

 - The Windows registry key used is now "VMware, Inc." instead of "VMware"
 - Added large file support (see #DR_FILE_ALLOW_LARGE)
 - Added support for decoding from a copy of code: decode_from_copy() and
   disassemble_from_copy().
 - Changed the default options to favor performance, and added the
   \ref op_memory "-opt_memory" runtime option to prioritize memory instead.

Release 0.9.6 is \b not backward compatible with prior releases 0.9.1-0.9.5.

The major changes between the 0.9.6 and 0.9.5 releases include 64-bit
support, multiple \clients, state translation, trace contents, and Linux
module events and fast system calls:

 - 64-bit applications and \clients are now supported.  This changed
   several function signatures: 
   - instr_encode()
   - decode_next_pc()
   - decode_sizeof()
   - decode_eflags_usage()
   - instr_init()
   - The binary trace dump format changed.
   .
   Several new functions were added:
   - set_x86_mode()
   - get_x86_mode()
   - instr_set_x86_mode()
   - instr_get_x86_mode()
   - opnd_create_rel_addr()
   - opnd_create_far_rel_addr()
   - opnd_is_rel_addr()
   - opnd_is_near_rel_addr()
   - opnd_is_far_rel_addr()
   - instr_has_rel_addr_reference()
   - instr_get_rel_addr_target()
   - instr_get_rel_addr_dst_idx()
   - instr_get_rel_addr_src_idx()
   - instr_shrink_to_32_bits()
   - opnd_shrink_to_32_bits()
   - reg_32_to_64()
   - reg_64_to_32()
   - reg_is_extended()
   - reg_parameter_num()
   .
   To build a 64-bit \client, set the \p X86_64 preprocessor define before
   including the \DynamoRIO header files, and link with the 64-bit build of
   \DynamoRIO (for a 32-bit \client, set \p X86_32).
 - Multiple \clients are now supported.  This changed the signatures of
   dr_init(), dr_get_options(), and dr_get_client_path().  It also changed
   how \clients are deployed and nudged, and how events are unregistered:
   explicit unregistration routines are now used.
 - State translation in the presence of \clients is now fully supported.
   This changed the signature for the basic block and trace event callbacks
   (see dr_register_bb_event() and dr_register_trace_event()), added a
   new event dr_register_restore_state_event(), and added new functions
   instr_set_translation(), instr_set_meta_no_translation(), and INSTR_XL8().
 - The trace callback (#dr_register_trace_event()) now presents original
   application code to the \client, rather than code that has already
   been modified for execution in the code cache.  The \client also has
   flexibility in which instrumentation is included from constituent
   basic blocks added to a trace (the \p for_trace parameter: see
   #dr_register_bb_event()).
 - Fast system calls (syscall and sysenter) are now supported on Linux.
 - Module load/unload events and module iteration are now supported on Linux.
 - System calls for 32-bit applications on 64-bit kernels are no longer
   hidden by vsyscall; related functions were removed:
   instr_is_lol64_syscall(), instr_is_32on64_syscall().
 - Due to both 64-bit support and full WOW64 (32-bit applications on 64-bit
   Windows) support, xmm registers were added to dr_mcontext_t, and a
   new function dr_mcontext_xmm_fields_valid() was added.
 - Far instr-type operands are now supported: opnd_create_far_instr(),
   opnd_is_near_instr(), opnd_is_far_instr().
 - Miscellaneous new functions were added:
   - instr_convert_short_meta_jmp_to_long()
   - instr_reads_from_reg()
   - LOCK()
   - OPND_CREATE_INT_32OR8()
   - OPND_CREATE_INT_16OR8()
   - instrlist_meta_append()
   - dr_using_all_private_caches()
 - The type of nudge arguments was changed from (void *) to uint64.
 - The signature of dr_lookup_module() changed.  It no longer has an
   IMAGE_SECTION_HEADER out argument.  See dr_lookup_module_section()
   for that functionality.
 - The disassemble-from-address routines now return NULL when pointed at
   invalid instructions (matching the decode routines).
 - The routines to access \DynamoRIO tls slots from the cache were changed.
   dr_insert_write_temp_tls was eliminated in favor of a generalized #dr_save_reg
   with more flexibility on which slot can be used.  #dr_save_arith_flags was
   similarly generalized.  Slots are now guaranteed to remain valid until the
   next non-meta instruction allowing access to saved registers during clean
   calls via #dr_read_saved_reg and #dr_write_saved_reg.  #dr_insert_mbr_instrumentation
   also now requires caller to specify the spill slot to be clobbered
   which must be less than dr_max_opnd_accessible_spill_slot().

The major changes between the 0.9.5 and 0.9.4 releases are:

 - The basic block hook (\ref sec_events_bb) passes completely unmodified
   application code to the \client (no mangling or elision).
 - The old \client hook exports have been replaced with an explicit event
   registration model.
 - Instruction representation simplification: the \client only sees fully
   decoded instructions.
 - Easier-to-use clean calls (see #dr_insert_clean_call).
 - Library support (-wrap on linux, ntdll on windows: see \ref sec_extlibs
   and \ref sec_utils).
 - Some features have been removed (these are easily implemented by a
   \client): there is no more edge-counting profile build, no more
   custom exit stubs, and no more prefixes.
 - Infrastructure improvements:
   - Thread-shared caches (can still request thread-private: \ref
     op_thread_priv "-thread_private option").  Note that there are
     some subtle changes stemming from using thread-shared: in particular,
     note that the context passed to the deletion event may be NULL
     (see #dr_register_delete_event).
   - Direct access to TLS slots (#dr_save_reg, dr_insert_write_temp_tls,
     #dr_insert_write_tls_field).
 - Module events (#dr_register_module_load_event),
   module iteration (#dr_module_iterator_start, #dr_lookup_module, etc.),
   and memory querying (#dr_query_memory, #dr_virtual_query).
 - The full API is now documented in html and pdf for easy browsing.
 - Numerous type and routine name changes.

\endif

***************************************************************************
\if future
FIXME not done yet

  - see \clients/mit_release/using.html: what's included, etc.
  - DRcontrol: specify target app and runtime options (TODO: case 10647:
                                                       external vs internal ops) 
  - GUI: just stats viewer, or control too?
  - DRview
  - native_exec (TODO: specify on module load; TODO: case 10846 eliding problems)
  - multiple \clients (TODO: multiple non-mutating, one mutating RIO \client;
                      multiple probe \clients w/ chaining; one prog shep \client?)
  - detach (TODO: ask each \client to clean up)
  - updating: load new library  

Options for communication in and out of the process are discussed in \ref
sec_comm.
\endif


***************************************************************************
\htmlonly
<table width=100% bgcolor="#000000" cellspacing=0 cellpadding=2 border=0>
  <tr><td><table width=100% bgcolor="#0000C0" cellspacing=0 cellpadding=1 border=0>
  <tr><td><table width=100% bgcolor="#0000C0" cellspacing=0 cellpadding=1 border=0>
  <tr><td></td></tr></table></td></tr></table></td></tr></table>
\endhtmlonly
\section sec_limits Limitations

\subsection sec_limit_clients \Client Limitations
The most important limitation on a \client is that it remain transparent.
This is described fully in \ref transparency.
Here we summarize the key points to remain transparent:

  - For full transparency, the \client should be a self-contained library
with linkage to nothing other than \DynamoRIO libraries.  We
provide private loading that makes some use of system libraries safe, but
global resource conflicts can still occur and \clients are cautioned from
using system-interacting library routines.
See \ref sec_extlibs for further details.
  - Currently, the communication API provided by \DynamoRIO is limited to file
I/O and nudges.
\if vmsafe
  - The security agent on the security monitor VM has no direct
    relationship with \DynamoRIO.  However, the VMsafe solution provider
    can architect their own communication channel between the security
    agent on the security monitor VM and the \client by using an agent on
    the \client's VM as an intermediary.
 \anchor limits_winlogon
  - There is a known false positive in the anti-piracy DRM code in the
    Windows Genuine Advantage component of winlogon.exe.  This code
    performs deliberate obfuscation and violations of standard coding
    behavior.
\endif
\if vmsafe
  - The \client can run in only one mode at a time, which cannot be changed during the
execution of the process.  Mode change can be done only on a subsequent
application restart.
  - The \client should call only the API corresponding to the mode it was started
in.  Mixing API from different modes can lead to unspecified results and is
not supported.
  - Probe API allows only one registration of probes, which must occur in
    dr_init().  
  - Probes can be specified only with library names and offsets from the base
    of the library in memory (\ref DR_PROBE_ADDR_LIB_OFFS).
  - When a probe callback finishes execution it can't request \DynamoRIO to
    perform any actions like logging, control flow changes, etc.
  - Probe locations can't include system call wrappers in ntdll.dll, i.e.,
    ntdll!Nt* routines can't be probe locations.
\endif

\subsection sec_limit_platforms Platform Limitations
  - \DynamoRIO currently supports the following NT-based 32-bit
    Windows systems : NT (all service packs), 2000 (all service
    packs), XP (32-bit, service packs 0-3), 2003 (32-bit, service
    packs 0-2), Vista (32-bit, service packs 0-1), and Windows 7 (32-bit,
    service packs 0-1). It does not support Windows 95, 98, or ME.
    Windows Server 2008 is likely to work but has not been tested.
  - This release of \DynamoRIO has limited support for running 32-bit
    Windows applications on the following 64-bit Windows operating
    systems : Windows XP Professional x64 (service pack 2), Windows 2003
    x64 (service pack 2), Vista x64 (service packs 0-1), and Windows 7
    (service packs 0-1).  Windows Server 2008 is likely to work but has not
    been tested.  Only the 32-bit
    code will be seen, and child processes created will not be injected
    into.  On 64-bit Windows 32-bit applications are automatically run
    through the Windows-On-Windows or WOW64 emulator so system call and
    indirect call processing \clients must be aware of
    #instr_is_wow64_syscall().
\anchor limits_64bit
  - This release of \DynamoRIO supports running
    64-bit Windows applications, using the 64-bit \DynamoRIO build, on
    the following 64-bit Windows systems: Windows XP Professional x64
    (service pack 2), Windows 2003
    x64 (service pack 2), Vista x64 (service packs 0-1), and Windows 7 x64
    (service packs 0-1). Windows Server 2008 is likely to work but has not
    been tested.
  - \DynamoRIO does not support any Itanium based Windows systems.
  - This release does not fully support applications that mix 32-bit and
    64-bit code.  Future releases will support such mixtures.
  - When running a cygwin application under control of \DynamoRIO,
    stderr and stdout output from \DynamoRIO or its \clients may not
    be visible.
  - The statistics viewer \c DRgui.exe is no longer supported on Windows
    NT.  Statistics still work, but the graphical application itself will
    not run on NT.
\if linux
  - This release of \DynamoRIO has support for most 32-bit and
    64-bit Linux
    distributions running on Intel-compatible hardware, including
    Ubuntu and Fedora.
  - This release of \DynamoRIO has support for running 32-bit
    Linux applications on 64-bit Linux operating systems on
    AMD64-compatible hardware.
  - Cross-architecture execve (e.g., a 32-bit process performing execve of
    a 64-bit executable) may stop working if the paths to the
    libdynamrio.so and libdrpreload.so libraries are renamed.
\endif

\subsection sec_limit_perf Performance Limitations

  - In order to present a more straightforward code stream to \clients,
    \DynamoRIO has several optimizations disabled in this release.
    System-call-heavy applications are the ones most likely to be affected.
    Future releases may allow \clients to choose performance versus
    visibility.  This release does provide the \ref op_memory "-opt_memory option"
    to enable prioritizing memory usage.
  - The performance \if vmsafe of Code Manipulation Mode and Memory 
    Firewall Mode \endif when starting up large desktop applications
    may be noticeably worse than native.  Upcoming releases will
    address this issue. \if vmsafe Note that Probe Mode incurs
    significantly less overhead than the other modes. \endif
  - The performance \if vmsafe of Code Manipulation Mode \endif when
    running  Java, .Net, Flash or similar managed execution
    applications can be noticeably worse then native.  This can
    include applications that load these components as in-process
    plugins (such as Internet Explorer). Upcoming releases will
    address this issue.  \if vmsafe Probe Mode and Memory Firewall
    Mode are not affected. \endif
  - When using \if vmsafe Code Manipulation Mode or Memory Firewall
    Mode \else \DynamoRIO \endif on all or many processes on a system
    simultaneously, memory usage may become a factor.  Upcoming
    releases will address this issue.  \if vmsafe Note that Probe Mode
    incurs significantly less overhead than the other modes. \endif

\subsection sec_limit_deploy Deployment Limitations

  - The dr_config.lib library is not multi-thread safe.  Users of the
    library should ensure that no more then one thread accesses the
    library at a time.
  - Other installed software that uses hooks may not always be
    interoperable with \DynamoRIO.
  - Other installed software may conflict with \DynamoRIO's use of the
    \c \\HKLM\\SOFTWARE\\Microsoft\\Windows\\WindowsNT\\CurrentVersion\\AppInit_DLLs
    registry key (only very rarely an issue).
  - On Windows 7 and Windows Server 2008 R2, the digital signature
    requirement for AppInit_DLLs libraries is disabled when systemwide
    injection is requested.  This can be a security concern.    
  - Cygwin processes may not work with \DynamoRIO due to cygwin's
    implementation of fork being incompatible with \DynamoRIO.
  - Though \DynamoRIO supports running on Windows NT, the provided
    installer will only run on Windows 2000 and higher.  After
    installation on a 2000 or higher machine the \ref sec_package may
    be copied to the NT machine for use there.
  - On Windows NT, a reboot is required after the initial
    dr_register_process() in order for \DynamoRIO to take control of any
    applications.  The reboot will not change or undo the registration.
  - A Windows application that does not statically link with
    user32.dll will not be run under control of \DynamoRIO unless its
    parent process (typically cmd.exe or explorer.exe, for manually
    launched applications) is already under \DynamoRIO control
    \if vmsafe (the parent can be in any mode)\endif or the drinject.exe
    utility is used to launch the application. Only some small
    non-graphical applications do not link with user32.dll.
  - When invoking an application in any way that is not from a parent
    process under \DynamoRIO control, \DynamoRIO takes control a little
    later and in some rare cases the application has already created a new
    thread.  This is in violation of the Windows specficiations, but cygwin
    processes do this.  This new thread will not be under \DynamoRIO
    control and can cause a variety of problems.  In such cases it is best
    to invoke from a parent process under \DynamoRIO control.  However, for
    32-bit applications on 64-bit Windows operating systems, \DynamoRIO
    takes over later even from the parent.  Future releases will address
    this limitation.
  - This release does not support running some Windows services under
    control of \DynamoRIO: System, smss.exe, csrss.exe, and protected
    processes on Windows Vista.
  - This release does not support nudging 64-bit processes.
\anchor limits_vista_service_messagebox
  - On Windows Vista most services are run in a separate session as a
    security feature.  This means that neither \DynamoRIO nor its
    \client will be able to display a messagebox when running in said
    services (they have no associated visible window station). See
    dr_messagebox().
  - On Windows Vista \if vmsafe in Code Manipulation Mode \endif the code from
    certain dlls (found mostly in services) and the code subsequently
    called from those dlls is run natively and is not visible to the
    instrumentation APIs.  This only applies to dlls that have a .pexe
    section (only 13 dlls have such a section in a standard Vista
    install) which seems to be associated with a likely obfuscation
    method involving kernel side components that this release has no
    other workaround for.
\if linux
\anchor limits_linux_preload
  - On Linux, the only provided deployment method in this release (aside
    from the \ref sec_startstop "app_start() app_stop()" interface, which
    requires soure code modification) is LD_PRELOAD.  This means that
    static binaries cannot be injected into.  Also, binaries with the suid
    or sgid permission bits set
    disallow absolute paths in LD_PRELOAD and ignore LD_LIBRARY_PATH, so
    alternate methods of specifying the path are needed there.  One method
    is to place libdrpreload.so's full path in /etc/ld.so.preload and copy
    libdynamorio.so to /usr/lib or some other system search directory.
    You'll need to use a \ref lin_deploy "separate configure step approach".
    Don't forget to run drconfig to create a configuration for the
    application first; otherwise, libdrpreload.so will refuse to take
    control.
\endif

\ifnot vmsafe 
***************************************************************************
\htmlonly
<table width=100% bgcolor="#000000" cellspacing=0 cellpadding=2 border=0>
  <tr><td><table width=100% bgcolor="#0000C0" cellspacing=0 cellpadding=1 border=0>
  <tr><td><table width=100% bgcolor="#0000C0" cellspacing=0 cellpadding=1 border=0>
  <tr><td></td></tr></table></td></tr></table></td></tr></table>
\endhtmlonly
\section sec_future Plans for Future Releases

We hope to include the following major features in future releases:

 - Libraries to facilitate building tools that use shadow memory, examine
   system calls, and insert heavyweight instrumentation
 - Earliest Windows injection.  Today drinject injects fairly late; from a
   parent process, injection is very early (before kernel32.dll is loaded),
   but we plan to provide injection at the very first user-mode instruction
   in the future.
 - More flexible (earlier, or later via attach) Linux injection
 - Persistent and process-shared code caches
 - Full control over trace building

To discuss current and future features, join the <a
href="http://groups.google.com/group/dynamorio-users/">\DynamoRIO Users
group</a>.

\endif

*/

Show details Hide details

Change log

r1379 by bruen...@google.com on May 23 (41 hours ago) Diff

fixes  issue 782 
+ add dr_clobber_retaddr_after_read() to
allow client to write
  to retaddr slot on stack after the ret
instruction reads it
+ implementation uses a combination of a
flag and a label to avoid
  interfering w/ client's note while also
avoiding a label search
+ added a test that also serves as a core
test of iret and
  far ret

Go to:

Project members, sign in to write a code review

Older revisions

r1363 by derek.bruening on May 3, 2012 Diff

fixes  issue 760 
+ add drwrap_replace_native() which
uses a meta call to
  invoke a replacement routine that
uses the app stack
...

r1360 by bruen...@google.com on May 2, 2012 Diff

export dr_snwprintf() and
dr_vsnwprintf()

r1353 by bruen...@google.com on Apr 30, 2012 Diff

update changelist for 3.3 release and
mark dr_register_persist_patch() as
experimental

All revisions of this file

File info

Size: 46295 bytes, 930 lines

View raw file