Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

document -prof_pcs and provide support for profiling joint DR+client #140

Open
derekbruening opened this issue Nov 27, 2014 · 9 comments
Open

Comments

@derekbruening
Copy link
Contributor

derekbruening commented Nov 27, 2014

From derek.br...@gmail.com on May 06, 2009 11:27:32

The older MIT releases documented -prof_pcs and provided other profiling
options. We took out the latter as they weren't worth the effort to support
and could be written as simple clients. It might be worth officially
documenting -prof_pcs, however, and provide support/tools for analyzing
performance of clients.

xref PR 225255: release PROFILE_LINKCOUNT build?
xref PR 225139: expose performance-affecting configuration runtime options

Original issue: http://code.google.com/p/dynamorio/issues/detail?id=140

@derekbruening
Copy link
Contributor Author

From bruen...@google.com on September 28, 2011 08:16:33

Labels: GoodContrib

@derekbruening
Copy link
Contributor Author

From bruen...@google.com on September 28, 2011 08:22:33

Owner: ---

@derekbruening
Copy link
Contributor Author

From zhao...@google.com on May 03, 2013 11:59:57

we could also export API to update dcontext->whereami, so the client can have customized information of sampling distribution.

@byron-hawkins
Copy link
Contributor

Two additional issues with profiling clients:

  1. If the client acquires locks, the profiler will potentially deadlock (or hit rank-order asserts) because for interrupts occurring in client code, whereami will still (erroneously) be WHERE_FCACHE.
  2. The profiler will crash on exit if samples are taken in client code, because (a) the profiler will label each client pc as WHERE_UNKNOWN (after failing to find it in the fcache), and (b) the profiler disassembles every pc labeled WHERE_UNKNOWN during the final report, which crashes because the client module has already been unloaded at that point.

@derekbruening
Copy link
Contributor Author

Just to clarify, 1. above should say "occurring in client clean callees out of the code cache", not arbitrary client code.

@byron-hawkins
Copy link
Contributor

The two client issues are fixed in 2675a1b. I'm leaving this issue open because the public documentation for -prof_pcs has still not been addressed.

@derekbruening
Copy link
Contributor Author

Revisiting this and leaning toward asking clients to do their own sampling using dr_set_itimer() (on Linux at least), and then providing a get-whereami routine. That allows clients to add their own whereami buckets, do their own bucketing at the granularity they want, get structured data out rather than having to parse our file, and control the sampling interval. (The downside is that it doesn't support diving deep into instrumentation: to do that with client sampling we'd want to export lockless heaps.)

derekbruening added a commit that referenced this issue Mar 15, 2018
Added dr_where_am_i() to better support client self-profiling via
sampling.  This also provides the fragment tag, by refactoring
the pcprofile code into a new helper fcache_refine_whereami().

Renamed the whereami types to better avoid name conflicts:
s/WHERE_xxx/DR_WHERE_xxx/ and s/where_am_i_t/dr_where_am_i_t/.
Exported the dr_where_am_i_t enum.

Fixed a bug where an uninitialized mcontext was passed to a
client timer callback: we only filled in the mcontext for a
DR-internal callback.

Added a test of client sampling to client.timer

Issue: #140
derekbruening added a commit that referenced this issue Mar 15, 2018
Added dr_where_am_i() to better support client self-profiling via
sampling.  This also provides the fragment tag, by refactoring
the pcprofile code into a new helper fcache_refine_whereami().

Renamed the whereami types to better avoid name conflicts:
s/WHERE_xxx/DR_WHERE_xxx/ and s/where_am_i_t/dr_where_am_i_t/.
Exported the dr_where_am_i_t enum.

Fixed a bug where an uninitialized mcontext was passed to a
client timer callback: we only filled in the mcontext for a
DR-internal callback.

Added a test of client sampling to client.timer

Issue: #140
@derekbruening
Copy link
Contributor Author

One problem with the new dr_where_am_i query used in a client itimer is that DR_WHERE_CLEAN_CALLEE is not set, because that's under a check for -prof_pcs to reduce overhead: so the client ends up seeing DR_WHERE_UNKNOWN instead.

To fix: enable if client ever called dr_set_itimer()?? Or have a new API
routine dr_enable_whereami() or sthg?

@derekbruening
Copy link
Contributor Author

A couple other DR_WHERE_UNKNOWN instances include:

  • Context switch code: today DR fails to check for shared vs private and for misc routines like do_syscall which we should consider "context switch" code.
  • Waiting for suspend signals: DR_WHERE_SIGNAL_HANDLER is not set enough.
  • Corner cases like this one which I do not plan to address:

This is literally right after enter_fcache() sets "dcontext->whereami = DR_WHERE_FCACHE":

(gdb) x/4i 0x0000562eae74a4da
   0x562eae74a4ca <dispatch+2522>:      callq  0x562eae7158f0 <set_fcache_target>
   0x562eae74a4cf <dispatch+2527>:      movl   $0x9,0x300(%r13)
=> 0x562eae74a4da <dispatch+2538>:      mov    %r13,%rdi
   0x562eae74a4dd <dispatch+2541>:      callq  *%rbx
   0x562eae74a4df <dispatch+2543>:      cmpb   $0x0,0x304(%r13)
   0x562eae74a4e7 <dispatch+2551>:      je     0x562eae74a140 <dispatch+1616>

It's unclear how to handle this one: checking whether in DR is not very precise.
If we're going to do that we should instead just keep the FCACHE label and
never call them UNKNOWN. I'm going to leave that as UNKNOWN.

derekbruening added a commit that referenced this issue Apr 18, 2018
Adds a new API routine dr_track_where_am_i() which enables tracking of
whereami for clean calls.  This is off by default for performance reasons,
which leads to clean calls being labeled initially DR_WHERE_FCACHE and then
refined into DR_WHERE_UNKNOWN.

Adds a query routine dr_is_tracking_where_am_i() for use in the tracer
where this tracking causes a short jump to not reach (#2147).

Cleans up a hole in whereami tracking to include shared context switch code
and do_syscall code along with other generated transition code as
DR_WHERE_CONTEXT_SWITCH.

Adds labeling of DR_WHERE_SIGNAL_HANDLER for at least the ksynch_wait on a
suspend signal; including other handler code is trickier and is left for
future work.

Issue: #140
derekbruening added a commit that referenced this issue Apr 18, 2018
Adds a new API routine dr_track_where_am_i() which enables tracking of
whereami for clean calls.  This is off by default for performance reasons,
which leads to clean calls being labeled initially DR_WHERE_FCACHE and then
refined into DR_WHERE_UNKNOWN.

Adds a query routine dr_is_tracking_where_am_i() for use in the tracer
where this tracking causes a short jump to not reach (#2147).

Cleans up a hole in whereami tracking to include shared context switch code
and do_syscall code along with other generated transition code as
DR_WHERE_CONTEXT_SWITCH.

Adds labeling of DR_WHERE_SIGNAL_HANDLER for at least the ksynch_wait on a
suspend signal; including other handler code is trickier and is left for
future work.

Issue: #140
derekbruening added a commit that referenced this issue Apr 24, 2018
The client syscall events are called out of dispatch separately from DR's
syscall handlers, so today they end up in the DR_WHERE_DISPATCH bucket.
Here we mark them as the more appropriate DR_WHERE_SYSCALL_HANDLER.

Issue: #140
derekbruening added a commit that referenced this issue Apr 24, 2018
The client syscall events are called out of dispatch separately from DR's
syscall handlers, so today they end up in the DR_WHERE_DISPATCH bucket.
Here we mark them as the more appropriate DR_WHERE_SYSCALL_HANDLER.

Issue: #140
derekbruening added a commit that referenced this issue May 2, 2018
Relaxes a curiosity assert triggered by 523af3b which added further use of
DR_WHERE_SYSCALL_HANDLER.

Issue: #140
derekbruening added a commit that referenced this issue May 2, 2018
Relaxes a curiosity assert triggered by 523af3b which added further use of
DR_WHERE_SYSCALL_HANDLER.

Issue: #140
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants