Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support swapcontext #189

Closed
ramosian-glider opened this issue Aug 31, 2015 · 47 comments
Closed

support swapcontext #189

ramosian-glider opened this issue Aug 31, 2015 · 47 comments

Comments

@ramosian-glider
Copy link
Member

Originally reported on Google Code with ID 189

AddressSanitizer does not fully support swapcontext. 
Sometimes, swapcontext causes the entire shadow region (16T) 
to be written by asan-internal routines (e.g. __asan_handle_no_return)
because the location of the stack changes w/o asan noticing it.
This may cause the machine to die or hang for a long time. 

I am not at all sure if asan can fully support swapcontext, 
but we at least should collect more test cases. 

Reported by konstantin.s.serebryany on 2013-05-22 07:40:59

@ramosian-glider
Copy link
Member Author

http://llvm.org/viewvc/llvm-project?rev=182456&view=rev adds a workaround and a better
test.

Full fix may require a significant surgery, so I'd like to see if a simple thing
is enough. 

Reported by konstantin.s.serebryany on 2013-05-22 09:04:27

@ramosian-glider
Copy link
Member Author

I've got a test case that gives a false-positive error around swapcontext:

"ERROR: AddressSanitizer: SEGV on unknown address 0x000000000"

When I blacklist the file making that call, the code then prints a warning referring
to this bug:

"WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: . . . 
False positive error reports may follow
For details see http://code.google.com/p/address-sanitizer/issues/detail?id=189"

It's from the test suite of the Charm++ parallel runtime system (http://charmplusplus.org).
If test cases for this would be useful, I'd be happy to help in understanding that
code. If you want it reduced, I can probably do some, but it's a fairly large system
with a lot of cross-dependencies.

Reported by unmobile on 2014-01-08 20:30:01

@ramosian-glider
Copy link
Member Author

Does asan actually report false positives after the warning about swapcontext?
A minimized test is always welcome, but we can not promise that we'll fix it -- 
swapcontext is a really tricky beast. 

Reported by konstantin.s.serebryany on 2014-01-09 04:50:37

@ramosian-glider
Copy link
Member Author

Note that it generally makes little sense in blacklisting the code that performs a NULL
dereference.

Reported by ramosian.glider on 2014-01-10 10:39:22

@ramosian-glider
Copy link
Member Author

Here is false positive.

When you destroy a std::exception_ptr allocated from another stack without rethrowing
it, then it crashes.

GCC 4.9.2 (on Gentoo). Boost 1.56.0 compiled with C++11 support.

{{{
==26409==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7fff0420b000;
bottom 0x63100000f000; size: 0x1cef041fc000 (31812891951104)
False positive error reports may follow
For details see http://code.google.com/p/address-sanitizer/issues/detail?id=189
=================================================================
==26409==ERROR: AddressSanitizer: stack-buffer-underflow on address 0x6310000104a0
at pc 0x7fd9fccdcde3 bp 0x631000010320 sp 0x63100000fac8
WRITE of size 240 at 0x6310000104a0 thread T0
    #0 0x7fd9fccdcde2 (/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libasan.so.1+0x2fde2)
    #1 0x7fd9fbe8b046 in _Unwind_Resume (/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libgcc_s.so.1+0x10046)
    #2 0x406dc9 in my_coroutine(boost::coroutines::pull_coroutine<std::__exception_ptr::exception_ptr>&)
(/tmp/a.out+0x406dc9)
    #3 0x41e7f4 in boost::coroutines::detail::push_coroutine_object<boost::coroutines::pull_coroutine<std::__exception_ptr::exception_ptr>,
std::__exception_ptr::exception_ptr, void (&)(boost::coroutines::pull_coroutine<std::__exception_ptr::exception_ptr>&),
boost::coroutines::basic_standard_stack_allocator<boost::coroutines::stack_traits>
>::run(std::__exception_ptr::exception_ptr*) (/tmp/a.out+0x41e7f4)
    #4 0x41bb88 in void boost::coroutines::detail::trampoline_push<boost::coroutines::detail::push_coroutine_object<boost::coroutines::pull_coroutine<std::__exception_ptr::exception_ptr>,
std::__exception_ptr::exception_ptr, void (&)(boost::coroutines::pull_coroutine<std::__exception_ptr::exception_ptr>&),
boost::coroutines::basic_standard_stack_allocator<boost::coroutines::stack_traits>
> >(long) (/tmp/a.out+0x41bb88)
    #5 0x7fd9fc89e710 in make_fcontext (/usr/lib64/libboost_context-cxx11-gcc4_9_2.so.1.56.0+0x710)

0x6310000104a0 is located 64672 bytes inside of 65536-byte region [0x631000000800,0x631000010800)
allocated by thread T0 here:
    #0 0x7fd9fcd04787 in malloc (/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/libasan.so.1+0x57787)
    #1 0x414890 in boost::coroutines::basic_standard_stack_allocator<boost::coroutines::stack_traits>::allocate(boost::coroutines::stack_context&,
unsigned long) (/tmp/a.out+0x414890)
    #2 0x40d975 in boost::coroutines::push_coroutine<std::__exception_ptr::exception_ptr>::push_coroutine<void
(&)(boost::coroutines::pull_coroutine<std::__exception_ptr::exception_ptr>&)>(void
(&)(boost::coroutines::pull_coroutine<std::__exception_ptr::exception_ptr>&), boost::coroutines::attributes
const&) (/tmp/a.out+0x40d975)
    #3 0x406ecf in main (/tmp/a.out+0x406ecf)
    #4 0x7fd9fbaf8dc4 in __libc_start_main (/lib64/libc.so.6+0x24dc4)
}}}

Reported by vdavid@vizrt.com on 2014-12-10 18:51:48


- _Attachment: [test_coroutine.cpp](https://storage.googleapis.com/google-code-attachments/address-sanitizer/issue-189/comment-6/test_coroutine.cpp)_

@ramosian-glider
Copy link
Member Author

Reported by ramosian.glider on 2015-07-30 09:05:31

  • Labels added: ProjectAddressSanitizer

@ramosian-glider
Copy link
Member Author

Adding Project:AddressSanitizer as part of GitHub migration.

Reported by ramosian.glider on 2015-07-30 09:06:55

@hbowden
Copy link

hbowden commented Jan 11, 2016

I ran into this bug as well and made a test case. It's derived from the test suite in a fuzzer I'm writing. https://github.com/2trill2spill/nextgen . This was tested on Mac OSX 10.11.12, and below is the output from clang --version.

nahs-MBP:desktop nah$ clang --version
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.2.0
Thread model: posix

And the test case, which was compiled with clang -fsanitize=address -o test.c test.

#include <setjmp.h>
#include <stdio.h>
#include <string.h>
#include <signal.h>

static jmp_buf return_jump;

static void signal_handler(int sig)
{
    longjmp(return_jump, 1);
}

static void setup_test_sig_handler(void)
{
    struct sigaction sa;
    sigset_t ss;
    unsigned int i;

    for(i = 1; i < 512; i++)
    {
        (void)sigfillset(&ss);
        sa.sa_flags = SA_RESTART;
        sa.sa_handler = signal_handler;
        sa.sa_mask = ss;
        (void)sigaction((int)i, &sa, NULL);
    }

    return;
}

int main(void)
{
    int rtrn = setjmp(return_jump);
    if(rtrn < 0)
    {
        perror("setjmp");
        return (-1);
    }

    setup_test_sig_handler();

    /* Cause signal. */
    memmove(NULL, "123456789", 9);

    return (0);
}

@kcc
Copy link
Contributor

kcc commented Jan 12, 2016

2trill2spill, why is the previous comment related to this bug?
The code does not even have swapcontext call.
Please open a separate bug explaining what exactly went wrong.

@hbowden
Copy link

hbowden commented Jan 12, 2016

The error message I get from running the test case points to this page, so I assumed that It was the same issue.

ASAN:SIGSEGV
=================================================================
==8425==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000105ab39e2 bp 0x7fff5a1ab9a0 sp 0x7fff5a1ab128 T0)
    #0 0x105ab39e1 in __sanitizer::internal_memmove(void*, void const*, unsigned long) (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/7.0.2/lib/darwin/libclang_rt.asan_osx_dynamic.dylib+0x569e1)
    #1 0x105a54a9e in main (/Users/nah/Desktop/./test+0x100000a9e)
    #2 0x7fff95a7c5ac in start (/usr/lib/system/libdyld.dylib+0x35ac)
    #3 0x0  (<unknown module>)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV ??:0 __sanitizer::internal_memmove(void*, void const*, unsigned long)
==8425==ABORTING
==8425==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7fff5a1ac000; bottom 0x000106b5d000; size: 0x7ffe5364f000 (140730297544704)
False positive error reports may follow
For details see http://code.google.com/p/address-sanitizer/issues/detail?id=189
ASAN:SIGSEGV
==8425==AddressSanitizer: while reporting a bug found another one. Ignoring.
`

@kcc
Copy link
Contributor

kcc commented Jan 12, 2016

I don't see such message on Linux, so it might be a OSX-specific issue,
unrelated to swapcontext. Please file a separate bug, will discuss it there.

ghost pushed a commit to facebook/folly that referenced this issue Mar 17, 2016
Summary:ASAN needs to know about the stack extents. Currently it has no knowledge of
fibers and so it can give false positives, particularly in cases of no-return
(think exceptions).

See: google/sanitizers#189

This change along with a related ASAN diff fixes that, and I've verified it
fixes false positive test failures I'm seeing when throws happen from fibers.

Also rips out some hacks that attempted to work around the limitations of
ASAN these changes should fix.

This change depends on:
D3017630
D2952854
D3017619

And will also depend on rollout of libasan.so to /usr/local/fbcode platform dirs on all machines.

Reviewed By: andriigrynenko

Differential Revision: D2952899

fb-gh-sync-id: 19da779227c6c0f30c5755806325aa4cba364cfe
shipit-source-id: 19da779227c6c0f30c5755806325aa4cba364cfe
@felixguendling
Copy link

felixguendling commented Apr 6, 2016

Can anyone explain how facebook/folly@2ea64dd works? I could not find anything like "__asan_enter_fiber"?

Is there something like VALGRIND_STACK_REGISTER / VALGRIND_STACK_DEREGISTER available for address sanitizer?

@blastrock
Copy link

Since there is nothing on the Internet about those functions, I guess that facebook has a fork of clang where they have implemented them.

I tried to implement the functions myself here, but I have very little knowledge of how things work, feedback is welcome. Note that the function prototypes are a little different from those used by folly. I tested this code with an implementation of coroutines on top of boost context v2, the warning about handle_no_return has indeed disappeared and it seems to work.

@felixguendling
Copy link

Thank you! I will try your modified version as soon as possible. You just include the "asan_interface_internal.h" header in your binary or do you use the approach from facebook/folly@2ea64dd (dlsym)?

@avikivity
Copy link

@pdziepak

@blastrock
Copy link

I only did some tests for the moment where I just declared the functions in my project:

extern "C"
{
void __asan_enter_fiber(void const* stack_top, void const* stack_bottom);
void __asan_exit_fiber();
}

Of course this fails to link if I don't compile with asan. I think folly's approach (with the weak symbol attributes and the fallback on dlsym) should work too.

@blastrock
Copy link

FYI my patch finally passed review (a lot of things were fixed in the meantime), http://llvm.org/viewvc/llvm-project?view=revision&revision=273260 . Now let's wait for clang 3.9 :)

@felixguendling
Copy link

That's great news! Thank you very much for your effort! 👍

@ioquatix
Copy link

ioquatix commented Jul 4, 2017

I just ran into this issue. I see the solution is to notify asan if switching stacks? I'm implementing coroutines.

@avikivity
Copy link

@ioquatix on gcc? Whoa!

I think gcc 7 and latest clang have better support for makecontext and friends.

@blastrock
Copy link

Yes, the idea is to annotate your code to notify asan when you switch context.

You can find some documentation here https://github.com/llvm-mirror/compiler-rt/blob/master/include/sanitizer/common_interface_defs.h#L166 and an example in the tests https://github.com/llvm-mirror/compiler-rt/blob/master/test/asan/TestCases/Linux/swapcontext_annotation.cc .

Since the test is still there, I don't think swapcontext has got any more support for asan.

xiaoxiang781216 pushed a commit to xiaoxiang781216/libmetal that referenced this issue Jul 13, 2021
N/A

leads to ASAN errors
libmetal/lib/system/nuttx/io.c:78:9: runtime error: applying non-zero offset 4124131344 to null pointer
metal/io.h:136:3: runtime error: applying non-zero offset 4124131440 to null pointer
metal/io.h:136:3: runtime error: applying non-zero offset 4124131440 to null pointer
metal/io.h:136:3: runtime error: applying non-zero offset 4124131440 to null pointer
metal/io.h:136:3: runtime error: applying non-zero offset 4124131552 to null pointer
metal/io.h:136:3: runtime error: applying non-zero offset 4124132000 to null pointer
==15484==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0xffd3c000; bottom 0xf3437000; size: 0x0c905000 (210784256)
False positive error reports may follow
For details see google/sanitizers#189

Change-Id: I68a9ff662a35de19de166347844565bf8eb44935
Signed-off-by: Jiuzhu Dong <dongjiuzhu1@xiaomi.com>
xiaoxiang781216 pushed a commit to xiaoxiang781216/libmetal that referenced this issue Oct 22, 2021
leads to ASAN errors
libmetal/lib/system/nuttx/io.c:78:9: runtime error: applying non-zero offset 4124131344 to null pointer
metal/io.h:136:3: runtime error: applying non-zero offset 4124131440 to null pointer
metal/io.h:136:3: runtime error: applying non-zero offset 4124131440 to null pointer
metal/io.h:136:3: runtime error: applying non-zero offset 4124131440 to null pointer
metal/io.h:136:3: runtime error: applying non-zero offset 4124131552 to null pointer
metal/io.h:136:3: runtime error: applying non-zero offset 4124132000 to null pointer
==15484==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0xffd3c000; bottom 0xf3437000; size: 0x0c905000 (210784256)
False positive error reports may follow
For details see google/sanitizers#189

Signed-off-by: Jiuzhu Dong <dongjiuzhu1@xiaomi.com>
@stsp
Copy link

stsp commented Jan 7, 2022

Hi guys.

Should the custom swapcontext()
be somehow annotated to asan?
I've got asan working by using the
glibc's swapcontext() and __sanitizer_start_switch_fiber
__sanitizer_finish_switch_fiber annotations.
But when using the custom, asm-written
swapcontext()-alike function, I can't
get things to work even with the same
switch_fiber annotations.
So should I somehow also annotate
the custom swapcontext function?

It crashes in a function epilogue that looks like this:

0x00005555560f49af <+1191>:	je     0x5555560f49d2 <co_switch_context+1226>
   0x00005555560f49b1 <+1193>:	movq   $0x45e0360e,(%rbx)
   0x00005555560f49b8 <+1200>:	movabs $0xf5f5f5f5f5f5f5f5,%rax
   0x00005555560f49c2 <+1210>:	mov    %rax,0x7fff8000(%r14)
   0x00005555560f49c9 <+1217>:	mov    0x38(%rbx),%rax
=> 0x00005555560f49cd <+1221>:	movb   $0x0,(%rax)

rax==0 here.
I don't understand what does this epilog
code do and why it crashes only with the
custom swapcontext().

@stsp
Copy link

stsp commented Jan 7, 2022

https://github.com/gcc-mirror/gcc/blob/master/libsanitizer/asan/asan_interceptors.cpp#L243
Obviously asan intercepts swapcontext()
and another *context functions.
So seems like there is no way to use
the custom swapcontext() with asan?

@ioquatix
Copy link

ioquatix commented Jan 8, 2022

Just for reference, this is how I implemented it: https://github.com/kurocha/concurrent/blob/6eee988ba7263f017a8d74560afde2f0396c1370/source/Concurrent/Fiber.cpp#L46-L70

@felixguendling
Copy link

Asan support is working for us like this:
https://github.com/motis-project/ctx/blob/master/include/ctx/impl/operation.h#L51-L86

We're using this in combination with deboost.context.

@stsp
Copy link

stsp commented Jan 8, 2022

Thanks, deboost.context indeed looks
like using its own asm for context switching,
and yet it works for you with asan with
just a basic *_switch_fiber() annotations...
Interesting.

As for "concurrent" project mentioned
by @ioquatix - I can't find the custom
context switching primitives there.

@stsp
Copy link

stsp commented Jan 8, 2022

https://github.com/septag/deboost.context/blob/master/asm/jump_x86_64_ms_pe_gas.asm#L164

 	    movq  %gs:(0x30), %r10
	    /* restore fiber local storage */
	    movq  0xb0(%rsp), %rax
	    movq  %rax, 0x20(%r10)
	    /* restore current deallocation stack */
	    movq  0xb8(%rsp), %rax
	    movq  %rax, 0x1478(%r10)
	    /* restore current stack limit */
	    movq  0xc0(%rsp), %rax
	    movq  %rax, 0x10(%r10)
	    /* restore current stack base */
	    movq  0xc8(%rsp), %rax
	    movq  %rax, 0x08(%r10)

@felixguendling - is this code snip
written specifically for asan? Or some
other purpose?

@ioquatix
Copy link

ioquatix commented Jan 8, 2022

@stsp
Copy link

stsp commented Jan 8, 2022

Thanks!
Its very simplistic:
https://github.com/kurocha/coroutine-amd64/blob/master/source/Coroutine/Context.s
Just pushes a few regs on stack.
And yet it works with asan...
Then perhaps I need to find
a problem in the context-switching
code I took from libtask...

@ioquatix
Copy link

ioquatix commented Jan 8, 2022

Coroutine transfer is a simple operation, it's a function call and return with a stack swap in the middle. Any implementation that makes it more complicated than that is wrong. IMHO :)

@stsp
Copy link

stsp commented Jan 8, 2022

You are right. :)
And still some guys (like myself)
can shoot their feet even here.
I had a Cish wrapper around asm
getcontext, and it wasn't marked
with always_inline attribute.
As the result, it was saving its own
stack frame to the context struct...
Your example, being that simple,
helped me to realize the stupidity.
I wonder why it never broke w/o asan...

ladeiko added a commit to ladeiko/SwiftCoroutine that referenced this issue Aug 5, 2022
…detect mods to free blocks

SwiftCoroutineDemo(39856,0x11266e600) malloc: nano zone abandoned due to inability to preallocate reserved vm space.
DispatchQueue::scheduleTask() <OS_dispatch_queue_main: com.apple.main-thread>
SharedCoroutineDispatcher::getFreeQueue(SwiftCoroutine.SharedCoroutineDispatcher)
CoroutineContext::init(stackSize=196608, guardPage=true, SwiftCoroutine.CoroutineContext) returnEnv = 0x000060f0000042d0 stack = 0x000000010effb000
SharedCoroutineQueue::start(SwiftCoroutine.SharedCoroutineQueue)
CoroutineContext::start(SwiftCoroutine.CoroutineContext) enter
CoroutineContext::stackTop(SwiftCoroutine.CoroutineContext) 0x000000010f02b000
CoroutineContext::start(Optional(0x00006060000b8400)) block enter
==39856==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ff7b7045000; bottom 0x00010f028000; size: 0x7ff6a801d000 (140697357373440)
False positive error reports may follow
For details see google/sanitizers#189
CoroutineContext::start(SwiftCoroutine.CoroutineContext) leave
DispatchQueue::scheduleTask() <OS_dispatch_queue_main: com.apple.main-thread>
SharedCoroutineDispatcher::getFreeQueue(SwiftCoroutine.SharedCoroutineDispatcher)
SharedCoroutineQueue::start(SwiftCoroutine.SharedCoroutineQueue)
CoroutineContext::stackTop(SwiftCoroutine.CoroutineContext) 0x000000010f02b000
=================================================================
==39856==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x00010f029ec1 at pc 0x00010949d77d bp 0x7ff7b7040e90 sp 0x7ff7b7040650
READ of size 4416 at 0x00010f029ec1 thread T0
    #0 0x10949d77c in wrap_memcpy+0x16c (libclang_rt.asan_iossim_dynamic.dylib:x86_64+0x1977c)
    belozierov#1 0x109ef7f83 in SharedCoroutine.saveStack() SharedCoroutine.swift:79
    belozierov#2 0x109eff514 in SharedCoroutineQueue.start(dispatcher:scheduler:task:) SharedCoroutineQueue.swift:39
    belozierov#3 0x109efc229 in closure belozierov#1 in SharedCoroutineDispatcher.execute(on:task:) SharedCoroutineDispatcher.swift:27
    belozierov#4 0x109ee7ed9 in OS_dispatch_queue.scheduleTask(_:) CoroutineScheduler+DispatchQueue.swift:16
    belozierov#5 0x109ee88ad in protocol witness for CoroutineScheduler.scheduleTask(_:) in conformance OS_dispatch_queue <compiler-generated>
    belozierov#6 0x109efbf48 in SharedCoroutineDispatcher.execute(on:task:) SharedCoroutineDispatcher.swift:26
    belozierov#7 0x109ee3616 in CoroutineScheduler._startCoroutine(_:) CoroutineScheduler.swift:51
    belozierov#8 0x109ee3c78 in CoroutineScheduler.startCoroutine(in:task:) CoroutineScheduler.swift:69
    belozierov#9 0x108ebe2ed in AppDelegate.test() AppDelegate.swift:43
    belozierov#10 0x108ebdd77 in AppDelegate.application(_:didFinishLaunchingWithOptions:) AppDelegate.swift:18
    belozierov#11 0x108ebe4b7 in @objc AppDelegate.application(_:didFinishLaunchingWithOptions:) <compiler-generated>
    belozierov#12 0x7fff2509a592 in -[UIApplication _handleDelegateCallbacksWithOptions:isSuspended:restoreState:]+0xd5 (UIKitCore:x86_64+0xcd7592)
    belozierov#13 0x7fff2509c283 in -[UIApplication _callInitializationDelegatesWithActions:forCanvas:payload:fromOriginatingProcess:]+0x101f (UIKitCore:x86_64+0xcd9283)
    belozierov#14 0x7fff250a1c23 in -[UIApplication _runWithMainScene:transitionContext:completion:]+0x4a5 (UIKitCore:x86_64+0xcdec23)
    belozierov#15 0x7fff246124a2 in -[_UISceneLifecycleMultiplexer completeApplicationLaunchWithFBSScene:transitionContext:]+0xb2 (UIKitCore:x86_64+0x24f4a2)
    belozierov#16 0x7fff2509e2ac in -[UIApplication _compellApplicationLaunchToCompleteUnconditionally]+0x3a (UIKitCore:x86_64+0xcdb2ac)
    belozierov#17 0x7fff2509e64b in -[UIApplication _run]+0x391 (UIKitCore:x86_64+0xcdb64b)
    belozierov#18 0x7fff250a32b4 in UIApplicationMain+0x64 (UIKitCore:x86_64+0xce02b4)
    belozierov#19 0x7fff59d55cc1 in UIApplicationMain(_:_:_:_:)+0x61 (libswiftUIKit.dylib:x86_64+0x21cc1)
    belozierov#20 0x108ebf357 in static UIApplicationDelegate.main() <compiler-generated>
    belozierov#21 0x108ebf2a0 in static AppDelegate.$main() AppDelegate.swift:11
    belozierov#22 0x108ebfc77 in main <compiler-generated>
    belozierov#23 0x1090f8f20 in start_sim+0x9 (dyld_sim:x86_64+0x1f20)
    belozierov#24 0x1125f352d  (<unknown module>)

0x00010f029ec1 is located 192193 bytes inside of 200704-byte region [0x00010effb000,0x00010f02c000)
allocated by thread T0 here:
    #0 0x1094c7bb3 in wrap_posix_memalign+0xb3 (libclang_rt.asan_iossim_dynamic.dylib:x86_64+0x43bb3)
    belozierov#1 0x7fff30c9e62c in swift_slowAlloc+0x4c (libswiftCore.dylib:x86_64+0x30762c)
    belozierov#2 0x109ede243 in CoroutineContext.init(stackSize:guardPage:) CoroutineContext.swift:32
    belozierov#3 0x109eddc1f in CoroutineContext.__allocating_init(stackSize:guardPage:) CoroutineContext.swift
    belozierov#4 0x109efe985 in SharedCoroutineQueue.init(stackSize:) SharedCoroutineQueue.swift:28
    belozierov#5 0x109efe5c3 in SharedCoroutineQueue.__allocating_init(stackSize:) SharedCoroutineQueue.swift
    belozierov#6 0x109efcb3d in SharedCoroutineDispatcher.getFreeQueue() SharedCoroutineDispatcher.swift:38
    belozierov#7 0x109efc20d in closure belozierov#1 in SharedCoroutineDispatcher.execute(on:task:) SharedCoroutineDispatcher.swift:27
    belozierov#8 0x109ee7ed9 in OS_dispatch_queue.scheduleTask(_:) CoroutineScheduler+DispatchQueue.swift:16
    belozierov#9 0x109ee88ad in protocol witness for CoroutineScheduler.scheduleTask(_:) in conformance OS_dispatch_queue <compiler-generated>
    belozierov#10 0x109efbf48 in SharedCoroutineDispatcher.execute(on:task:) SharedCoroutineDispatcher.swift:26
    belozierov#11 0x109ee3616 in CoroutineScheduler._startCoroutine(_:) CoroutineScheduler.swift:51
    belozierov#12 0x109ee3c78 in CoroutineScheduler.startCoroutine(in:task:) CoroutineScheduler.swift:69
    belozierov#13 0x108ebe2ed in AppDelegate.test() AppDelegate.swift:43
    belozierov#14 0x108ebdd77 in AppDelegate.application(_:didFinishLaunchingWithOptions:) AppDelegate.swift:18
    belozierov#15 0x108ebe4b7 in @objc AppDelegate.application(_:didFinishLaunchingWithOptions:) <compiler-generated>
    belozierov#16 0x7fff2509a592 in -[UIApplication _handleDelegateCallbacksWithOptions:isSuspended:restoreState:]+0xd5 (UIKitCore:x86_64+0xcd7592)
    belozierov#17 0x7fff2509c283 in -[UIApplication _callInitializationDelegatesWithActions:forCanvas:payload:fromOriginatingProcess:]+0x101f (UIKitCore:x86_64+0xcd9283)
    belozierov#18 0x7fff250a1c23 in -[UIApplication _runWithMainScene:transitionContext:completion:]+0x4a5 (UIKitCore:x86_64+0xcdec23)
    belozierov#19 0x7fff246124a2 in -[_UISceneLifecycleMultiplexer completeApplicationLaunchWithFBSScene:transitionContext:]+0xb2 (UIKitCore:x86_64+0x24f4a2)
    belozierov#20 0x7fff2509e2ac in -[UIApplication _compellApplicationLaunchToCompleteUnconditionally]+0x3a (UIKitCore:x86_64+0xcdb2ac)
    belozierov#21 0x7fff2509e64b in -[UIApplication _run]+0x391 (UIKitCore:x86_64+0xcdb64b)
    belozierov#22 0x7fff250a32b4 in UIApplicationMain+0x64 (UIKitCore:x86_64+0xce02b4)
    belozierov#23 0x7fff59d55cc1 in UIApplicationMain(_:_:_:_:)+0x61 (libswiftUIKit.dylib:x86_64+0x21cc1)
    belozierov#24 0x108ebf357 in static UIApplicationDelegate.main() <compiler-generated>
    belozierov#25 0x108ebf2a0 in static AppDelegate.$main() AppDelegate.swift:11
    belozierov#26 0x108ebfc77 in main <compiler-generated>
    belozierov#27 0x1090f8f20 in start_sim+0x9 (dyld_sim:x86_64+0x1f20)
    belozierov#28 0x1125f352d  (<unknown module>)

SUMMARY: AddressSanitizer: stack-buffer-overflow (libclang_rt.asan_iossim_dynamic.dylib:x86_64+0x1977c) in wrap_memcpy+0x16c
Shadow bytes around the buggy address:
  0x0001344ad380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0001344ad390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0001344ad3a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0001344ad3b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0001344ad3c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0001344ad3d0: 00 00 00 00 f1 f1 f1 f1[01]f3 f3 f3 00 00 00 00
  0x0001344ad3e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0001344ad3f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0001344ad400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0001344ad410: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0001344ad420: 00 00 00 00 00 00 00 00 00 00 00 00 ca ca ca ca
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==39856==ABORTING
AddressSanitizer report breakpoint hit. Use 'thread info -s' to get extended information about the report.
(lldb)
ChuanqiXu9 added a commit to alibaba/async_simple that referenced this issue Oct 31, 2022
Currently the sanitizer can't handle user thread properly.
See google/sanitizers#189 for details.
Close it temporarily to make CI sound.
@kyrieSun-wow
Copy link

kyrieSun-wow commented Dec 1, 2022

Hi everyone, @ioquatix I met an issue when I tried to enable Asan in the c program using swapcontext function, are you free to provide some advice?

The following are the details:
Build server: gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)

With the help of ___sanitizer_start_switch_fiber/_sanitizer_finish_switch_fiber, even if I turned on the Asan, my program worked fine in jumpping from one to another coroutine(like main thread to coroutine func2, coroutine func2 to coroutine func1), but as soon as I try to restore the old coroutine by jumping back to it, Asan stops working(stack-buffer-overflow can not be detected).

And I noticed that the argument func2_thread_stack below got nothing all the time:

    void *func2_thread_stack = NULL;
    __sanitizer_start_switch_fiber(&func2_thread_stack,uctx_func1.uc_stack.ss_sp,uctx_func1.uc_stack.ss_size);
    ……… (swap from func2 to func1, and then func1 swap back func2)
    __sanitizer_finish_switch_fiber(func2_thread_stack, &from_stack, &from_stacksize);

Could you give me some advice?

Sincerely,

@idleroamer
Copy link

I need to share my experience with boost-asio and asan briefly for all other fellows that are me three days ago.

If this is your problem: "You need to make asan and boost-asio with coroutines get along" then your are in for a treat.

I put some pieces together but it is essentially no-brainer, boost-asio does not have the incentives to move to coroutine2 chriskohlhoff/asio#603 but thanks to https://github.com/cbodley/spawn (a stand-
alone header-only library of the latter PR) it can work.
After switching all boost::asio::spawn to spawn::spawn you are through the tedious parts.

Obviously you need to build boost with asan support as mentioned here boostorg/coroutine#30 (comment)

context-impl=ucontext -DBOOST_USE_ASAN -DBOOST_USE_UCONTEXT

And simplest part build your project with -DBOOST_USE_ASAN DBOOST_USE_UCONTEXT flags.
voila

@kyrieSun-wow
Copy link

kyrieSun-wow commented Dec 12, 2022

Thanks to @ioquatix , the compatible issue between ASAN and swapcontext is solved in my program. He was very friendly and very patient, and he taught me a lot about asan. I have compiled some of the lessons he taught me so that more people like me can learn from it:

Compatible issue between ASAN and swapcontext()
1.Phenomenon
ASAN does not fully support swapcontext technology, as asan has indicated in log:
==1000==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!

Under this constraint, if function swapcontext() is introduced in your program, there will be some false positives reported after coroutine was changed. 
The detection capability of ASAN is almost ineffective, and even seriously affects the normal operation of the program.

2.Mechanism of asan
To solve this problem, we need to understand why these false positives occur.

And to understand why these false positives occur, we need to learn how asan works:
ASAN needs to allocate and store a shadow stack for each fiber, to track usage.
You should also poison the stack when it’s no longer in use (e.g. if you track a high water mark, or completely free it).

3.Way to make swapcontext() compatible with asan
Note: The flag 'ASAN_OPTIONS=detect_stack_use_after_return=true' is necessary when the swapcontext() function is used on your program.

Therefore, we need to find a way to notify ASAN before/after we exchange the fiber.

To make things easier, I recommend adding fake_stack pointer for every fiber when ASAN is enabled.

For this fake_stack:

  • The fake stack is required per context/fiber/coroutine for the purpose of tracking memory usage in ASAN.
  • ASAN will allocate the fake stack.

And when we try to jump to new(target) coroutine by executing swapcontext(), we need to store the fake_stack of old(current) fiber, so that when we try to return to the old fiber, we can restore the stack of old fiber with the fake_stack we ever stored before. 

Here introduce two function provided by ASAN to manage the fake_stack:

// Fiber annotation interface.
// Before switching to a different stack, one must call
// __sanitizer_start_switch_fiber with a pointer to the bottom of the
// destination stack and its size. When code starts running on the new stack,
// it must call __sanitizer_finish_switch_fiber to finalize the switch.
// The start_switch function takes a void** to store the current fake stack if
// there is one (it is needed when detect_stack_use_after_return is enabled).
// When restoring a stack, this pointer must be given to the finish_switch
// function. In most cases, this void* can be stored on the stack just before
// switching. When leaving a fiber definitely, null must be passed as first
// argument to the start_switch function so that the fake stack is destroyed.
// If you do not want support for stack use-after-return detection, you can
// always pass null to these two functions.
// Note that the fake stack mechanism is disabled during fiber switch, so if a
// signal callback runs during the switch, it will not benefit from the stack
// use-after-return detection.
void __sanitizer_start_switch_fiber(void **fake_stack_save,
                                    const void *bottom, size_t size);

void __sanitizer_finish_switch_fiber(void *fake_stack_save,
                                     const void **bottom_old,
                                     size_t *size_old);

The implementation of these two function is in here: https://github.com/llvm/llvm-project/blob/a2ef44a5d65932c7bb0f483217826856325b60df/compiler-rt/lib/asan/asan_thread.cpp#L526-L551

From the source code, we can see that, __sanitizer_start_switch_fiber will assign the fake_stack IF and ONLY IF you provide a pointer.

This is how I handle swapcontext() issue:

//vthctx: Context of main fiber/coroutine
//vth:  Context of fiber/coroutine A 

Step1: Try to exchange from main fiber to fiber A =========================================================================:
//On the main fiber.

//Argument0: The container for asan to allocate the fake_stack for current fiber.
//           - If we want the current fiber to stay still(we are going to jump back later),then one valid pointer(&vthctx->fake_stack here) shall be passed to argument 0 to store the fake_stack of current fiber;
//           - If we don't want to keep the current fiber alive(we won't jump back), 'NULL' shall be passed to argument 0 to notify asan to delete the fake_stack of current fiber.
//Argument1: The info of target fiber we are going to jump to.
//Argument2: The info of target fiber we are going to jump to.
__sanitizer_start_switch_fiber(&vthctx->fake_stack, vth->uctx.uc_stack.ss_sp, vth->uctx.uc_stack.ss_size);

//exchange to target fiber A. 
swapcontext(&vthctx->tmp_outer_uctx, &vth->uctx);

Step2: On the trigger function of fiber A =========================================================================:
//On the fiber A
const void *from_stack;
size_t from_stacksize;

//Argument0: We are the first time to jump into this fiber, so NULL shall be set as argument 0;
//           - Set argument 0 to 'NULL' means that we have no historical stack to restore for this fiber;
//           - If we have been to this fiber and have historical stack to restore for this fiber, then set the historical stack to argument 0.  
//Argument1: The container for asan to return the info of old fiber we were in before we jumped over.
//Argument2: The container for asan to return the info of old fiber we were in before we jumped over.
__sanitizer_finish_switch_fiber(NULL, &from_stack, &from_stacksize);

 Step3: jump back from fiber A to main fiber=========================================================================:
//Argument0: To store the fake_stack of old fiber before jumping out.
//           - Pass 'NULL' to argument 0 if we won't jump back to fiber A, then asan will delete the fake_stack of fiber A for us.
//           - Pass '&vth->fake_stack'to argument 0 if we plan to keep fiber A alive and we will jump back in the future,and asan will keep the fake_stack of fiber A for us.
//Argument1: The info of target fiber we are going to jump to.
//Argument2: The info of target fiber we are going to jump to.
__sanitizer_start_switch_fiber(NULL, vthctx->tmp_outer_uctx.uc_stack.ss_sp, vthctx->tmp_outer_uctx.uc_stack.ss_size);
 
//exchange to main fiber.  
swapcontext(&vth->uctx, &vthctx->tmp_outer_uctx);

Step4: Restore the fake_stack on main fiber =========================================================================:
//At the point of the main fiber we're jumping back to

//Argument0: The fake_stack sotred before(see Step1),also the one we try to restore for this fiber.
//Argument1: The container for asan to return the info of old fiber we were in before we jumped over.
//Argument2: The container for asan to return the info of old fiber we were in before we jumped over.
__sanitizer_finish_switch_fiber(vthctx->fake_stack, &from_stack, &from_stacksize);

ASAN only cares about tracking the stack swapping, so as long as you wrap the stack exchange operation (coroutine transfer) correctly, ASAN should work well with swapcontext().

@ioquatix
Copy link

I wonder if the above comment can be added to the Wiki?

https://github.com/google/sanitizers/wiki

dotnwat added a commit to dotnwat/redpanda that referenced this issue Dec 1, 2023
New checks in abseil revealed:

13: [/home/nwatkins/src/redpanda/vbuild/debug/clang/rp_deps_install/include/absl/container/internal/raw_hash_set.h : 1259] RAW: Invalid iterator comparison. Comparing iterator with an end() iterator from a different hashtable.
13: ==1795828==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fa25cf16000; bottom 0x7ffd07b05000; size: 0xffffffa555411000 (-389411696640)
13: False positive error reports may follow
13: For details see google/sanitizers#189
13: unknown location(0): fatal error: in "complex_uuid_types_test": signal: SIGABRT (application abort requested)

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests