Comment 1:

Interesting.
So what is a statically linked binary to do?

Comment 2 by amluto:

The same thing as a dynamically-linked binary.  AFAICT the fact that that glibc's
dynamic loader doesn't really work right when statically linked is purely an artefact of
how glibc works.
For example, the attached program can find the vDSO just fine when compiled with -static.

Attachments:

1. auxv.c (998 bytes)

Comment 3:

Okay, it's there, but if you're statically linked how do
you know what addresses to use to call into it?
Are static binaries really supposed to use the vsdo
on Linux 3.1?  If not, what are they supposed to do?
I've looked around for documentation but all I can
find is a few blog posts that just say how to extract a
copy of it.  I've seen the Linus pig email too but that
doesn't say how to use it either.  Is there something
I should be looking at?
Thanks.
Russ

Comment 4 by amluto:

I don't think it's documented very well.  The vDSO is an ELF DSO that's mapped into your
address space by the kernel, even if you're statically linked.  It is the preferred (and
AFAIK only) way to access non-deprecated fast timing functions.
The attached code can be built without libc and almost finds symbols.  I'm not sure how
to find the symbol string table, though.  And it's worth double-checking this stuff.

Attachments:

1. static-vdso.c (3186 bytes)
2. static-vdso-setup.s (85 bytes)

Comment 5 by amluto:

Here's a version that works.
It probably still wants version parsing.  (And it's a bit gross.)

Attachments:

1. static-vdso.c (3681 bytes)

Comment 6:

This can't possibly be what programs linked with gcc -static do.
I'm willing to wait until gcc -static has something.

Status changed to LongTerm.

Comment 7 by amluto:

gcc -static uses the vsyscalls.  Ulrich tells people not to use statically-linked glibc.
I'll see if the kernel folks want to stick a liberally-licensed example implementation
of how to find stuff in the vDSO in the kernel tree somewhere.

Comment 8 by amluto:

...and don't use my code -- it's totally wrong.  I misunderstood the purpose of section
headers and program headers.

Comment 9:

Labels changed: added priority-later.

Comment 10 by amluto:

The changes are in Linux 3.1, and new code should stop using the vsyscall page (with one
exception, noted below).  There is now a reference parser that works in static binaries
in Documentation/vDSO.  Its license should be compatible with the Go runtime.
The exception is that sys_getcpu is not available as a real syscall on all kernels.  So
if statically-linked code needs to work on old kernels and uses getcpu, it should try
the syscall first and fall back to the vsyscall page if it gets -ENOSYS.  There are no
kernels that will fault on vsyscall invocation and will also return -ENOSYS from
sys_getcpu.

Comment 11:

Let's wait a few years.

Comment 12 by amluto:

Or just use the syscall unconditionally.  In Linux 3.1, the vsyscall is never faster
than the syscall, and someone might want to run Go programs in vsyscall=none mode
someday.  (Any distribution with new enough glibc will already work in vsyscall=none
mode, and even older distributions are mostly functional.)

Comment 13:

Labels changed: added priority-someday, removed priority-later.

Comment 14:

I have done a proof-of-concept of using VDSO from a statically-linked Go programs.
The sources (very dirty now) are available here (only works on Linux x86-64):
https://github.com/krasin/vdso/tree/master/vdso-extract
To run it, just do:
go get github.com/krasin/vdso/vdso-extract
vdso-extract
It's statically linked:
$ ldd ~/bin/vdso-extract 
    not a dynamic executable
It runs benchmarks:
Usual syscall.Time(): total: 1.72839s, per iteration: 1.728us
VDSO time: total: 933.261ms, per iteration: 9ns
I'm not sure if these are legitimate numbers, but vdsoTime() indeed returns the same as
syscall.Time().
The kernel is:
Linux li189-163 3.4.2-x86_64-linode25 #1 SMP Tue Jun 12 15:03:23 EDT 2012 x86_64 x86_64
x86_64 GNU/Linux
Please, let me know if you need more data or something did not work for you.

Comment 15 by amluto:

That seems to be making some scary assumptions, such as that __vdso_time is the third
symbol and that /proc/self/maps is usable and points there.
Have you seem Documentation/vDSO/parse_vdso.c in the kernel sources (any recent version)?

Comment 16:

Thanks for taking a look at the code. Yes, it's in the state "something works, but not
suitable for any real use".
I have removed the assumption about the __vdso_time index:
https://github.com/krasin/vdso/commit/17427fb9edabafa4d17547acbb77f2189d2ba360
Thanks for pointing to this hack. I have completely forgot about it.
/proc/self/maps and /proc/self/mem usage is inapropriate for production use, but at
least they are correct. 
I have not yet taken a look at Documentation/vDSO/parse_vdso.c, but I should and I will.

Comment 17:

In order to discover VDSO location, aux vector needs to be parsed. Since argv is hidden
by Go runtime (see http://golang.org/src/pkg/runtime/runtime.c?h=argv#L172), it's not
possible (at least, w/o hacks) to access it from the regular Go program. 
So, hacks-free usage of VDSO is only possible with modified Go runtime. There's no risk
in this, but since my goal was to run benchmarks and show the feasibility, I don't think
it makes sense to do right now.
If Go team decided to support VDSO, I would be happy to create a CL for that.

Comment 18:

We do certainly want to support VDSO.

Comment 19:

I am confused a bit. It appears that Go already has a limited support of VDSO (at least,
for Linux/386):
http://golang.org/src/pkg/runtime/signal_linux_386.c?h=runtime%C2%B7linux_setup_vdso#L141
It has been added by the following CL: http://golang.org/cl/4817054/

Comment 20:

Should be easy to add for amd64 Linux, then.

Comment 21 by amluto:

It looks like the x86 code uses AT_SYSINFO, but x86_64 needs AT_SYINFO_EHDR.  However,
parse_vdso.c will happily resolve vdso symbols using the auxv pointer there, for example.

Comment 22:

Andrew, what's is the difference between linux-gate.so and linux-vdso.so? What's the
difference between x86-32 and x86-64 kernels?

Comment 23:

The work-in-progress CL is here: http://golang.org/cl/6454046/
It parses auxv and vdso on startup, but does not yet expose vdso_sym and does not hook
the corresponding syscalls (like time(2)).
The issue that have taken me a full day is that I have only 120 bytes of stack and a
non-optimizing compiler that likes to spill values on stack (instead of keeping them in
regs). Is any way to increase the stack size here? 240 bytes should be enough, after
some tweaks. 300 bytes would make the code more readable than it would be with 240.

Comment 24:

The work-in-progress CL is here: http://golang.org/cl/6454046/
It parses auxv and vdso on startup, but does not yet expose vdso_sym and does not hook
the corresponding syscalls (like time(2)).
The issue that has taken from me a full day is that I have only 120 bytes of stack and a
non-optimizing compiler that likes to spill values on stack (instead of keeping them in
regs). Is any way to increase the stack size here? 240 bytes should be enough, after
some tweaks. 300 bytes would make the code more readable than it would be with 240.

Comment 25:

That sure seems like a lot of code to have to run at the startup of
every program. Is it really worth not using the vsyscall page?
Perhaps Ian knows a better way to get at this data? I find it hard to
believe that glibc really parses the ELF header to find this stuff.
Russ

Comment 26:

The situation for ordinary executables on GNU/Linux is different.  They are dynamically
linked.  The dynamic linker is already parsing shared library hash tables to hook up the
symbols referenced by the program.  To the dynamic linker, the VDSO is just another
shared library.  Very little additional effort is required.
But what about statically linked executables?  (I hear you cry.)  They just don't take
advantage of the VDSO, which is OK because glibc and GNU/Linux don't like statically
linked executables anyhow.
That said, I'm fairly sure this code can be simplified somewhat, and I've been making
suggestions on http://golang.org/cl/6454046/ .

Comment 27:

The time to initialize vdso is under 5 us (5E-6 sec).
Yes, glibc also parses ELF. See, for example, glibc/unix/sysv/linux/init-first.c
(unfortunately, VDSO setup is split across tens of files, so it's extremely hard to
follow)

Comment 28:

I was wrong in the reading glibc sources (I have missed #ifdef SHARED in the critical
place).
The following small test confirms Ian's claims:
lala.c:
#include <stdio.h>
#include <time.h>
int main(void) {
  for (int i = 0; i < 1000000000; i++) {
    time(NULL);
  }
}
krasin@krasin-gaz:~$ gcc lala.c -o lala -std=c99 && time ./lala
real    0m3.107s
user    0m3.104s
sys 0m0.000s
krasin@krasin-gaz:~$ gcc lala.c -o lala -std=c99 -static && time ./lala
real    0m39.089s
user    0m7.052s
sys 0m32.010s

Comment 29:

This issue was closed by revision 5287175.

Status changed to Fixed.

Comment 30:

Labels changed: added go1.0.3.