Logo Search packages:      
Sourcecode: virtualbox-ose version File versions  Download package

VBoxREM Hacks on AMD64

There are problems with building BoxREM both on WIN64 and 64-bit linux.

On linux binutils refuses to link shared objects without -fPIC compiled code (bitches about some fixup types). But when trying to build with -fPIC dyngen doesn't like the code anymore. Sweet. The current solution is to build the VBoxREM code as a relocatable module and use our ELF loader to load it.

On WIN64 we're not aware of any GCC port which can emit code using the MSC calling convention. So, we're in for some real fun here. The choice is between porting GCC to AMD64 WIN64 and comming up with some kind of wrapper around either the win32 build or the 64-bit linux build.

  1. Porting GCC will be a lot of work. For one thing the calling convention differs and messing with such stuff can easily create ugly bugs. We would also have to do some binutils changes, but I think those are rather small compared to GCC. (That said, the MSC calling convention is far simpler than the linux one, it reminds me of _Optlink which we have working already.)
  2. Wrapping win32 code will work, but addresses outside the first 4GB are inaccessible and we will have to create 32-64 thunks for all imported functions. (To switch between 32-bit and 64-bit is load the right CS using far jmps (32->64) or far returns (both).)
  3. Wrapping 64-bit linux code might be the easier solution. The requirements here are:
    • Remove all CRT references we possibly, either by using intrinsics or using IPRT. Part of IPRT will be linked into VBoxREM2.rel, this will be yet another IPRT mode which I've dubbed 'no-crt'. The no-crt mode provide basic non-system dependent stuff.
    • Compile and link it into a relocatable object (include the gcc intrinsics in libgcc). Call this VBoxREM2.rel.
    • Write a wrapper dll, VBoxREM.dll, for which during REMR3Init() will load VBoxREM2.rel (using IPRT) and generate calling convention wrappers for all IPRT functions and VBoxVMM functions that it uses. All exports will be wrapped vice versa.
    • For building on windows hosts, we will use a mingw32 hosted cross compiler. and add a 'no-crt' mode to IPRT where it provides the necessary CRT headers and function implementations.

The 3rd solution will be tried out first since it requires the least effort and will let us make use of the full 64-bit register set.

Comparing the GCC and MSC calling conventions

GCC expects the following (cut & past from page 20 in the ABI draft 0.96):

    %rax     temporary register; with variable arguments passes information about the
             number of SSE registers used; 1st return register.
             [Not preserved]
    %rbx     callee-saved register; optionally used as base pointer.
    %rcx     used to pass 4th integer argument to functions.
             [Not preserved]
    %rdx     used to pass 3rd argument to functions; 2nd return register
             [Not preserved]
    %rsp     stack pointer
    %rbp     callee-saved register; optionally used as frame pointer
    %rsi     used to pass 2nd argument to functions
             [Not preserved]
    %rdi     used to pass 1st argument to functions
             [Not preserved]
    %r8      used to pass 5th argument to functions
             [Not preserved]
    %r9      used to pass 6th argument to functions
             [Not preserved]
    %r10     temporary register, used for passing a function's static chain
             pointer [Not preserved]
    %r11     temporary register
             [Not preserved]
    %r12-r15 callee-saved registers
    %xmm0-%xmm1  used to pass and return floating point arguments
             [Not preserved]
    %xmm2-%xmm7  used to pass floating point arguments
             [Not preserved]
    %xmm8-%xmm15 temporary registers
             [Not preserved]
    %mmx0-%mmx7  temporary registers
             [Not preserved]
    %st0     temporary register; used to return long double arguments
             [Not preserved]
    %st1     temporary registers; used to return long double arguments
             [Not preserved]
    %st2-%st7 temporary registers
             [Not preserved]
    %fs      Reserved for system use (as thread specific data register)
             [Not preserved]

Direction flag is preserved as cleared. The stack must be aligned on a 16-byte boundrary before the 'call/jmp' instruction.

MSC expects the following:

    rax      return value, not preserved.
    rbx      preserved.
    rcx      1st argument, integer, not preserved.
    rdx      2nd argument, integer, not preserved.
    rbp      preserved.
    rsp      preserved.
    rsi      preserved.
    rdi      preserved.
    r8       3rd argument, integer, not preserved.
    r9       4th argument, integer, not preserved.
    r10      scratch register, not preserved.
    r11      scratch register, not preserved.
    r12-r15  preserved.
    xmm0     1st argument, fp, return value, not preserved.
    xmm1     2st argument, fp, not preserved.
    xmm2     3st argument, fp, not preserved.
    xmm3     4st argument, fp, not preserved.
    xmm4-xmm5    scratch, not preserved.
    xmm6-xmm15   preserved.

Dunno what the direction flag is... The stack must be aligned on a 16-byte boundrary before the 'call/jmp' instruction.

Thus, When GCC code is calling MSC code we don't really have to preserve anything. But but MSC code is calling GCC code, we'll have to save esi and edi.

Generated by  Doxygen 1.6.0   Back to index