I wanted to get clisp running on my Nokia N810. Unfortunately it seems that no-one had yet ported it to the new ARM EABI (known as "armel" in Debian). Several things changed, such as the alignment of doubles and structure packing and so on, and clisp knows enough about the way its own stack works that this causes trouble.
So I decided to have a go at porting it myself, just to see. I don't know much about assembler or ARM processors or anything, but what the hell. I did actually make some progress, but ultimately failed to get the last piece to work. So I just put my notes up on the web to see if they would be useful.
To my astonishment, they were. Max Lapan carried on where I left off, and soon polished the project off. His code has now been committed to the clisp trunk, and it's just a matter of letting this filter down through the distributions. Hooray!
To commemorate this triumph, I here leave my notes for posterity...
As far as I can tell, simply by finding all files with "arm" in the name or "__arm__" in the code, clisp's architecture-specific code is confined to the ffcall library. This provides a variety of mechanisms for calling C functions outside the usual C idiom. Get ffcall working, and you'll probably get clisp working too. The patches appear to work without modification under both source trees. ffcall is available as a separate library in Debian as libffcall1. It has far fewer build dependencies, so it's best to work with that tarball instead of clisp directly.
There are three basic lumps of code in the library. vacall implements a kind of varargs syntax, so you can define a function that takes a struct argument containing all the real arguments, and then call it as "f(arg1, arg2, arg3);". avcall is similar but in reverse: the caller builds up an argument list and then uses it to call a function with multiple arguments. Both of these bits work with my patch below.
The third part is the trampoline, which implements a kind of closure. That is, it allocates a piece of memory that wraps up an existing, statically defined function with some data. So say you have a function "int add_constant(int x);" defined somewhere, which adds a constant value stored in a global variable to the argument. You can use trampoline to define at runtime a function "add_3" which can be called like any other, which wraps up add_constant together with the value "3". Simulatenously you can define another function "add_7" which wraps up add_constant with the value "7". Same code, different behaviour. This is the bit that I failed to get working.
Under the callback directory there are slightly different versions of vacall and trampoline. These are re-entrant, achieved by using registers to supply the arguments rather than global variables. It was fairly straightforward to get vacall_r working by copying the changes I made to vacall.
My patches so far are available here. To build, you first have to generate the assembly code from the C files. So building under Debian looks like this:
apt-get source libffcall1
wget http://mat.exon.name/logs/ffcall-arm.diff
patch -p0 < ffcall-arm.diff
make -C ffcall-1.10+2.41/ffcall/avcall -f Makefile.devel avcall-arm.S
make -C ffcall-1.10+2.41/ffcall/callback/vacall_r -f Makefile.devel vacall-arm.S
make -C ffcall-1.10+2.41/ffcall/vacall -f Makefile.devel vacall-arm.S
cd ffcall-1.10+2.41/ffcall
./configure
make extracheck
Of course, this will fail at the trampoline. But it should pass the other tests.
The tricky part is the .S files. These are assembly routines generated by gcc. As supplied, they don't compile, probably due to the fact that they are gcc version 2.6.3, circa 1995. So they need to be regenerated. This is done with the also-supplied but not-automatically-run Makefile.devel file, plus some kind of Rube Goldberg machine of sed scripts and macros to take away underscores and put them back again later, for some reason. I don't understand it, and so my patch hacks things up to bypass most of it.
There are actually two trampolines, one inside trampoline and one inside callback/trampoline_r. The former relies on a global variable to hold the data, and so it'll break if the same trampoline is used more than once at the same time. The latter, reentrant one corrects this by storing the data in the temporary register, "ir" aka "r12".
As far as I can tell, the non-reentrant version can't have worked for a long time, because the interface seems to have changed. Instead of a pointer to a function pointer, the macro now takes just a function pointer, and instead of a pointer to the data, the macro takes the data itself. However, the reentrant version appears to be correct and up-to-date. I just can't get it to work.
That is to say, the non-reentrant version seems to work under scratchbox. But as soon as I move it to the actual device, it segfaults. The reentrant version fails in any case. The problem is, it works fine if I step through it instruction-by-instruction in gdb, which is pretty maddening. I've tried debugging it by adding instructions to shift temporary values into spare registers such as r2 and r3, but can't get it to produce anything comprehensible.
Unfortunately, I'm out of time (and patience), so I have to give up. But the problem that remains is fairly well constrained, so maybe someone with more experience of debugging ARM processors will hear my plea for help and finish this off.
The rest of this file is the raw notes I made while trying to get it to work. They probably make very little sense.
For clisp, I'm doing what I probably should have done first, which is replace "-DSAFETY=3" with "-DNO_GENERATIONAL_GC" in makemake.in. And now I can try ./configure. This is 2.41, by the way.
Same error as always.
./lisp.run -B . -N locale -E 1:1 -Efile UTF-8 -Eterminal UTF-8 -norc -m 1800KW -x "(and (load \"init.lisp\") (sys::%saveinitmem) (ext::exit)) (ext::exit t)"
qemu: uncaught target signal 4 (Illegal instruction) - exiting
make: *** [interpreted.mem] Error 252
[sbox-CHINOOK_ARMEL: ~/clisp/clisp/clisp-2.41.orig/src] > mv .gdbinit disabled.gdbinit
[sbox-CHINOOK_ARMEL: ~/clisp/clisp/clisp-2.41.orig/src] > gdb ./lisp.run
(gdb) run -B . -N locale -E 1:1 -Efile UTF-8 -Eterminal UTF-8 -norc -m 1800KW -x "(and (load \"init.lisp\") (sys::%saveinitmem) (ext::exit)) (ext::exit t)"
Starting program: /home/mexon/clisp/clisp/clisp-2.41.orig/src/lisp.run -B . -N locale -E 1:1 -Efile UTF-8 -Eterminal UTF-8 -norc -m 1800KW -x "(and (load \"init.lisp\") (sys::%saveinitmem) (ext::exit)) (ext::exit t)"
Don't know how to run. Try "help target".
Note that officially, clisp depends on gcc-4.1. That could well be significant. So maybe I should have another go at that. That won't work, because installing it depends on a newer version of libc. Hmm, or not. What does it depend on? autogen dejagnu (>= 1.4.3) expect-tcl8.3 gperf (>= 3.0.1) bison (>= 1:2.3) libmpfr-dev realpath (>= 1.9.12) chrpath make (>= 3.81) graphviz (>= 2.2) gsfonts-x11. I reckon I have a go compiling it locally. It wants to compile java, and I don't want it to. According to this:
[sbox-CHINOOK_ARMEL: ~/clisp/gcc-4.1/local/gcc-4.1-4.1.1ds2.orig/gcc-4.1.1/gcc] > grep language= */config-lang.in
ada/config-lang.in:language="ada"
ada/config-lang.in:boot_language=yes
cp/config-lang.in:language="c++"
fortran/config-lang.in:language="fortran"
java/config-lang.in:language="java"
objc/config-lang.in:language="objc"
objcp/config-lang.in:language="obj-c++"
treelang/config-lang.in:language="treelang"
I don't want any of those. Oh, apart from c. "./configure --enable-languages=c".
After a long time it failed like this:
ar rc ./libgcc.a libgcc/./_udivsi3.o libgcc/./_divsi3.o libgcc/./_umodsi3.o libgcc/./_modsi3.o libgcc/./_dvmd_lnx.o libgcc/./_muldi3.o libgcc/./_negdi2.o libgcc/./_lshrdi3.o libgcc/./_ashldi3.o libgcc/./_ashrdi3.o libgcc/./_cmpdi2.o libgcc/./_ucmpdi2.o libgcc/./_floatdidf.o libgcc/./_floatdisf.o libgcc/./_fixunsdfsi.o libgcc/./_fixunssfsi.o libgcc/./_fixunsdfdi.o libgcc/./_fixdfdi.o libgcc/./_fixunssfdi.o libgcc/./_fixsfdi.o libgcc/./_fixxfdi.o libgcc/./_fixunsxfdi.o libgcc/./_floatdixf.o libgcc/./_fixunsxfsi.o libgcc/./_fixtfdi.o libgcc/./_fixunstfdi.o libgcc/./_floatditf.o libgcc/./_clear_cache.o libgcc/./_enable_execute_stack.o libgcc/./_trampoline.o libgcc/./__main.o libgcc/./_absvsi2.o libgcc/./_absvdi2.o libgcc/./_addvsi3.o libgcc/./_addvdi3.o libgcc/./_subvsi3.o libgcc/./_subvdi3.o libgcc/./_mulvsi3.o libgcc/./_mulvdi3.o libgcc/./_negvsi2.o libgcc/./_negvdi2.o libgcc/./_ctors.o libgcc/./_ffssi2.o libgcc/./_ffsdi2.o libgcc/./_clz.o libgcc/./_clzsi2.o libgcc/./_clzdi2.o libgcc/./_ctzsi2.o libgcc/./_ctzdi2.o libgcc/./_popcount_tab.o libgcc/./_popcountsi2.o libgcc/./_popcountdi2.o libgcc/./_paritysi2.o libgcc/./_paritydi2.o libgcc/./_powisf2.o libgcc/./_powidf2.o libgcc/./_powixf2.o libgcc/./_powitf2.o libgcc/./_mulsc3.o libgcc/./_muldc3.o libgcc/./_mulxc3.o libgcc/./_multc3.o libgcc/./_divsc3.o libgcc/./_divdc3.o libgcc/./_divxc3.o libgcc/./_divtc3.o libgcc/./_eprintf.o libgcc/./__gcc_bcmp.o libgcc/./_divdi3.o libgcc/./_moddi3.o libgcc/./_udivdi3.o libgcc/./_umoddi3.o libgcc/./_udiv_w_sdiv.o libgcc/./_udivmoddi4.o
/scratchbox/compilers/cs2005q3.2-glibc2.5-arm/bin/sbox-arm-linux-ar: libgcc/./_udivsi3.o: No such file or directory
make[3]: *** [libgcc.a] Error 1
make[3]: Leaving directory `/home/mexon/clisp/gcc-4.1/local/gcc-4.1-4.1.1ds2.orig/gcc-4.1.1/host-arm-unknown-linux-gnu/gcc'
make[2]: *** [libgcc.a] Error 2
make[2]: Leaving directory `/home/mexon/clisp/gcc-4.1/local/gcc-4.1-4.1.1ds2.orig/gcc-4.1.1/host-arm-unknown-linux-gnu/gcc'
make[1]: *** [all-gcc] Error 2
make[1]: Leaving directory `/home/mexon/clisp/gcc-4.1/local/gcc-4.1-4.1.1ds2.orig/gcc-4.1.1'
make: *** [all] Error 2
I'm quite disappointed by that. Oh, it's possible that I ran out of space. Right, I won't try to compile xfree86 at the same time as gcc4.1. xfree86 is definitely an overnight job. Just gcc4.1
Note that the floating point problems are noticed here, which links to a more recent report of clisp problems here.
Interesting. vacall-arm.S was generated by gcc 2.6.3 from vacall-arm.c. ldfeqs is only mentioned in the .S file, not the .c file. So presumably if I regenerate the .S file, it will work.
[sbox-CHINOOK_ARMEL: ~/clisp/clisp/clisp-2.41/ffcall/vacall] > make -f Makefile.devel vacall-arm.S
gcc -V 2.6.3 -b arm-acorn-riscix -O2 -fomit-frame-pointer -DHAVE_LONG_LONG -D__arm__ -S vacall-arm.c -o vacall-arm.s
sbox-arm-linux-gcc: `-V' must come at the start of the command line
make: *** [vacall-arm.S] Error 1
GCC failed exactly the same way as before. So it's not a disk space thing.
So why use -V or -b at all?
[sbox-CHINOOK_ARMEL: ~/clisp/clisp/clisp-2.41/ffcall/vacall] > gcc -O2 -fomit-frame-pointer -DHAVE_LONG_LONG -D__arm__ -S vacall-arm.c -o vacall-arm.s
vacall-arm.c:23: warning: call-clobbered register used for global register variable
vacall-arm.c:24: warning: call-clobbered register used for global register variable
vacall-arm.c:26: warning: register used for two global register variables
vacall-arm.c: In function `__vacall':
vacall-arm.c:124: error: insn does not satisfy its constraints:
(insn 437 205 206 22 (set (reg/v:SF 16 f0 [ fret ])
(reg:SF 3 r3)) 155 {*arm_movsf_soft_insn} (nil)
(nil))
vacall-arm.c:124: internal compiler error: in reload_cse_simplify_operands, at postreload.c:391
Please submit a full bug report,
with preprocessed source if appropriate.
Send email to arm-gnu@codesourcery.com for instructions.
The warning lines are (line numbers added):
23: register __vaword iret __asm__("r0");
24: register __vaword iret2 __asm__("r1");
25: register float fret __asm__("f0");
26: register double dret __asm__("f0");
The error line is the last closing brace of the only function in this file, __vacall. Hmm:
/* MAGIC ALERT! * This is the last struct on the stack, so that * &args + 1 == &return_address == &firstword - 1. * Look at the assembly code to convince yourself. */
I don't think that's the problem though. Clearly, it's something to do with this "f0". Read this for more information.
Note that this code is all from libffcall. So I could start by tackling that. But I think they're exactly the same version. Yes, same problem.
So my theory is that if I remove all the register stuff around that, the compiler will try to do something sensible for the architecture. I've done that, and generated a .s file. Let's see if it compiles. This line in the generated file should help:
.fpu softvfp
Well, that helped configure get over its hurdle. But there are other files with the same problem. vacall_r/vacall-arm.s appears to be hand-written, not auto-generated.
Ach, I think I was looking at the wrong file before. It's the one in vacall_r that's actually the problem. Nevertheless. If ffcall manages to build, then I may have achieved something. And I haven't: it fails exactly the same way. Ah, I think I have to "mv vacall-arm.s vacall-arm.S". No, that doesn't help either. You know what? That .s file is being auto-generated each time. And of course it's being done with that -b arm-acorn-riscix flag. So let's just remove those flags and see what happens. By the way, I'm building with dpkg-buildpackage here, so it's possible that this is deliberately asking for them to be rebuilt. Even that didn't help. It builds the file like this:
make[2]: Entering directory `/home/mexon/clisp/libffcall/ffcall-1.10+2.41/ffcall/vacall'
gcc -E `if test false = true; then echo '-DASM_UNDERSCORE'; fi` ./vacall-arm.S | grep -v '^ *#line' | grep -v '^#' | sed -e 's,% ,%,g' -e 's,//,@,g' -e 's,\
$,#,g' > vacall-arm.s
gcc -x none -c vacall-arm.s
Ah. That might mean that I trashed my own file. Yes, they're completely different. Lucky I saved a copy! No, still the same problems. I don't believe it, I still have a file generated by gcc 2.6.3. This is very strange.
I think configure creates src/callback/vacall-arm.s by copying it from somewhere, and therefore I have to fix things before running configure for the first time. There are two vacall-arm.c files, and I need to do them both.
[sbox-CHINOOK_ARMEL: ~/clisp/clisp/local/clisp-2.41.orig] > find . -name \*vacall-arm\*
./ffcall/callback/vacall_r/vacall-arm.c
./ffcall/callback/vacall_r/vacall-arm.S
./ffcall/vacall/vacall-arm.c
./ffcall/vacall/vacall-arm.S
Aw man. The c files are identical. The S files are just a little bit different. Well, I reckon I edit the last two, and then copy the results of compiling to .S into the first one.
OK. Now configure fails like this:
gcc -g -O2 -I. -I../../ffcall/avcall -c ../../ffcall/avcall/minitests.c
/bin/sh ./libtool --mode=link gcc -g -O2 -x none minitests.o libavcall.la -o minitests
gcc -g -O2 -x none minitests.o -o minitests ./.libs/libavcall.a
./minitests > minitests.out
qemu: uncaught target signal 4 (Illegal instruction) - exiting
That's a different library, avcall. Same deal I think, but with avcall-arm.S. Except until configure, it only exists in one place, ffcall/avcall.
No, I can't just copy over those files for vacall and vacall_r, they have different function name. They really are different. Oh, no they're not, same input c file remember? But they're otherwise so similar that I reckon one is a branch of the other with a different function name. So I think what I do is change the name of the function in ./ffcall/callback/vacall_r/vacall-arm.c and recompile both files.
diff ./ffcall/vacall/vacall-arm.S ./ffcall/callback/vacall_r/vacall-arm.S | less 12,17c12,14 < LC0: < .word C(vacall_function) < .align 0 < .global C(__vacall) < DECLARE_FUNCTION(__vacall) < C(__vacall:) --- > .global C(__vacall_r) > DECLARE_FUNCTION(__vacall_r) > C(__vacall_r:)
Hmm, it seems that in fact it's the other way round. Except that in vacll_r it doesn't compile. So what I do is compile it in the other directory, copy it over, and change the name.
Still failed in exactly the same place. I thought at first I'd forgotten to do avcall, but no, it's there.
Ah, interesting, running minitests by hand generates heaps of correct output, before this:
long long f(float,long long,int):(1.4,0x35c6f707fffffffa,0xe)->0x35c6f70800000009
long long f(float,long long,int):(1.4,0xe35c6f707,0x9999999a)->0xdcf6090a2
void* f(void*,double*,char*,Int*):(0x14d2e,0x14da0,0xc5dc,0x14c98)->0x14da1
void* f(void*,double*,char*,Int*):(0x14d2e,0x14da0,0xc5dc,0x14c98)->0x14da1
Int f(Int,Int,Int):({1},{2},{3})->{6}
Int f(Int,Int,Int):({1},{2},{3})->{6}
J f(J,int,J):({47,11},2,{73,55})->{120,68}
J f(J,int,J):({11,2},73,{55,-1243263332})qemu: uncaught target signal 11 (Segmentation fault) - exiting
Man, this was really written by the kind of person my parents warned me about. I'm going to have to figure this out slowly and carefully. tests.c is the file, by the way.
OK, reading avcall-arm.c, I realise that this is completely broken for armel.
If I want to fix this, here's what I have to do. Read up on the differences in the new ABI. Chase up as much documentation as I can find. Read the various comments in the existing functions which explain how function calls work and in particular how structs are packed. Then write a whole new avcall-armel.c file to do the job. It's a fairly constrained problem: those files aren't long. But it will be very hard to get my head around.
With the new ABI, default structure packing changes, as do some default data sizes and alignment (which also have a knock-on effect on structure packing). In particular the minimum size and alignment of a structure was 4 bytes. Under the EABI there is no minimum and the alignment is determined by the types of the components it contains. This will break programs that know too much about the way structures are packed and can break code that writes binary files by dumping and reading structures.
You know, I'm pretty confident I can get at least structs working.
I should definitely do all of this in ffcall before moving on to clisp itself.
There's some chatter on internet tablet talk.
Note that ffcall, unlike clisp, has build depends of only debhelper and autotools-dev. So I might even be able to fix this on the road.
After a night of hacking, I have solved one alignment problem in avcall. It looks like this:
#if defined(__arm__)
#define av_float(LIST,VAL) \
(++(LIST).aptr > __av_eptr(LIST) \
? -1 : (((float*)(LIST).aptr)[-1] = (float)(VAL), 0))
#define av_double(LIST,VAL) \
fprintf(out, "double aptr %x args %x\n", (LIST).aptr, (LIST).args ); \
fflush(out); \
(((LIST).aptr += 2 + (((int)((LIST).aptr) % 8) ? 1 : 0)) > __av_eptr(LIST) \
? -1 : \
((LIST).tmp._double = (double)(VAL), \
(LIST).aptr[-2] = (LIST).tmp.words[0], \
(LIST).aptr[-1] = (LIST).tmp.words[1], \
0)); \
fprintf(out, "double aptr %x args %x\n", (LIST).aptr, (LIST).args )
#endif
Or at least, you remove the prints. Then the avcall-arm.s file has to be generated, and I'm still not clear on that point, although I managed it earlier. No changes to avcall-arm.c, just generate the file. Once that's done, then build. But you can't build, because the same thing has to be done to the vacall-arm.s file and probably the _r directory as well. Nevertheless, this represents progress.
Here are the commands I run to get things working:
cp -r ffcall-1.10+2.41.orig ffcall-1.10+2.41
patch -p0 < my-third-patch
make -C ffcall-1.10+2.41/ffcall/avcall -f Makefile.devel avcall-arm.S
make -C ffcall-1.10+2.41/ffcall/callback/vacall_r -f Makefile.devel vacall-arm.S
make -C ffcall-1.10+2.41/ffcall/vacall -f Makefile.devel vacall-arm.S
cd ffcall-1.10+2.41/ffcall
./configure
I have figured out that the problem lies with struct return values, while struct args are fine. That's probably got something to do with __av_start_struct3 and friends.
Beyond that, I have little understanding of how it actually works. In particular, aptr only seems to be used for the fourth and fifth elements. The first three must be being sent in registers somehow. Oh, OK, let's say it's like this. Up to four words can be sent in registers. There's a two-word struct, then an int, and then the third argument which is also a two-word struct doesn't fit in registers, so it's sent on the stack.
I also think that we actually have up to six registers that can be used for arguments, but we only appear to be using four. But it works, so I guess it works.
It doesn't help that ARM's "information center" appears to be completely broken under Firefox.
This is the comment from the existing implementation:
To return a structure, the called function copies the return value to the address supplied in register "%r0".
I do not appear to have __AV_PCC_STRUCT_RETURN set, but I do appear to have the other two (small and gcc).
It is possible that it is precisely this which is wrong. It shouldn't be PCC, but "normal".
I don't know what the provenance of this is, but:
When compiling functions that return structures or unions, GCC output code normally uses a method different from that used on most versions of Unix. As a result, code compiled with GCC cannot call a structure-returning function compiled with PCC, and vice versa.
The method used by GCC is as follows: a structure or union which is 1, 2, 4 or 8 bytes long is returned like a scalar. A structure or union with any other size is stored into an address supplied by the caller (usually in a special, fixed register, but on some machines it is passed on the stack). The target hook `TARGET_STRUCT_VALUE_RTX' tells GCC where to pass this address.
By contrast, PCC on most target machines returns structures and unions of any size by copying the data into an area of static storage, and then returning the address of that storage as if it were a pointer value. The caller must copy the data from that memory area to the place where the value is wanted. GCC does not use this method because it is slower and nonreentrant.
On some newer machines, PCC uses a reentrant convention for all structure and union returning. GCC on most of these machines uses a compatible convention when returning structures and unions in memory, but still returns small structures and unions in registers.
You can tell GCC to use a compatible convention for all structure and union returning with the option `-fpcc-struct-return'.
Two pages from Microsoft. Including: "ARM's compiler returns some 4-byte structures in R0. CLARM and CLTHUMB always return structures in the calling function's stack space to which the first argument points."
The segfault I'm getting certainly makes it look like the calling function is trying to store the return value in a pointer supplied in a register, but I'm not doing that.
See, even though I only have an 8 byte return value here, I can't see anywhere in the code where it... hold on!
if (l->rsize == 2*sizeof(__avword)) {
((__avword*)l->raddr)[0] = i;
((__avword*)l->raddr)[1] = iret2;
As I was saying, I can't see anywhere in the code where it does the "address in a register" thing. But there it is right there! It's setting a return address instead of just returning.
No, wait, that's irrelevant. This part of the code isn't the problem. The problem is that the called function is expecting raddr to be stored in some register, and that isn't being done. so the only question is: which register?
I just compiled a trivial example to assembler and figured out what it's doing. I'm pretty sure the address for the return value goes in r0. That might explain why the arguments appear to be one word to the left of where they should be. In fact, it would totally explain it. So all I have to do is add that to the args list as effectively an extra arg.
And it already has infrastructure for that. The only thing to do is make it do that if it's a struct always, regardless of whether it fits in the registers or not. In fact, this seems to be controlled by __AV_GCC_STRUCT_RETURN. Hmm, no, a four-byte struct doesn't get returned that way. A six-byte struct does. So let's try removing the "|| ((TYPE_SIZE) == 8" part and see what happens. No difference. then again, that might be because I was editing the i386 section. Ah, yes exactly:
avcall.h.in: __AV_GCC_STRUCT_RETURN = 1<<2, /* consider 8 byte structs as small */
So how do I convince that to, like, not? Well, I just added a "!defined(__arm__)". And all my tests now pass. How nice.
If I turn on the extra tests, a couple seem wrong:
Char f(Char,double,Char):({'A'},0.2,{'C'})->{'B'}
Char f(Char,double,Char):({''},0.2,{''})->{''}
T f(T,char,T):({"the"},' ',{"fox"})->{"box"}
T f(T,char,T):({"��"},'/',{" "})->{""}
X f(B,char,double,B):({0.1,{1,2,3}},'',0.3,{0.2,{5,4,3}})->{"return val",''}
X f(B,char,double,B):({2.65022e-314,{2,3,0}},'3',0.2,{8.48798e-314,{3,0,902231815}})->{"return val",'3'}
The common factor: chars. So it's a bit early to declare victory yet. Just to clarify: the fix was in avcall.h, and probably needs to really be applied to avcall.h.in:
#if defined(__GNUC__) && !defined(__arm__) __AV_GCC_STRUCT_RETURN | #endif
I managed to knock off one more bug. The problem was that after the last one-byte-long struct has been pushed onto the arguments list, the arguments list is left being an odd size. This gets rounded down by "arglen = l->aptr - l->args", which means that the last argument never gets written to the argframe. To solve, it round up when calulating the arglen:
/* Make the argument count round up, in case just one byte got added
to the argument list or something */
l->aptr = (__avword*)((void*)l->aptr + sizeof(__avword) - 1);
int arglen = l->aptr - l->args;
__avword i;
But I'm still not done. The remaining two tests both fail as well.
Oh, nearly forgot to mention, the whole reason for that second bug popping up was what I did to fix the first one. Here it is, complete with debugging statements and context:
#if defined(__m88k__)
#define __av_struct(LIST,TYPE,TYPE_SIZE,TYPE_ALIGN,ASSIGN,VAL) \
(((LIST).aptr = \
(__avword*)(((((__avword)(LIST).aptr+(TYPE_SIZE)+(TYPE_ALIGN)-1) & -(long)(TYPE_ALIGN))\
+sizeof(__avword)-1) & -(long)sizeof(__avword))) \
> __av_eptr(LIST) \
? -1 : (ASSIGN(TYPE,TYPE_SIZE,TYPE_ALIGN,(void*)((__avword)(LIST).aptr-(TYPE_SIZE)),VAL),\
0))
#endif
#endif
#if defined(__arm__)
#define __av_struct(LIST,TYPE,TYPE_SIZE,TYPE_ALIGN,ASSIGN,VAL) \
fprintf( out, "Yes, we're really here. args %x aptr %x\n", (LIST).args, (LIST).aptr ); \
(((LIST).aptr = (__avword*)(((long)(LIST).aptr+(TYPE_SIZE)+TYPE_ALIGN-1) & -(long)TYPE_ALIGN)) \
> __av_eptr(LIST) \
? -1 : (ASSIGN(TYPE,TYPE_SIZE,TYPE_ALIGN,(void*)((__avword)(LIST).aptr-(TYPE_SIZE)),VAL),\
0)); \
fprintf( out, "After, aptr %x\n", (LIST).aptr)
#endif
#if defined(__m68k__) || defined(__convex__)
/* Structures are passed as embedded copies on the arg stack.
*/
#define __av_struct(LIST,TYPE,TYPE_SIZE,TYPE_ALIGN,ASSIGN,VAL) \
(((LIST).aptr = (__avword*)(((long)(LIST).aptr+(TYPE_SIZE)+sizeof(__avword)-1) & -(long)sizeof(__avword))) \
> __av_eptr(LIST) \
? -1 : (ASSIGN(TYPE,TYPE_SIZE,TYPE_ALIGN,(void*)((__avword)(LIST).aptr-(TYPE_SIZE)),VAL),\
0))
#endif
#if (defined(__sparc__) && !defined(__sparc64__)) || (defined(__powerpc__) && !defined(__powerpc64__) && !(defined(_AIX) || (defined(__MACH__) && defined(__APPLE__))))
The explanation is basically that it is the same as the m68k and convex ones below, but is uses the specified alignment as the offset instead of assuming that everything is aligned according to __avword. Everything still demands to be aligned on four-byte boundaries, but this strategy puts the char into the first byte instead of the last, which seems to be what the ABI demands.
I think I marginally improved things. The three-byte struct return value does get passed in a register. I changed it to rather than being specific sizes, anything less than eight bytes goes on the stack. But I don't know if it's less than 8 or less than 4, so I added a test to check for that.
That previous case turned out to be unnecessary - I can use the existing one for i386 and other little-endian friends.
OK, good, avcall passes its tests. vacall segfaults, so I'm about one-third of the way through. I'd better check in my patch.
vacall fails when trying to return a value. I expect this is due to the following:
/* MAGIC ALERT! * This is the last struct on the stack, so that * &args + 1 == &return_address == &firstword - 1. * Look at the assembly code to convince yourself. */
I may well do that, thanks.
Yesterday I spent ages staring at the assembly code for my test trying to figure out how it's trying to get hold of the return value address, and failed. But hopefully something is gelling in my mind.
One interesting thing is the difference between compiling with -fomit-frame-pointer and without. With that flag, it uses "bx lr". Without, it uses more complicated stuff, there's no bx instruction. In fact, tests is compiled without the flag. But vacall-arm.S is compiled with the flag.
Reading this hint, this is probably deliberate. That is, it's only the __vacall function which should have no frame pointer, because it's the called function that has to understand __vacall's layout.
I'm also referencing some Microsoft documentation about the ARM stack layout and the Wikipedia page about stack layout. Also this summary of the ARM instruction set, which unfortunately is a bit cryptic for me.
Right, I finally managed to do something useful with vacall. The secret seems to be that it deliberately screws with its stack. It knows that anything past the fourth argument will get passed on the stack. So it simply goes back four places, scribbles its four argument registers over whatever is there, and calls the function. It saves away the return address, but not the other three, even though it has a structure there to hold the new values. So I simply changed things so that it stores all four stack positions before scribbling over them, and restores them all afterwards. Note that the function which gets called only gets an opaque pointer to some area of memory, which in this case happens to be this hacked up region of stack. It won't do any damage outside this region though.
Now I'm left with the usual problems, similar to all of the ones for avcall. Hopefully I can get through them fairly quickly.
I've done two, both just by moving __arm__ to something more appropriate. The second was that the __va_arg_adjusted thing is little endian instead of big endian. That solves the char problems. The second is that __arm__ moves from the null version to the "__VA_alignof(double) > sizeof(__vaword)" version. Which happens to be the case.
And the third is that __va_arg_longlong moves from the "(at most) word-aligned" bit to "have alignment 8" bit.
So now it's just that structures are broken.
I fixed one set of those problems by adding the __va_start_struct_return and __va_start_struct1 macros for ARM. Now my only problem is small structs that get returned. Large structs work fine.
Right, vacall is done and checked in. Next thing to fail is trampoline. Conveniently, there's a step-by-step guide to porting it. After that I'll have to do trampoline and vacall in callback. So I've done two directories and I have three to go. So call that 90% done I guess.
The thing to do with the trampoline is start small. Write some machine language that moves data from one address to another via a register and then returns. See if I can execute it. That is all.
OK, that's one thing explained: configure does test for the existence of mprotect, but it never checks that it can allow execution. That's no good.
I've managed to load and execute some code from memory by mmaping it. Here's the code:
#include#include #include #include #include #include #include char* buffer; int (*funptr)(int); int fun(int x) { return x + 2; } int main() { int pagesize = getpagesize(); int fd = open("memory", O_RDWR | O_CREAT); buffer = (char*)mmap(NULL, pagesize, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_SHARED, fd, 0); if (buffer == MAP_FAILED) { fprintf(stderr, "Couldn't map memory: %d\n", errno); return 0; } fflush(stdout); funptr = (int (*)(int))buffer; return (*funptr)(3); }
Compile it. Dissassemble fun. Produces this: