Porting code_gen.c and db_fileio.c

(This entry was written offline during a flight from Las Vegas to Milwaukee today, June 21st 2012. I'm continuing the task of merging Waverous into Stunt, or put another way, getting Stunt to be C++ compliant).

Here's something you hope to never see:

-*- mode: compilation; default-directory: "/Users/swain/Sites/projects/moowork/waverous/server/" -*-
Compilation started at Thu Jun 21 07:10:01

make
[ config.h : config.status ]
./config.status
config.status: creating Makefile
config.status: creating config.h
config.status: config.h is unchanged
g++ -m32 -O -DHAVE_CONFIG_H  -c -o code_gen.o code_gen.c
code_gen.c: In function 'Bytecodes stmt_to_code(Stmt*, GState*)':
code_gen.c:1220: error: invalid conversion from 'int (*)(int*, int*)' to 'int (*)(const void*, const void*)'
code_gen.c:1220: error:   initializing argument 4 of 'void qsort(void*, size_t, size_t, int (*)(const void*, const void*))'
make: *** [code_gen.o] Error 1

Compilation exited abnormally with code 2 at Thu Jun 21 07:10:01

The issue here is the standard qsort() function wants a pointer to a function whose signature has two const void pointers, but LambdaMOO's implementation has two int pointers:

     void
     qsort(void *base, size_t nel, size_t width,
         int (*compar)(const void *, const void *));

I never caught this when I ported Waverous to C++ because I never defined BYTECODE_REDUCE_REF. This suggests to me a way of testing the server by compiling it with every possible combination of flags... this would be one hell of a build-and-smoke test. For every combination of compile-time options: compile the server, start it up, log in and do some basic tasks, shut down. I would have caught this long ago if I'd thought of writing such a test.

After some dedicated surfing via my phone at Milwaukee's airport, which does not have free wi-fi (pththth) I found the solution on www.cplusplus.com:

#ifdef BYTECODE_REDUCE_REF
// See http://www.cplusplus.com/reference/clibrary/cstdlib/qsort/ for
// this usage of standard C's qsort() function
static int
bbd_cmp(const void *a, const void *b)
{
  return ( *(int *)a - *(int *)b );
}
#endif    /* BYTECODE_REDUCE_REF */

It's curious and non-obvious to me that the args within the function are cast and that this satisfies the compiler. On a side note I think being able to google our compiler errors and find solutions is an exception to Fred Brooks's "No Silver Bullet" argument. It feels like an order of magnitude increase in productivity to get help so easily.

I'm now past one of the stickier parts of porting: past the TRY/EXECEPT/ENDTRY preprocessor macros in code_gen.c. I had to edit the file by hand and incorporate the needed expansion of the macros because the block of code in question had added blank lines spacing out the code. I would think patch would have a command line flag to account for this situation, but patch can only handle changes in leading and trailing whitespace.

Waverous-to-Stunt patching procedes

I'm making progress patching my fork of Stunt LambdaMOO by generating patches from the git log of Waverous LambdaMOO. There have been the occasional rejections by patch that have puzzled me a bit; I wonder if it's a white space problem? I doubt it. I didn't look terribly close because doing the edits by hand only took a minute. I'm now up against the dreaded TRY/EXCEPT/ENDTRY set of C preprocessor macros. By my commit log in Waverous:

commit c7b9cf521033fa816fbe650c8d00c720e453cfdf
Author: Steve Wainstead 
Date:   Sun Jun 7 19:28:43 2009 +0000

    Replaced the TRY, CATCH and ENDTRY macros with their respective
    code bodies to solve a void pointer problem.

 server/db_file.c |  150 ++++++++++++++++++++++++++++++++++++------------------
 1 file changed, 101 insertions(+), 49 deletions(-)

Geez, that was three years ago? I was hacking away in my little second-bedroom-turned-office in Jackson Heights, Queens; and these days I'm in a high rise in downtown Las Vegas.

Anyway this patch was completely rejected. It's late and my brain is too short on glucose to get the job done. Time to push to master and try to catch up on MOO-talk again. I can't keep up with it lately!

Hacking Waverous changes into Stunt

A few months ago Todd Sundsted, the owner/developer of the Stunt LambdaMOO server code, emailed me inquiring about the effort it took to get LambdaMOO to compile as C++. I ventured it would be interesting to try to apply my changes in Waverous LambdaMOO to a branch of Stunt LambdaMOO. A month or so ago I did fork Stunt on Github and made a local clone. I then looked at the git log of Waverous and chose my first commit that converted some of the files from C to C++. Applying that as a patch to Stunt was about, I'd say, 65% successful: some of the hunks applied, some failed and gave me .rej files.

patching file ast.c
Hunk #9 succeeded at 209 (offset 11 lines).
Hunk #10 succeeded at 220 (offset 11 lines).
Hunk #11 succeeded at 344 (offset 29 lines).
Hunk #12 succeeded at 413 (offset 29 lines).
patching file ast.h
Hunk #1 succeeded at 83 (offset 7 lines).
Hunk #2 succeeded at 112 with fuzz 2 (offset 9 lines).
Hunk #3 succeeded at 190 (offset 11 lines).
patching file db.h
Hunk #1 succeeded at 298 (offset 62 lines).
patching file exceptions.h
Hunk #6 FAILED at 159.
Hunk #7 succeeded at 182 (offset 3 lines).
1 out of 7 hunks FAILED -- saving rejects to file exceptions.h.rej
patching file execute.h
Hunk #1 FAILED at 46.
Hunk #2 FAILED at 94.
2 out of 2 hunks FAILED -- saving rejects to file execute.h.rej

I didn't have oodles of time so I let it slide for a few weeks... my time was being taken up by a Udacity course, running LVDev, campaigning for a political candidate, working on a GreenerBlocks.org project and more! But the itch is there and wants scratching. Tonight I packed my laptop and went over to the Vegas Jelly in /usr/lib above The Beat Coffeehouse. It meets every Thursday night. The Linux, Bitcoin and All Things Open Source group was meeting in the conference room (Hi, Julian!) and I took a seat in the back of the room. I worked while a debate raged about the prospects of Bitcoin but I barely took notice. What I first found was gcc, make and friends were not present on my system anymore. I upgraded my laptop to OS X Lion recently when I participated in an iOS Boot Camp. But I installed XCode; wtf? Googling told me I needed to install the command line tools via Preferences -> Downloads. Completely obvious in retrospect, heh. I renamed configure.in to configure.ac, copied my Waverous version of the file over it and ran autoconf. All good. I ran ./configure:

bash-3.2$ ./configure
checking for bison... bison -y
checking for g++... g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
configure: error: cannot find install-sh, install.sh, or shtool in "." "./.." "./../.."

Another wtf moment. I googled around but didn't really find anything specific enough for my situation; maybe upgrading the autotools? I did port update on autoconf and automake; but still the same error. Finally I just copied install-sh from the automake directory; this is scratch hacking after all, and with version control you never have to say you're sorry. (EDIT: Duh, I forgot to run automake --install-missing, was all).

bash-3.2$ ./configure
checking for bison... bison -y
checking for g++... g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking for gcc... gcc
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C++ preprocessor... g++ -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking whether gcc needs -traditional... no
checking how to run the C preprocessor... gcc -g -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/config.h usability... no
checking minix/config.h presence... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking for library containing strerror... none required
checking for strftime... yes
checking for gcc -g option to accept ISO C99... unsupported
checking for gcc -g option to accept ISO C89... (cached) none needed
checking for gcc -g option to accept ISO Standard C... (cached) none needed
checking sys/bsdtypes.h usability... no
checking sys/bsdtypes.h presence... no
checking for sys/bsdtypes.h... no
checking sys/select.h usability... yes
checking sys/select.h presence... yes
checking for sys/select.h... yes
checking for memory.h... (cached) yes
checking for pid_t... yes
checking for mode_t... yes
checking for size_t... yes
checking whether struct tm is in sys/time.h or time.h... time.h
checking for struct tm.tm_zone... yes
./configure: line 4810: syntax error near unexpected token `sqrt,'
./configure: line 4810: `MOO_HAVE_FUNC_LIBS(sqrt, -lm /lib/libm.a "-static -lm")'

OK, more wtf material... I called it an evening, had a nice discussion with a couple of the Linux guys about LambdaMOO and the problems of getting the masses to use new software, and headed home.

Back in my home office I googled the error a bit, editing out the particulars of the error message to make the search query more generic; somewhere I found one forum post where the reply was "Your aclocal got borked." Ah! Did I not move some of the MOO m4 macros to another file to reduce the size/complexity of configure.ac?

Indeed I did, and I left a helpful log entry in git to that effect.

So I copied acinclude.m4 over from the Waverous project, ran aclocal, ran autoconf and finally ./configure and everything went well. I now have a Makefile that will correctly call g++ for the project. Time to commit my work, push it to github and sleep on it a bit.

What comes next, should I undertake it, is a lot of bare knuckle hacking to resolve conflicts between Waverous's code base and Stunt's. Todd has made a lot of edits to the C sources, I'm sure, and I started from the most recent version of LambdaMOO on Sourceforge. It means I'll have to come to understand a lot of the changes Todd made, and that will be challenging.