Yes skidoo, that is well known and I referred to it in my original post:

I'm writing code now to circumvent this by having the system automatically create an xorg.conf file specifying the"vesa" driver if the framebuffer is enabled and if there is no xdrvr=xxx boot parameter and if there is no graphics driver (nouveau, i915, nvidia, nouveau, radeon) already loaded.

This solution of writing a default xorg.conf to use vesa is very clumsy and I was asking if anyone knew of an alternative way to set the default driver back to vesa without writing an xorg.conf file. Another obvious solution is to not install fbdev or move the fbdev driver out of the way when X starts but this can also cause problems. For now we have decided that writing a default xorg.conf is the lesser of two evils but we still have to get it all working correctly. There have been some surprises.