The worst thing that will happen is the resistors will cook and open up. As you said this would require leaving the amp on with no adapter and the volume pot up pretty far. If you wanted some protection from this, you could plug in the 1/4" adapter and carefully paint some clear nail polish on the portion of the 1/4" jack contacts that short closed when you unplug the headphones. This will leave the 8 ohm resistors in the circuit even without the adapter in place. You may also be able to use a few small squares of electrical tape to the same effect.
