GRUB overwrites the Master Boot Record of the hard drive, so there is
no "fixing", per se. I needed to rewrite the MBR back to the
original state.

Only problem was that HP ships their computers with a backup of the
original disk on a protected partition, they no longer ship with an
install disk.

So my option was to 1. blow everything off the disk. 2. Find an
install disk and run FixBoot.exe from the repair section of the
install software.

Option 2 worked. I borrowed a friend's Vista 64 install disk and
reparied the MBR, and GRUB is gone.

Unfortunately, the install on the USB drive seems to rely on GRUB to
launch. If I try to boot from the USB drive now, the startup crashes
immediately. So now I have to figure out how to get Ubuntu installed
on the USB hard drive, without messing with the boot on the main
drive, and getting me in trouble with my wife.