The boot loader should be on the pendrive or you will have problems with the
bootloader when the pendrive is removed. You need to set your BIOS to boot from
usb devices first. That way it will load the usb device if it is in and then go
to the next device if it is removed. If the bootloader is on the hard drive it
becomes messed up when the usb is removed. You can have more than one grub
bootloader.
If your bootloader is not working then you will have to edit it. However, if the
usb device boots and then hangs the bootloader is fine. It is the installation
that you need to work on. You need to find out what is making it hang. To do
that you should remove the words splash --quiet from the grub screen. It will
appear in the window near the bottom. just press the backspace key till those
words are deleted. That way you will see the text scroll as it loads and get an
idea what is causing the system to hang.