There is often a key to tap during boot that allows you to select a
one-time change in the boot device - [F12] is a common one but have
seen [F11], [F10], [F9] and [F8] used for this !! Sometimes the option
needs turning on in BIOS too.

Next thing to look at is the Hard Drives listed and some BIOS's need to
to change the USB device to the first in the list but it will only
show this when the thing is plugged in.

A system that new *will* be able to boot from a USB device, just need
to work out the way yours does it