Problem is - a date  IS a number. Any number (within reason) can be
formatted as a date. And any date can be formatted as a number.

Ask it for its format ...

=CELL("format",$A4)

if the string  returned starts with "D" then Excel believes it is a date (or
at least, that it is formatted as a date).