I didn't lead you to refactorying, I said enterprise and large  scales are decision parameters. If you have done similar enterprise projects  with nearly the same requirements, Then you may accept the risk of ignoring elaboration and go for refactoring.
Elaboration phase is very important phase that let you to attack architectural risks early. But at some situations PMs and architects could reach to this conclusion that they don't need a large elaboration phase.