To clarify upon the slightly confusing mix of answers here, first up; v7-A1:
In the standard configuration the only exception that can interrupt a multiple access instruction is a synchronous data abort, so they are effectively atomic in terms of interrupts (though not in terms of memory accesses).
This is not true, however, if the low-latency interrupt configuration is supported and has been enabled. Here IRQs, FIQs and asynchronous aborts can also interrupt the instruction. To quote the v7-A ARM ARM on this:
ARM deprecates any software reliance on the behavior that an interrupt or asynchronous abort cannot occur in a sequence of memory transactions generated by a single load or store instruction that accesses Normal memory.
Note
A particular case that has shown this reliance is load multiples that load the stack pointer from memory. In an implementation where an interrupt is taken during the LDM, this can corrupt the stack pointer.
An instruction interrupted this way will be abandoned and, if returned to, execution restarted form the beginning (thus for stores the lower addresses may see two writes).
Secondly v7-M2, with its wacky exception model:
Here it's low-latency all day every day. Exceptions can always be taken during multiple access instructions, but the architecture allows (in certain conditions) for continuing execution from the point of interruption as per the suggestion in the question. Abandon-and-restart behaviour is also permitted as an alternative, and is the only option for non-continuable instruction/exception combinations.
[1] sections A3.5.3 and B1.8.12 of the ARMv7-A ARM (DDI0406C.b)
[2] section B1.5.10 of the ARMv7-M ARM (DDI0403D)
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…