You know if you check the chest/coffer, you'll get two different confirmations.
I'd quote if I could remember what they were, but basically one ends in "..." meaning you can't open it and if you try it'll despawn as an illusion. If it ends with a solid statement and "." then use a key/pick or whatever.