The escape key should do it, but it doesn't always work properly, so I'll have a look into it.
EDIT: I looked into it again, and wasn't able to get it to fail (not close on pressing "esc"), so it seems stable, but if you can find a case where that doesn't work, let me know!