Despite the workarounds it still makes sense to me to have a toggle which allows objects on an invisble layer not to respond to inputs. There would be no reason i can see where this would be an issue. I know this is an old post but i think its still a valid point.