I'll bump this for you... as I have a similar problem. For you, I'd try increasing the shadow caster height to about 100 or 1000.
My understanding was that 0 was infinite, no matter the height, but apparently not.
For me, I have the camera scaled out slightly, and when a shadow casting object comes in to view, for a split second, it doesn't render the shadow, then suddenly does. It's very jarring to see what's behind a wall for a second.
No idea how to fix this, as none of my tweaks work.