What is actually wrong with this?
When you press 'P', the side-scroll bar appears?
This only happens, for me, when I shrink my window down. I think it's appending several breaks (<br />) under the canvas. I think it's because the button itself messes with the formatting of the page.
Try it, but instead of using a Button, try just a simple sprite or text, and see if it does the same thing.
Edit: I removed the button, and it didn't have the problem after that.
Rather than it being a 'layers bug', it's more an unintended behaviour of the Button plugin, that only shows itself when it's visible. Still, if that unintended behaviour is able to be cleaned up by Ashley, that'd be awesome.