Layers have virtually no performance impact unless you:
1) add a shader effect
2) change the opacity from 100%
3) change the blend mode to something other than "normal"
4) enable "force own texture"
All of the above will force the layer to render to an intermediate texture, which has a memory and GPU fillrate overhead. Note parallax has no effect, it does not cause this.