Hey otaconnor,
I'm not sure if there's an elegant solution to this issue with the default Physics behavior, but there's a 3rd party physics plugin, based on the Chipmunk physics engine, that specifically has the ability to separate objects into different "collision layers" which won't interact with each other.
You can find it at the post linked below.