If you're interested in another physics behavior I've been working on one based on chipmunk2d here:
It already has more features than the bundled physics behavior, although I'm unsure how it compares performance wise. It's beta but I've fixed every bug I've found so far, however it just needs more testing.
Comparing it to your list it gets about half of them right now.
1a It currently has collision groups and layers
1 not currently, but it's a simple change to get it working.
2 It has every joint chipmunk2d has to offer.
3 Not currently. I can add segments with a radius as another collision shape so you can just use objects. Other than that I'm not sure of a good workflow, but I'm open to ideas.
4a not directly but a pivot and gear joint together should work. It is possible to give bodies multiple shapes in chipmunk.js but I haven't come up with a good workflow, so only one shape is used.
4 It can do ray tracing with line segment queries currently.
5 Not yet. I'll add it to my todo.