I upgraded my project to 5, so I probably won't be going back. Also, two of my books use Unity version 5 as well (Unity in Action, and Unity A.I. Game programming), so I'll be sticking with 5. I'll be starting the A.I. book in a day or two.
One of my gripes is that projectors don't work. For the top-down game I'm working on, it would have been useful to project a selection circle around characters or bases (like Company of Heroes), but it isn't reliable enough. In fact, I think it's completely broken at the moment.
I forget what else I don't like, but I'm a bit tired at the moment.
One thing I did notice with 5.4 is that selecting objects in the viewport seems more responsive. I've had issues before with selecting specific objects, but it seems to be better now (so far at least).
I don't think they've fixed mixed lighting not casting shadows on dynamic objects, but I'm just using one real-time directional light at the moment since I've my game outdoors. So, this doesn't really affect me.
I hope they continue to iron bugs out. I don't want to have to go back to Unreal, even if I think it had a better engine. Unity is just so much easier for me now that I know how it works.
Also for android the performance is worst in the 5 series because of some canvas memory leak ( not sure exactly) and now it puts extra permissions like READ STATE PHONE IDENTITY which will have users uninstalling in seconds.