When doing this, it's a good practice to know when, exactly, you are picking things up, and have it always turn off the physics behavior while dragging, but back on while not. It seems to work for me more often than not, and I do it with multiple behaviors.