If the peer position bounces back to the original position, that means that the host either not receiving the peer inputs or not properly applying the peer's inputs to movement.
The host is authoritative - if the peer events move an object but the host does not, the host will correct the peer.