Maybe you can set scroll-to on another object, and pin that object to the player most of the time, but when you need to scroll somewhere else pin or move the scroll-to object to another position instead of on the player? Then you can pin it back to the player when you need to move it back...