The problem is you only pick the corresponding PortalButton with the overlap condition. The Else defaults to ALL PortalButtons.
Also, there is no reason to duplicate the code three times. This can be simplified by giving both objects the same ID, which you do already for PortalButton (nr), and then the one can pick the other when necessary.