Is there any reason why you are using a grid of sprites and not a tilemap? Tilemap would be massively better for performance, easier to update and save, the save file will be much smaller etc.
If you must to use the sprites, try hierarchy instead of Pin behavior. It may be faster and you won't have to pin/unpin all the time. Another option is to scroll a layer with the sprites, this way you won't have to pin anything at all.