If the build server itself was causing the disconnects, it would probably be happening to everyone. But it's not.
Unfortunately we can't be sure from this information what the problem is. But it may be a problem with the Internet connection in your area. If that's the case then I'm afraid it's unreasonable to ask us to investigate or fix the problem. It's simply out of our control and it's impossible for us to help, no matter how much anyone insists we deal with it.
I would instead suggest that the next best place to go to for help would be your ISP. If there is indeed a problem with the network in your area, they may be able to identify what's going on, and if the issue is with their network they would be the ones who can resolve it.