ICE stands for Interactive Connectivity Establishment. They are the servers that help peers connect to each other even if they are behind NAT. These include STUN servers (which try to connect peers directly), and TURN servers (which if peers cannot connect directly, will act as a relay forwarding data between them).
The Multiplayer plugin uses a default list of public STUN servers hosted by Google and Mozilla. You could add your own STUN server, but I don't think there's much point, because the public servers do a good job already.
The most interesting case is to add your own TURN server. According to stats on our signalling server, about 10% of the time peers cannot connect directly due to NAT restrictions. These peers will simply fail to connect because there are no default TURN servers. If you add your own TURN servers then these peers will be able to connect still. The problem is this can degrade the connection quality, since the latency, packet loss etc. will accumulate since data must first go to the TURN server, then be relayed on to another peer. Also this makes TURN servers expensive to run, because you have to pay for the bandwidth of all the game data that passes through it, whereas normal direct peer connections run through their own ISPs and cost you nothing. Still, the option to add your own TURN server is there if you want to.