Actually I found a solution to that:
RegexReplace(TextInput.Text, "[^0123456789-]", "g", "")
So I match any character not included in [].
The problem now is that users can type the "-" symbol as well.
So I need to figure out how to allow users to type only numbers, and the system format the phone number.
e.g. (555)555-55555