Be aware that the speech synthesis depends on the voice that is being used and so the voice that is "coded" within your browser, which is not the same in various languages.
In French for example, the voice reads negative numbers without issue as far as my test go.
As far as my test go with English as well, the voice does speak a "negative" for a negative result.
Nevertheless, you are making it speak the text of a text object, so in the end, all depend on what that text is.
If you have modified it in any way that would make it NOT a number, that might be the reason you are getting issues.
At any rate, consider the system expressions str() and float() to cast some number into text (string) and some text (string) into a number. That might help out with your issue.
Consider posting your complete project as well, the issue might be somewhere else rather than where you think it is.