According to this Chrome issue, it is mainly due to Android devices not supporting low latency playback, or being misconfigured and the device indicating it does not support it when it can. It looks like a difficult issue to resolve, since it's hard to tell between a device that is misconfigured and says it doesn't support low latency audio when it really does, and a device that is correctly configured and truly doesn't support low latency audio.
I've released my first game with audio delays too. It doesn't matter if you put the audio files in the "sound" or "music" folder, the delay is still there once you build the apk.
So is there any workaround to solve this?