If your server is incorrectly configured, it will not serve AppCache or .m4a files correctly, which will break both offline caching and audio in IE and Safari. This is a problem with your server and not a C2 bug. To fix it make sure .m4a serves with the MIME type audio/mp4, and .appcache serves with text/cache-manifest.
This and changing the URL fixed the issue for me, thanks!