Yeah I don't think you're ever going to get a reliable way to auto-detect and change settings during gameplay - provide an option and maybe use the framerate of the first level to set a default and then leave it.
Yup, in my current project I default to 'auto' settings which basically do just that (except, I'm running my tests prior to the actual gameplay). However, I also have "High, Med, Low" GFX settings.
While I've never had a lot of luck doing 'realtime' tweaks, I certainly think it's possible...it just requires more forethought about how such changes will affect the game, and how to avoid 'ping-pong' effects.