Yup. That was one of the first ideas I tried - I even tested it with a 3DBox as well - works fine as long as you do not alter the pitch/roll/yaw values, which I sadly need to do...
I'll either live with the changing FOV or just render the view at the highest supported res and downscale it, though it's a huge waste of resources...
Thank you for your help.