I know my post wont help a lot, but what I did was stick to Letterbox scale and just let the players handle black bars on sides.
If nothing else works I would say go with this, especially if for desktop computers, as black bars are more fogiving there than on mobile.
Yeah, I'm not working with mobile, which is why I have a switch to allow the player to change the scaling mode to whatever they please, rather like tools like Dosbox and ScummVM allow, for instance. I just need to be able to calculate the window width for the users screen to use as a size for the layer scale, to keep the HUD the full width of the window in "scale outer".