I'm not sure what you are trying to do..
A "normal" physical joystick also had/has freedom of movement in all direction (although there were only four buttons underneath being pushed)..
As an example, what would you like to happen when first dragging down and then dragging left?
Should the player first go back up to the middle before going left?