So I finally kind of managed to "fix it"...
My screen refresh rate was 165Hz, and I need the stepping mode for the physics behavior to be set on "Fixed", otherwise it goes wacky like in the video... However, when I set stepping mode to Fixed it also looks like player is able to accelerate faster. So with this stepping mode, will physics objects go at different speed on an other machine ? (depending on its refresh rate ?)
Speed at which the player moves relatively to the level actually matters a lot in my game. In worst case maybe I should lower my screen refresh rate to 60Hz ?