I can't seem to reproduce or understand your issue.
The file you provide has an object "Player" that will go up and down, bouncing from "wall" and "wall2".
There is no issue of "changing angle to left and right", the movement is perfectly vertical on my computer.
You used the Bullet behavior and configured it to bounce from Solid anyway.
There is not much to do using events there.