Check my example, the red ball will work. I did the new trajectory with the ENTER one. The only problem is if it hits the wall in a weird place the angle will look wrong, which is why I said you have to fine tune your angles and speeds. Figure out places on the wall where if hit would look "good" going straight into a hole.
Also I manually apply a "drag" after it hits the wall - which emulates mass and energy being transferred to the wall (since we don't have actual physics in that one). So you want to make sure the drag is not too "high" if the hole that the ball is going to is too far away.
I would script set holes for different points on the wall. So each ball will have it's own approach angle/speed and hole destination pre-determined. Then you can randomize between those sets and fire a mass of balls going to their prospective wall points and destination holes.