I'd say compare where on the ball the mouse clicks (left right top bottom), and for each of those apply the appropriate impulse. ie: If mouse.X < Sprite.X, apply point 2 impulse to kick right.
Edit: Oh, I'd increase your Angular damping also.
Edit2: with an upwards impulse also.