Finally, I got it!
Actually the idea came after helping someone else!
What I did was to destroy the ledgehelper after x seconds of grabbing the ledge and after the Player starts falling, re spawn it after x seconds in order to avoid colliding with the ledge again. If a fall is too short, I re-spawn the ledgehelper on the floor as well.
I don't know if this is the way of doing it but so far it works fine. If anyone got any other ideas please let me know.
Thanks!!