Lerp the acceleration perhaps.
Mouse is down
->object has line of sight to mouse.x,mouse.y, set position to lerp(self.x,mouse.x,lerp(speed,minspeed,acceleration*dt)*dt) lerp(self.y,mouse.y,lerp(speed,minspeed,acceleration*dt)*dt)
->else set position to lerp(self.x,mouse.x,lerp(speed,maxspeed,acceleration*dt)*dt) lerp(self.y,mouse.y,lerp(speed,maxspeed,acceleration*dt)*dt)
That is with the line of sight behavior on the object, and an else to the los event.
Of course that wouldn't cover when the mouse was up.
Might need a boolean to signal that if you want it to continue to where the mouse was.