Sorry, my comment didn't really make sense—my mind was wrapped up in something else. Though I'm curious to know what the 'is by wall' issues are...
I understand how the offset works now—are the collision polygons, for both player and solid tiles, rectangular?
If not, there could be a frame where the lower point of the player collision polygon clips the tile when moved to the right (and up) even though majority is still above the tile.
Also, is the image the exact spot this happens? and will the player change directions mid-air? I was thinking if maybe on the jump they clip the corner of a 'new' ceiling, as in there is not ceiling above them but above and to the right there is, it could cause the change in direction. But unless it is programmed to not change directions until on the ground you would notice right away.
...those probably aren't the issues, but just trying to think out of the box as I don't see any issues in what you posted.