can you rotate the layer instead?
edit: I didb't fully read the first post...
when you are drawing on the canvas, you might have to calculate the layer coordinates for the layer the canvas is on by using LayerToCanvas, and then CanvasToLayer
but I haven't tried that...
edit2: ok, just tried it and got funny results - so tried it in C2 and it seemed to work the way I would expect, so it might be a bug in C3.