Thank you!
I had tried that from the project browser and there it is not possible. But when you posted thats possible I tried to do it in the layout editor. And there it works. That's a bit confusing, because you'd expect that the project browser is the main place to do such things.