Well firstly in this situation where levels are involved, I would use a separate layout instead, or at least destroy the old level upon transitioning to the new. If I needed to preserve the level to revisit later, I would definitely use a separate layout.
If in the case of a menu and clicking, I usually have the "layer is visible" condition in the menu events.
I can't think of much for the layer off the top of my head, but I can definitely say there are a billion and one uses for invisible helper sprites, and I definitely want those interactable. I don't see why layer visibility should behave different than object visibility. The question is why make an entire layer invisible to begin with? I'd imagine maybe for inventory or menus and other interface elements, but again a single condition on the input event solves that. Usually doesn't involve solids.
The main use case for having solids on layers that become visible and invisible would be maybe something like floors in a building. IIRC this is one of the big issues the collision filtering and tag feature was developed to address, besides stuff like half height objects that block movement but not visibility/bullets ect.
You don't even need to manually enter a lot of collision tags for solids - just use a loop on start of layout to set the tag to the layer name the object is on. When the player changes layers, set their solid collision filter to the layer they are on, so they can only interact with things on that layer.
Also, the solid behavior can be enabled or disabled. If you're already changing layer visibility, you can enable solids on the new layer and disable on the old just as easily. There are many ways to deal with this.
One use case I just thought of for hiding a layer and still have solid interactions is if you wanted a certain set of information hidden from the player, say they got blinded, or hidden enemies. They still need to pathfind, bullets still need to hit walls, ect. even when they are not visible.
Anyways sorry that was a bit long and rambly, so TLDR back to "So is there a way to make a layer 'not interactable'?". There are a few options, as usual, depending on what the situation requires.
A. Use the "Layer is visible" condition
B. Use collision filters. Doesn't necessarily mean a lot of work, and it was designed for stuff like this.
C. Disable solids on the layer that isn't in use.
D. Don't use a layer like this to begin with (ie. destroy and create objects as needed instead of just making them invisible), or move to a different layout instead.