Check "Destroy on startup" in the Attributes section of the object's properties.
Then when you need to create it during run-time, use the System action "Create object."
Edit:
Oh, you mean you don't want them cluttering up the layout itself while you're working in the IDE? I dunno. You'll just have to stick them somewhere. You could always just use the Hide feature if they're in your way or whatever.
Edit 2:
The "Hide Selected" option appears not to be working (it's grayed out). But the "Hide Unselected" and "Unhide All" are working ok.