What you can currently do is set "Minimize to the tray" in options, then run it as "bitcoin -min" so it starts minimized. The only visible part will be a small (20x20) icon on the tray, which can be doubleclicked if you want to access the UI. Note: there's a bug with tray icons sometimes disappearing on 64-bit Karmic Koala, not sure if it's from 64-bit or Karmic, it was fine on 32-bit Jaunty.
We didn't have time to implement the "Start Bitcoin on system startup" feature on Linux in time for 0.2 so it's greyed out. I figured Linux people wouldn't mind doing that manually anyway. I guess they need to know about the -min switch to do it right.
You can locate the data directory where you want with the "-datadir=<directory>" switch. I know someone is already doing that to put it on a TrueCrypt USB drive.