for me this is a common pattern:

  ssh x

then in other windows, I do things like this:

  scp file x:
  ssh x command
  rsync from/ x:to/
  emacs /ssh:x:filename    (this is tramp mode)

edit: one more thing

  ssh x -o BatchMode=yes command

if you have an automatic script to run on a remote machine, adding BatchMode=yes will make it error out instead of stopping and asking for a password