User level separation, while it has improved over the years, was not originally designed assuming unprivileged users were malicious, and even today privilege escalation bugs regularly pop up. If you are going to use it as a sandboxing mechanism, you should at least ensure the sandboxed user doesn't have access to any suid binaries as these regularly have exploits found in them.
VMs are common, consider going that additional step. Once you have one agent, it's natural to want two agents, and now they will interfere with each other if they start running servers that bind to ports. One agent per VM solves this and a lot of other issues.
Same with the browser agents, they are used in a browser where you‘re also logged into your usual accounts. Means in theory they can simply mail everyone something funny, do some banking (probably not but could work for some banks) or something else. Endless possibilities
An agent can be designed to run with permissions of a system/bot account; however, others can be designed to execute things under user context, using OAuth to get user consent.
It's not even hard to do! *NIX systems are literally designed to handle stuff like this easily.