No! Don’t do that! This will screw up the group settings for your user (at least on Ubuntu 10.10). What you should rather do is:
sudo usermod -a -G vboxusers your-user-name
which appends to the existing groups for your username (haven’t tested this, but this seems the logical thing to do). Also see this post in connection to this issue:
http://x.thexs.us/tag/usermod/
If you already did the above and can’t sudo anymore, follow the instructions here:
http://ubuntuforums.org/showthread.php?p=4273283#post4273283
HTH
Sebastian