The function of Groups is to help categorize a users's working roles within your setup. While this sounds fairly similar to departments, there is a distinct difference between the two, not only in semantics but in functional purposes as well. The main differences between groups and departments are:
In short, use departments when you are trying to represent a user's placement within an organizational chart, and use groups to represent the various functions a user might carry out.
Figure 1: Setup Users - Setup Groups
To manage groups, go to Setup Users > Setup Groups. Note that each group created can be tied to a specific organization, although not necessarily so. As mentioned earlier, a user can belong to multiple groups and, unless restricted, a group can have members that span several organizations.