There is one active party icon

and inactive disbanded icons

The active icon is the one you move and "talk" with, regardless of how many of your party is in that icon group, could be all 7 or just 1. If there are check to do, I think it goes with the 1st player in the group.
For example you could disband the one female character from your party and send them into the female bathroom, and the charisma check would be on her. In other cases the whole group would enter dialog with another person.