Not great, but what I did to work around until I can script it

1.  open vtb_board.LIST

2. filter for my Task board by name

3. Open the board, go to the members related list at the bottom

4. Edit the members list

5. Use the filter to get the users i need, (In my case, I filtered on start with "IT ") and Run Filter

6. Select All in the Left pane, Add all to right, then pick and choose the folk I don't want