Does the page itself have the department taxonomy/custom field information assigned to it ?
If so then we can use a dynamic filter to filter the view based on the value that the page itself provides. If you're not sure then what you can do is to provide me with access to the site and a link to a page where you would want team members from a specific category to be displayed.