Calculated attributes are not stored in the database which is the reason why you cant use them in xpath constraints or use the aggregate functions on them. Instead they are calculated by a microflow when a user navigates to a page where it is being displayed. I would change your TotalCost and TotalParticipants to the data type Decimal in order to run the aggregate functions.
Once you change the data type for TotalParticipants your xpath would be this (if your datagrid is showing the tourgroup as the entity)
TotalParticipants > 5
Another idea is to create a enum status or boolean on your Tourgroup that is triggered when your it reaches 5 or more registration. Then you can set your data grid's xpath to read.
[(insert_boolean_name) = true()]
You could use the xpath count function to count the connected regustrations https://docs.mendix.com/refguide/xpath-count