I think the easiest way to do this is to make a distinction between MainGroup and Group. You need the following references:
In the first form show a DataGrid with MainGroups. This will only show your top level groups. You can also define a form showing a Group, its sub-groups and its products. This form can be used for each Group (recursively).
You can consider creating a custom widget showing a tree view with your groups.
Instead of splitting the first category of the rest of the categories, I found out that you can also use the XPath constrain
[not(Module.Group_Group)] in the first form. This has the advantage that you do not have to split categories.
I think what you need to do is create a reference from product group to product group. I this way, any product group can refer to any other productgroup.
Formwise, when for instance creating a new productgroup, you will need to add a reference selector that will allow you to select existing product groups. In this way you can make as many levels op categories as you want.
More info on references and their properties can be found here
To create indefinit product groups you can add an recursive association 0-1. In the product group forms you can add a reference selector to select the other product groups.
One thing remains, I get the idea, but how do I prevent the first form from showing all categories instead of only the 'parent' categories.
Another thing, when I use the method Johan suggested and using the default context settings, the Group created from the nested data grid in the data view that shows another Group object is linked to the MainGroup instead of the Group that is the shown in the data view. I think this can be prevented by changing a context property. Does anyone know how?
I know there is a workaround by using a micro flow to show the form, create the Group and set the link for you, however it is neater to do this without using a micro flow.
You can probably use 'constrained by' for this (but I'm must admit I'm not entirely sure if this also works for recursive references)