This problem should be solved with the latest version, however it seem to pop up every now and then. You can try the previous version which does an explicit commit as you describe.
Is there a one on one relation somewhere of those filedocuments? I have seen some strange behaviour when working with filedocuments and one on one relations and trying to replace an object that uses that reference. Sollution was to clear the reference first before setting the new reference.
I had this issue as well. Using the latest version possible for mendix 7.5.1 (Dropzone 4.0 / tag 3.2).
The unique constraint error occurred in the ‘Association to context’ object. This object was not yet committed and probably got auto-committed (for every document?) when setting the association.
I committed the object I need to set the association to on create which seems to solve the issue. On Cancel I had to remove the object myself as rollback was not working after the commit.
It might be related to https://forum.mendix.com/link/questions/89644 see the answer of Jaap Pulleman