Nikel, your expectation is correct. The rollback of an uncommitted object should have removed the object. And your conclusions are all correct, please enter a ticket for this problem so R&D can fix that the rollback will immediately remove the object.
The default cancel button however is working correctly, the objects cancelled do get reverted.
Also this applies to non-persistent objects too.
This issue is because of the garbage collection logic of the platform, normally all (non-)persistent objects that are no longer accessible will be removed from memory. If the object has no associations, it would have been removed form memory when the microflow finishes, but since your microlfow has an association to an actively used object the object stays in memory. (I just wanted to add this info so I don't create the perception that we have a memory leak)
FYI, this is the microflow I tried to do a simple test:
Did you commit another object that referenes the object you attempted to rollback? In that situation your object would be auto-committed. Not sure whether a rollback action would delete the object.
After you create the object do you have any "Change Object" activities that modify that newly created object? Under the properties of the "Change Object" activity there is an optional action that will do a Commit based on that change before you think that you are doing an explicit commit. I had done this myself not too long ago and felt that I was seeing exactly the same behavior that you are describing where I didn't think that a Rollback was behaving properly. In my case, at least, the Rollback was doing the proper thing and rolling back to that "Change Object" commited version of the object. Once I removed the "Commit" from the Change Object the rollback began behaving as I (and you) are expecting: it removed the object entirely.
While this may not be what you are seeing or you may have already checked for such "accidental" commits, I certainly got fooled by this for a time.