It could be that one of the consistency checks is inefficient. It this happens for you then it could be caused by a particular type of document you're using (for example an app service or a navigation document), commonly used documents like microflows and pages are a potential candidate as well but not as likely because they are used so often.
You could file a ticket for this or try removing documents of a certain type and see if that speeds things up significantly and THEN file a ticket for that :)
Information that is good to include is when this started happening (if you remember) and what kind of things you were working on at the time. Obviously also if it started after you moved to a certain version.
Cause of the problem seems to be the Stories API. Removing this API speeds up the modeler considerably. Unfortunately (re)connecting the Story API slows down the modeler again.