Merge Problems and the svn:ignore Story

40
Some people have been experiencing issues with merging branches recently and I want to take some time to explain what happened, because it has become quite a complicated story. The story starts simple. It starts with a ticket on Mendix 7.15.1. A team was experiencing crashes when merging branches to the main line. And when I say 'crash', I really mean crash. The Modeler would disappear without a word. Such a hard crash can only happen if it happens inside a native library, like the Subversion C library in this case. We were invited to the project and we could reproduce the issue. We found that the merge failed while merging the svn:ignore property. Let me explain what the svn:ignore property is. Some files are inside your project directory, but they are not (and should not be) on the Team Server. For example, the ‘deployment’ directory is on disk but not on the Team Server. Everyone has their own ‘deployment’ directory; it is not a shared thing. The same holds for backup files of the project and some other files. Subversion keeps track of which files are ignored in the svn:ignore' property. If you want to see this property yourself, right-click your project directory in the Windows File Explorer and choose TortoiseSVN > Properties. There will be a line in the grid for this property and its contents will look something like this: modeler-merge-marker .mendix-cache <PROJECT>.mpr.lock <PROJECT>.mpr.bak .project .classpath deployment <PROJECT>_main.launch If you merge a branch line to the main line, two of those svn:ignore properties meet. If they are both changed, it is very likely that there will be conflict. Resolving a conflict on this property is simple because you can just combine both lists, remove duplicates and be done with it. And that is what the Modeler does for you. In case of the project of the customer, we found that there were some funny symbols in the value of this property, namely so-called byte order marks (BOM). It took us a while to find those symbols, because they are invisible. We concluded (wrongly) that this was the issue and disabled the automatic merging if we encountered BOM characters. This was released as part of Mendix 7.19.0 and it fixed the issue for the customer. They could merge again. Everybody happy! Well, no. For the people for whom the automatic merge still triggered, another issue arose, having to do with newline conventions in the 'svn:ignore' values. You've got to love all those invisible characters spoiling the fun! If you have this issue, you will see messages complaining about a property change failing and something called PROPPATCH. See, for example, https://forum.mendix.com/link/questions/92119 We provided a workaround for this problem on that forum question and here in the documentation: https://docs.mendix.com/howto/collaboration-requirements-management/troubleshoot-version-control-issues#2-6-getting-an-error-with-the-message-sharpsvn-svnrepositoryioexception-at-least-one-property-change-failed-repository-is-unchanged Also, the real fix for this newline problem will be released as part of Mendix 7.23.0. Everbody happy! Well, no. The next thing that happened is that we received a similar ticket on Mendix 6, which was odd because the cause for the BOM characters was not in Mendix 6! The plot thickened. We investigated again and our new theory was that the size of the ignore lists was becoming a problem. The projects in both tickets had lived for a long time and contained a lot of branches. For every branch in which you deploy, the Modeler creates and ignores a .launch file with a name like this: <PROJECT>-<BRANCH>.launch. Over time, those ignores start to accumulate and even though the documentation of Subversion states that there is no limit on the size of properties, we are now convinced that some of the underlying code actually cannot handle large ignore lists. So, the problem was not a new problem but it has always been there. Only now projects are becoming old and large enough to trigger it... By cleaning up the ignore lists to a reasonable size, we were able to merge again. We supplied the customer with a way to clean up the ignore lists and they could continue working again. To clean up the ignore list yourself, take the following actions: Install TortoiseSVN 1.7.15 if you do not have it yet: https://sourceforge.net/projects/tortoisesvn/files/1.7.15/Application/ Go the project directory of a particular branch. Right-click the white background of the project directory. Choose TortoiseSVN > Properties. Double click the line with the "svn:ignore" property. You will see a long list of ignore lines in a text editor. Remove all lines that end in ".launch" Also remove lines that look like this: /branches/branch:4183-4252. Those lines are actually corrupted data which should not be here at all. There may be other corrupted lines that look like this: 80-13ba-4c6f-98da-a23cd0afecdf svn:mergeinfo 3557 /branches/branch:4107-4155. Remove those as well. Make sure you do not remove other lines like "releases". Add a line containing the following text "*.launch". You should end up with a small list, something like:   modeler-merge-marker .mendix-cache <PROJECT>.mpr.bak .project deployment .classpath <PROJECT>.mpr.lock releases *.launch Click OK and OK again. Open the project from the Desktop Modeler. Commit the change to the 'svn:ignore' property with a special message, e.g. "Clean up ignore list". (Note: you will not see any changes in the Commit dialog but you can still commit). It was a mistake to ignore every .launch file individually and in Mendix 7.23.2, we will start ignoring them all at the same time by including the following line in the svn:ignore property: *.launch Based on the unreliabiltiy of the automatic merge feature, we have decided to disable the feature in 6.10.18 and 7.23.2. This means that you will have to resolve merge conflicts on the 'svn:ignore' property yourself. Here is the manual on how to do that. https://docs.mendix.com/howto/collaboration-requirements-management/troubleshoot-version-control-issues#2-7-resolving-conflicts-on-the-svn-ignore-property However, because we will be ignoring all .launch files at once (*.launch), branches will almost never contain changes to the ignore property anymore and conflicts will only happen if you ignore files or directories yourself. Everybody happy! I sure hope so. So, what should you do for your project? You can check whether the ignore list has grown a lot and if so, clean it up by using the steps above. Migrating to 6.10.18 or 7.23.2 (once they are released) will fix the problem for good, but there is no rush to do that. Finally, I want to apologize sincerely to the people who lost valuable time because of these issues. Knowing that you could not continue working, caused me sleepless nights as well. If you come to Mendix World, I can personally apologize to you. And there are lots of other reasons to come, too.
asked
5 answers
1

Thanks for the explanation! We encounter the same problem in 6.2.1. 
Question: What do you consider to be a long ignore list?  

answered
0

Thank you a lot.

answered
0

Thank you. Tested with only few merges, but looks like this solution resolved merging issues.

Cleaning up svn:ignore only on mainline did not help. Branch which I wanted to merge to mainline still had lot of data in svn:ignore, so merge crashed modeler again. So, for all new branches this will work, however for existing ones, both branches needs to have clean svn:upload.

answered
0

For the record. This issue is still present in 7.23.3. Also, none of the above steps worked for us.

answered
0

Facing the same issue repeatedly in Mendix 8.6.0 while taking updates. “SharpSvn.SvnDiffException: Unexpected end of svndiff input”. Any workaround or fix for this?

Thanks.

answered