My previous project was on premise and we were able to create branches. Are you sure you are picking the correct project in the branch line manager? Also are you using the mendix team server for your project?
I think you make it yourself very difficult if you want to try to setup your own teamserver and I am even not sure if this would be possible. You might try a support ticket and see if you could some help from Mendix.
But in my opinion your main focus should be the reason why you want to do this. From a security standpoint I do not see the point. See also this documentation on the teamserver part: https://docs.mendix.com/refguide/team-server-faq
I finally have solved this problem.
we are using proxy . when I disabled the proxy setting, the new button is enabled....
I will submit this problem to support .
Thanks a lot!