First open SQL Server Configuration Manager
On the left-hand side, under SQL Server Network Configuration, select the protocols for your server instance. Most likely you will now see on the right-hand side that the TCP/IP is “Disabled”. Enable it by right clicking on TCP/IP and selecting “Enable”. Ensure you expose your desired port in the properties (again, right click on TCP/IP and select “Properties”)
In your configurations, ensure that you use integrated security
If you don’t want to use the Windows authentication, then you should not use the integrated security. You should then add the username and password of the database user (which you can define in SQL Server Management Studio). In this case, you should also change the allowable authentication on the SQL server. In SQL Server Management Studio, right click the server, select properties. In the left-hand side, select “Security” and under “Server authentication” select “SQL Server and Windows Authentication mode”
Hope it helps!
I dont know if you still need an answer but for all others finding this: You can set the compatability level of your database in the database properties.
Right click your database --> Properties → Options
The minimum supported version depends on your Mendix Version, the supported SQL versions can be found in the mendix documentation: System Requirements | Mendix Documentation