In the prod-log: 404 - file not found for file: robots.txt
in our logs, there is a message about “404 - file not found for file: robots.txt”. robots.txt is used for informing the crawlers what pages they can inspect. Found previous questions on this: https://forum.mendix.com/link/questions/87790 and https://forum.mendix.com/link/questions/4678 Where did the robots.txt go? They are no longer in the /theme-directory. Are they no longer needed? If so, how to prevent the error showing up in the log?
Tim v Steenbergen
You can create your own robots.txt file and drop it in the theme folder. Not sure why it’s not included by default anymore, but it should be simple to recreate, and you’ll get to set it according to your application’s needs.