If it can integrate via a java action or api such as rest, soap, jdbc, flat file, message bus, you can use it in Mendix.
Short answer is no. Mendix runs on Java and it is not possible to add R code to a java runtime. You can look at java libraries for machine learning such as weka https://www.cs.waikato.ac.nz/ml/weka/
Have a look at this blogpost for some ideas