In 2.4.6 this can be solved rather complex.
The solution I could think of is, since it is possible to make fields read-only based on an enumeration value, by triggering a microflow that sets a dummy enumeration when the reference selector is changed. When you give this dummy enumeration two values, one for read-only and one for normal and use this in your form it could be solved.
For other types of validation like required you could use a before commit microflow. When the object doesn't conform the validation (you can check this in your micoflow) you can trigger a validation with the 'Send validation message' activity and return false in your microflow (this means the object won't be committed).
I don't know if this will be easier in 2.5, maybe someone else could give that answer. Anyhow, I hope this will help you. Good luck!