A few pointers from my experience:
Upgrading to a major version has a lot more impact than between minor versions (days vs. hours). Example: going from Mendix 4.x to 5.x is a big change. Going from Mendix 4.3.2 to 4.3.3 will generally not cause any unexpected changes.
Don't upgrade to 'first versions' (x.x.0) if you are in production. Wait for a .1 version or a hotfix, and try to identify beforehand if you are upgrading to a stable version
Always create a back up of your old version as downgrading is not really possible
Look for issues specifically in java / widgets / app store content / (sometimes) layout after upgrading. There could still be issues elsewhere, but the core functionality i.e. forms/pages, microflows, domain model is generally stable across versions
Make sure your development team is not working / does not have uncommited work when upgrading, and get everyone on the new version before continuing
Read release notes carefully and determine if there are major changes to functionality that will need changes from your side, i.e. CSS / layout between versions 4 & 5 and java.
We don't normally upgrade along with every new version in existing applications. The main reasons we would upgrade are:
Last one especially is important. If the new version offers major new functionality that opens up new possibilities (such as in Mendix 5), clients are often quite receptive to an upgrade which they will pay for.