You would have added field level validation message in your widgets which is there in the page. Remove the validation from page level and add it in microflow which should get triggered on click of Save button.
Once the above is fixed, validation message wont get triggered on click of cancel button
Page-level validation has been triggered on default cancel button.
As Steffy mentioned, add the validations in microflow
If the validation needs to be in page-level then
Add a microflow for cancel button and in microflow settings, set Abort on Validation errors as NO as shown below
Hope it helps!!!