in your example, you would have to retrieve (in your microflow) Material from Order "By Association" or - if both have been commited to database already - "From Database". Then you can multiply Material/UnitCost with Order/OrderQuantity.
Please allow me to recommend having a look at the Mendix online learning paths: here you will find great resources to get kickstarted with Mendix:
You can try calling action ‘Change object’ in your microflow and change your Order’s TotalAmount to take value of this expression:
$OrderObject/Questionnaire.Order_Material/Questionnaire.Material/UnitCost * $OrderObject/OrderQuantity
I hope this helps.