I have implemented this at a customer in Mendix 6. The steps needed were:
1. Replace the login action to always throw an error.
2. Enable anonymous users and create a custom login page, containing username, password and authenticator inputs
3. Create a login microflow where you check username, password, authenticator and all other relevant checks (user is active, not anonymous, not a web service user etc).
4. Upgrade the users session through Java with Core.InitializeSession(User, SessionID) (this step breaks in Mendix 7, you will need to store the information and redirect the user to a custom request handler :( )
5. Redirect the user to the homepage with the URLRedirectorWidget
This takes care of your logging in. Then, you need to manage the secret key. For this I took the following steps:
6. Add an object to store secret keys and associate it to System.User.
7. When a user logs in without secret key (see step 9), create such an object, generate a key and store it and show the QR code to the user.
8. Allow an administrator to delete the above object. When an administrator deletes this object, set a time a short time in the future (say, 5 minutes) for this user.
9. Allow users to log in without authenticator code if they are within the time set at step 8.
I want to create a reference implementation and publish it as an AppStore module. I am currently planning to do this next week Tuesday (14-03). I will invite you to the project once I have it set up, so you can have a look.
Update: I have finished the reference implementation. I'll start writing documentation and see if I can get it approved for the AppStore. If you are interested, you can contact me through LinkedIn and I'll gladly share it with you.
The github page describes how to use it: https://github.com/RoelandSalij/GoogleAuthenticator