From version 2.4.3 of OpenVPN onwards, this is now possible using a 'token' after the initial auth takes place - and using the new token for all auth requirements during a renegotiation.
To implement, download my yubikey-auth-tokens script and place it in /etc/openvpn on your OpenVPN server.
Edit the script and add your username and YubiKey ID into the %yubikeys definition. You can change the $tokenstore variable to somewhere that the OpenVPN server can write to. By default, this is /etc/openvpn/jail/token_store.bin
Now add the following to your OpenVPN server configuration file:
auth-user-pass-verify /etc/openvpn/yubikey-auth-tokens via-file
If all goes well, use the username and YubiKey specified to connect.
Happy VPN'ing :)
Revised 2018-02-17 Rob Johnson has graciously dug further into the YubiOTP API documentation and found that "the API can do more thorough validation using the secret key. I've added a nonce, signature and various other aspects to validate that the response came from Yubico and is current.". I have updated the script with his revised version. Thanks Rob!