mstsc on server core

There is no “install feature” method of doing this and I can imagine it's not supported by Microsoft. But you can add the RDP Client (mstsc) to server core by copying the following files from a server (preferably the same OS) with full GUI installation to the Hyper-V server:

From System32 folder:

From System32\en-US folder: