decrypt date data type

I accidentally used date data type (rather than datetime) to encrypt a column. As I understood from online research, that
necessarily entails conversion of the date data type to varbinary(200) to begin with, then encrypt.  Encryption by this method worked without issue.

Now I need to decrypt it:

select convert(varbinary(200), decryptbykey(encrypteddate)) myvarbincol from mytable

.. works fine,
but when I try the next step of converting it back to date:

select convert(date,convert(varbinary(200), decryptbykey(encrypteddate))) mydatecol from mytable

.. it yells at me, and I can't use it to insert or update my destination
columns:
Msg 241, Level 16, State 1, Line 1
Conversion failed when converting date and/or time from character string.

Is there any way to recover my encrypted values and get them into a proper datetime without starting over?