I was a bit out of ideas and was thinking that rebuilding the whole server is my only option. After all, at some point, you start wasting more hours than it takes to build a new server from scratch. Fortunately, I won't have to do it. It seems that Ransomeware protection is responsible for my misery. It's a new feature in Windows 2019 and in Windows 10 1809 but it's also a feature that needs to be configured properly.
By default this feature is disabled, and someone has turned it on with default settings thinking it solves all the problems.
Turning it off makes the error of non-existing redirection to C:\Windows\system32\config\systemprofile\Desktop go away. I wish Microsoft would give better notification to a user when such a thing happens. It took me a while to understand what is happening and it wasn't that obvious. It was a bit harder because it wasn't me who configured this feature, so I didn't know it's even on. But from now on it's something on my checklist to verify any issues I get with Windows Server 2019. New server version brings built-in antivirus and ransomware protection. It's a significant improvement to the system since you don't have to worry about AV anymore. While antivirus works out of the box, ransomware protection requires some configuration. As Microsoft puts it “Most of your apps will be allowed by Controlled folder access without adding them here. Apps determined by Microsoft as friendly are always allowed.”. So while some apps on the server worked correctly, others didn't, and there was no clear way to tell why the error happens from the message presented to the user.