We can use [Windows.Markup.XamlReader]::Load to load XAML file in PowerShell, but is there some way to specify some PowerShell script as code behind?

Now there is also WPK, in the PowerShellPack which makes creating WPF UI in PowerShell extremely straightforward and fun.