In my script I name an argument $out and then trying to use it in a filename like this:

My problem is that $out_ is recognized as “not only $out“. The file name MUST be like this.
I already tried quotations or double quotations or + with no appropriate results.

Multiple options.

1. Use {} to qualify the variable name (as pointed out by @PetSerAl):
   "C:\folder\${out}_date.txt"
2. Use the -f operator to expand $out before placing it in the string:
   'C:\folder\{0}_date.txt' -f $out
3. Use the backtick () escape character to stop parsing of the variable name:
"C:\folder\$out_date.txt"
4. Use a sub-expression ($()) to evaluate the variable:
   "C:\folder\$($out)_date.txt"