Thanks for all your answers.
I was able to open it with v3.5.11 (uninstalled 4.2.8, installed 3.5.11 withou a reboot) but i did some further investigations because i noticed, that the size in memory is only 8.837.044K with 3.5.11, which is far less than over 11GB of free/avaiable memory.
By the way: The used memory size with 4.2.8 is only 7.433.952K.
First of all, let me say that because of speed reasons, i do not use a pagefile, so there is a hard limit of 16GB ram-memory if pagefile is deactivated. I noticed, that 3.5.11 starts to opening the file no math