Windows 11 detects rustdesk.exe using a scanning program; this does not happen on Windows 10.

[josefredohd]

I used to use r77 for an exam program on Windows 11, but today I tried it and it detects rustdesk.exe—something it didn't do before. However, this doesn't happen on Windows 10; it doesn't detect RustDesk, and I can control my computer from another one.

[bytecode77]

it detects rustdesk.exe

Do you mean that you normally use rustdesk, but with r77 installed, your AV detects it as malicious?

I can only guess, but r77 does inject running processes and the injected payload (the rootkit) is malicious according to AV software at least. Things like AV detecting random running processes as malicious because of that could happen as long as r77 is installed.

(if I understand your question right...)

[josefredohd]

The AV is disabled, I mean that a program to take an online exam detects the rustdesk.exe process even though it is hidden with r77, this did not happen before, in Windows 10 the program does not detect it rustdesk, could it be due to a Windows 11 update?

[bytecode77]

Ah, so your exam software detects cheating...

This can be done using many different methods:

• Detect the process by name
• Detect the folder name in ProgramFiles
• Detect the software in Registry (installed programs)
• UI Element on the desktop with certain strings in it

Will be very difficult to reliably hide it with a rootkit, unless you know what is detected.

But what you definitely can do is take the exam inside a VM, where nothing but the OS + a browser is installed; Then run rustdesk on the host machine. That way the exam software can never detect that it's being observed...

[josefredohd]

Thanks, I'll give that option a try. Do you think using a rootkit with a kernel driver to hide processes could actually solve this?

[bytecode77]

Do you think using a rootkit with a kernel driver to hide processes could actually solve this?

Hardly.

Your exam software probably uses trivial mechanisms to detect remote administration software, such as listing proceses, specific directory names under ProgramFiles, etc. It will not have sophisticated algorithms that detect tampering in user-mode memory of its own process, that's simply out of scope for something like an exam software.

So, unless you know exactly how your exam software detects rustdesk, you don't know what to hide. Once you know what to hide, then a ring 3 rootkit will be sufficient.

This is a hell of a different story for something like online games. Valve developed a highly sophisticated anti cheat engine. It can detect injection into the game, API hooks, etc. So for this, you must go kernel mode. But an exam software is really the product of a small team where such knowledge simply isn't there.

[josefredohd]

I understand. What's strange is that on Windows 10, the same exam software doesn't detect the RustDesk process or any other process hidden using r77. I'll have to use Windows 10, but it would be much more convenient for me if it also worked on Windows 11.