fix: stability and OPSEC correctness improvements

[mranv]

Hey, been playing with PolyEngine for a bit and found a handful of things worth fixing. Nothing earth-shattering but a couple of them could bite in production builds.

What's in here

MutationEngine.c

• Added an overflow guard before HeapAlloc — if payloadLen gets large enough, maxStubSize + payloadLen wraps to a small value and the subsequent memcpy walks off the end of the heap allocation. Not a real-world concern with normal payloads but the guard is two lines and costs nothing.
• Dropped a HeapFree(GetProcessHeap(), 0, NULL) in the template validation error path. It's a no-op (docs say so) but it looked like cleanup that wasn't actually cleaning anything up.
• Moved junk/NOP insertion to fire before every setup block, including the first one. Previously block[0] always landed at offset 0 — a consistent anchor that static sigs could grab onto.
• Swapped out rand()/srand() for a local XORshift. Builder was running two independent PRNG states (stdlib + the XORshift in Crypto.c) seeded from the same rdtsc window. Unified them.

Syscalls.c

• Replaced the bubble sort in SortSyscalls() with insertion sort. ntdll's Zw* exports come out of the export table nearly sorted by RVA already, so insertion sort is basically O(n) here vs O(n²) for bubble.
• FindCleanTrampoline() now validates that a matched 0F 05 C3 site sits inside a RUNTIME_FUNCTION entry before returning it. FindJmpRbxGadget() already enforced this — trampoline should too, otherwise an EDR stack walker starting from the syscall RIP could see a frame without unwind info.

ApiHashing.cpp

• DJB2_SEED was RandomCompileTimeSeed() % 0xFF which allows 0. Seed=0 gives a degenerate distribution for short strings. Changed to % 0xFE + 1 (range [1,254]), matching what XOR_SEED in Evasion.cpp already does.

OpsecFlags.h / Stub.cpp / Builder.cpp

• EVASION_FLAG_UNHOOK renamed to OPSEC_FLAG_UNHOOK. The unhooker is a loader technique, not an evasion check disable — the EVASION_FLAG_NO_* namespace is for per-check disables. Trivial rename, no behaviour change.
• --preset RANDOM could pick the same DLL index for all three slots. Added a small dedup nudge so all three indices always resolve to different DLLs.

Test plan

• Build Stub (MSVC) — clean
• Build Builder (MSVC) — clean
• Pack a test PE, verify it loads correctly
• --preset RANDOM a few times — check log shows three distinct DLL indices
• --unhook flag still accepted and works

[mranv]

Hey @LongWayHomie — been going through this codebase pretty carefully over the last few days and wanted to flag this PR for your attention.

The changes in here are things I noticed while trying to understand how the whole pipeline hangs together. Added inline comments to the modified files so the reasoning is easier to follow without having to cross-reference everything — should make the review less painful.

Quick summary of what's in there if you don't want to read through all of it:

• MutationEngine: ripped out the stdlib PRNG (rand()/srand()) and replaced it with a local XORshift — same approach Crypto.c already uses, so at least it's consistent now
• Syscalls: swapped bubble sort for insertion sort on the SSN table (near-sorted input anyway, insertion sort is just better here), and made the trampoline search consistent with the FindJmpRbxGadget logic that was already doing RUNTIME_FUNCTION validation
• ApiHashing: the DJB2 seed range was [0, 253] which breaks on single-char strings when seed hits zero — clamped to [1, 254]
• Builder: the RANDOM preset was picking duplicate DLL indices for module stomping — added dedup nudge
• Flags: renamed EVASION_FLAG_UNHOOK → OPSEC_FLAG_UNHOOK to match the rest of the flag namespace

Nothing dramatic, just tightening things up. Would appreciate a look when you have a minute — happy to answer questions or adjust anything you're not happy with.