CHERI extends conventional ISAs with hardware-enforced capabilities to provide fine-grained memory protection and its integration in RISC-V is gaining momentum with RVY. As adoption grows, implementations must be evaluated to ensure working CHERI protection mechanisms. We show that existing memory-corruption exploit implementations do not directly carry over to CHERI-enabled architectures, and that observed exploit failures (i.e., unsuccessful exploits) do not necessarily imply effective protection. To resolve this ambiguity, we propose a methodology that temporarily disables CHERI enforcement within a RISC-V VP. Comparing exploit behavior with and without CHERI enforcement under otherwise identical conditions makes it possible to distinguish exploit failure from effective CHERI protection.