Trace vector decoder

From Wikipedia, the free encyclopedia

Trace vector decoder is a system that uses a microprocessor's trace mode to decode encrypted code just-in-time before it is executed and possibly re-encrypt it after the execution. It can be used to enforce copy protections for some computer systems.

[edit] Trace Vector in Motorola 68000

As an example, Motorola 68000 has a trace mode where a trace exception vector is executed before each instruction in the main program. The processor automatically changes execution to the trace exception vector before executing any instruction from the main program. The trace exception vector decodes the instruction that will be executed after the exception vector. The next time trace exception happens the old decoded location may possibly be re-encrypted.

Following code snippet is an example of a program initializing a trace exception routine.

        MOVEM.L Stack,D0-D7/A0-A6      ; Initialize registers
Stack   MOVE.L  #$4E730000,-(SP)       ; Start loading trace exception
        MOVE.L  #$00000010,-(SP)       ; vector into stack
        MOVE.L  #$0004DDB9,-(SP)
        MOVE.L  #$BD96BDAE,-(SP)
        MOVE.L  #$B386B586,-(SP)
        MOVE.L  #$D046D246,-(SP)
        MOVE.L  #$0246A71F,-(SP)
        MOVE.L  #$00023C17,-(SP)
        MOVE.W  #$2C6F,-(SP)
        MOVE.L  SP,($24).W             ; Set trace exception vector
        ORI.W   #$A71F,SR              ; Enter trace mode
        NOP
        ; Trace vector happens now for the first time.
        ; Code after this line is encrypted.

A disassembly of the trace exception vector that is loaded on the stack:

TraceCode:
        MOVE.L  (2,SP),A6              ; Load return address from
                                       ; supervisor stack.
        MOVE.W  (SP),D6                ; Load condition codes of the main
                                       ; program.
        AND.W   #$A71F,D6
        ADD.W   D6,D0
        ADD.W   D6,D1
        EOR.L   D1,D6
        EOR.L   D2,D6
        EOR.L   D6,(A6)                ; Decrypt 8 bytes ahead in main
        EOR.L   D6,(4,A6)
        RTE                            ; Return from exception

Note that registers altered in the trace vector affect the main program that is being traced. Usually registers are pushed onto stack in any exception vector, because altering them would break the main program. However, purpose of this vector is to obfuscate the code against reverse engineering.

It should also be noted that condition code register (CCR) affects the decryption process. For example, an arithmetic operation in the main program having the 0 number as a result, will cause zero flag bit to be set in CCR. This will cause the value in (SP) to be changed in the trace vector. This is done to obfuscate against reverse engineering.

[edit] List of trace vector decoder systems