From the feedback I've read (I've not tested it myself) it seems for Vista the rendering mode to be selected under (rightclick >) Options > Preferences > Video > [Video rendering]
would be (allowing Aero to be switched off) "Internal renderer overlay (default)" or "Internal renderer, DirectDraw surface" or alternatively "VMR9".