AFAIK frame by frame forward depend on the codec used to decode. So, first be sure you have the right codec. DivX movies are played using DivX codec and XviD codec is used for XviD movies. So you will be sure that there is no messup. If in filter dialog you have FFDshow, uninstall it and install latest builds of above mentioned codecs.