describing your problem more precisely would help...
can't you make captures at all ? are your captures 'scrambled' ?
see here
http://forum.bsplayer.org/viewtopic.php?p=3169
if your captures are 'scrambled', you can try and change overlay settings (preferences --> video), disable overlay, use VMR9...
in current build 501, it looks like using the convert formats option is not a good idea