Does the DVDtoAVI program rip and encode on-the-fly? Perhaps you may need to play the movie for a few seconds so the entire structure of it is recognized.
On the fly ripping/encoding can be detrimental for a DVD drive. It can place a lot of stress on DVD hardware. Try ripping the whole DVD to hard disk, then encode that to AVI.
DVD Decryter and
SmartRipper are some of the programs that come highly recommended for backing up/ripping DVDs.
Programs like
VirtualDub, or any of the popular mods, like
VirtualDubMod, for example, can be used to encode from the DVD/VOBs etc to AVI files.