ok, I have no idea what you are talking about, at least I'm going to be honest about it
The xbox can only output 1 thing at a time. Whether it be composite (red/white/yellow), Svideo, component (r/w/y and green, blue, red), VGA, or HDMI.
Think of your monitor as a TV, its just a display, what it does with it's inputs has nothing to do with your PC.
So yes, you can hook your xbox with a VGA cord right into your monitor if it has multiple inputs, same with an HDMI or an HDMI to DVI cable (both of which will require that dongle I mentioned earlier for red/white sound outputs). However, even though your computer and xbox share a monitor, you cannot record what is on the screen because the xbox isn't plugged into the monitor.
Now, you bought a USB thing that inputs composite cables or s-video and outputs it to your PC directly through USB, this is how you can record
and view your game on your monitor through windows. You need a program that does this. If one doesn't come with that USB thing on a disc, use Media Player Classic
http://mpc-hc.sourceforge.net/ The biggest problem with this is there is almost always a bit of lag where your computer needs to process what the USB thingy is outputting to it and then it displaying that information through windows onto the monitor. This means that what you see on the screen happened a second ago or so.
What you can do to "fix" this buy some composite splitter cables and plug them into the xbox's composite cables. Then you run one of the splits to your USB recorder, and the other to your TV where you can play like normal without lag.
There is no good way to run a game console through a PC where you can watch/play at the same time as it's recording without game-breaking lag. Especially not with HD.