VirtualDub Game Recording Guide
Windowed Mode List
VirtualDub Game Recording Guide
2011-10-26 Links to sample footage have been fixed. Also, a few details in the guide have become outdated or inaccurate since its initial posting.
This guide is the result of my search for an alternative to game recorders such as Fraps, Xfire, or Gamecam. While those applications work well, I wanted something that could offer greater flexibility and customization for my game recording needs. This method, based on VirtualDub and other freely available software, provides almost total control over capture settings and system performance.
Read on for a summary analysis, then continue with the detailed guide to try this for yourself. I welcome opinions, tips, and feedback, so please feel free to contribute by visiting the discussion thread.
VirtualDub - versions 1.7.8-1.8.5 have been tested. I recommend the latest stable version.
VH Screen Capture Driver - to download click HERE.
HotkeyP - also download this configuration file and unzip it in the HotkeyP program directory.
PICVideo MJPEG Video Codec - the current version of this codec is shareware; however, the old freeware version (used for this guide) can still be obtained via the Showshifter Demo. Simply install the demo, reboot, then uninstall it. The codec should still be on your system after uninstalling. If the uninstaller asks if you wish to keep the codec installed, choose 'Yes', of course.
Additional useful software:
LAME ACM MP3 codec - for realtime compression to MP3 audio within an .avi file.
ShellEnhancer - utility that allows for X-style window moving and resizing (Alt+Lclick anywhere on a window to move it), window position locking, and much more. Very useful for running games in windowed mode.
Huffyuv lossless video codec - scroll down to the 'Codecs' section to find it.
RivaTuner - enables anti-aliasing for games that don't feature the setting in-game.
Muvixer - application to preseve soundcard recording settings for games that force particular input (Battlefield 2, Team Fortress 2, etc.). Simple and very useful.
Open VirtualDub and enter capture mode by selecting File->Capture AVI. You should see this window (perhaps showing a preview):
Select Device->VHScrCap (DirectShow) to activate the VH Screen Capture Driver. The VirtualDub window should show a preview image of your desktop. If it does not, verify that you have either Overlay or Preview selected in the Video menu. (If neither option works, you might need to reinstall the VH Screen Capture Driver). Later on, when you are ready to being capturing gameplay, be sure to turn off the preview by selecting Video->No Display. There's no need to waste CPU cycles on a preview that will not be visible.
To configure the VH Screen Capture Driver, select Video->Capture filter. You should now see this dialog:
For fullscreen game recording, select the 'Track screen' option as shown. The Width and Height fields should auto-fill to your current resolution. With this setting the driver will automatically adjust its capture area to your current resolution, so if you play some fullscreen games at a lower or higher resolution you won't need to make any changes to this setting beforehand. Once finished, click the Settings button.
tip: hover your mouse over dialog elements to view pop-up descriptions.
Here most of the options are self-explanatory. For now, start with a Frame rate of 30 and leave everything else unchecked. If you want to try using the resize options later on, then check the boxes for Resize to output size, Use linear resize in 32bit mode (better quality, especially for text), and Specify output size. Align video is a setting that will force any resolution you choose to auto-fill to the nearest power-of-4 dimensions (some video codecs are picky about this). I generally leave this checked because it does nothing unless I mess up my resolution settings with odd numbers or ratios--which has never happened (yet). Border color is the color used by 'Align video' to fill the video frame. Leave it black, unless you just love puce. Show Mouse does what it says. If you want to see the cursor, click the box. I have yet to use it. RTS players might want to turn this on, though I got a kick out of watching my Zero Hour units 'magically' perform my commands. Leave the Optimize settings off--they actually reduced performance in my tests.
Here is a list of some common resolutions you can try when using the Resizing options:
When you have finished configuring the capture driver, click the X in the upper-right corner to return to VirtualDub. Go to Video->Set custom format [SHIFT-F] to open this dialog:
You need to make sure that the 'Use custom size' fields have the same resolution values you selected in the screen capture driver, and always double-check this setting after configuring the driver (conflicting settings can cause VirtualDub to error/crash). The 'Data format' field will automatically select 32-bit ARGB, since this is what the driver uses. I don't believe this can be changed--when I tried selecting a different format VirtualDub would always switch it back to 32-bit. Once your resolution values are correct, close the dialog.
To set the capture framerate in VirtualDub you can use the 'fps' button on the status bar:
Set the framerate to the same value you chose in the capture driver. If your framerate doesn't appear in the list--like 24fps--you can set it via the Capture->Settings menu option:
Enter your framerate here, but make sure the Wait for OK to capture option is not selected before you close the box. The framerate should now display correctly in VirtualDub's status bar.
Now go to Video->Compression [C]:
Select the PICVideo MJPEG Codec (as shown) and click the 'Configure' button. The codec settings dialog should look like one of these two screens:
While you can make quick adjustments using the slider, I recommend clicking the 'Advanced' button to expose all of the encoder's settings. The picture on the right shows the settings I use most often. I set the slider at 17 and then bump the Chrominance up to 15 just so it matches the Luminance. The human eye is more sensitive to Luminance, so you can get away with lower values for Chrominance if you wish. I have found that the PICVideo codec offers great picture quality at very reasonable filesizes. Try different compression levels and see what you prefer.
Subsampling - I would leave at 4/2/2. If you want to tweak this setting, this article provides good reference information.
2 Fields If More Than - leave this off. Computer games generate progressive images--no need to deal with fields.
Swap Decompress Fields - off, same as above.
Force YUY2 Output - didn't notice any change or benefit when I tried it, so I recommend off. Experiment if you wish.
Codec Enabled - self explanatory. Make sure this is on.
Once you've chosen your settings, close the dialogs to get back to VirtualDub's main window.
If you want to capture audio in your recordings, select the Audio->Enable audio capture menu option. Next select the audio input from the menu shown here:
On my system the "What U Hear" input option works best. Other audio cards might have this labeled as "Stereo Mix". If you can't find an input like these, try the 'Wave' setting instead. After selecting the input, choose the audio format with the "Raw capture format" option or via the audio button on the main window's status bar:
I generally use 48KHz, 16-bit, stereo audio since it provides great quality and is DVD-compliant. Note that some audio format options may only be selectable via the 'Raw capture format' menu option, as the screenshots above show for 96KHz.
You can choose various audio compressors with the Audio->Compression option:
I recommend the '<No compression (PCM)>' option for a few reasons. First, the disk space needed for audio is negligible compared to what you need for the video, so you won't get any significant space savings by compressing the audio in realtime. Second, compression will add some processing overhead on your CPU that could hinder game performance. Third, capturing uncompressed audio gives you a 'master' source that you can then compress to any other format without the quality reduction incurred by transcoding from one lossy format to another. If you do wish to compress the audio in realtime, I recommend using the LAME ACM MP3 codec--just make sure you match the MP3 encoder settings to the same frequency and channels you selected in VirtualDub's audio format menu. I have successfully tested capturing to MJPEG+MP3 as well as Xvid+MP3 .avi files, so any number of compressor combinations could be possible.
Once you have finished configuring your audio and video capture options, got to File->Set capture file [F2] to choose a directory and filename for your game recordings. You must choose a capture filename every time you open VirtualDub's capture window, even if you do not change any video or audio capture settings. If you have multiple hard drives I recommend capturing to a drive other than the one with your game installations and/or Windows OS. This will guarantee the best write performance possible and will prevent stressing a single drive with game loading, video recording, and managing swap space simultaneously. Also be sure to enable the Capture->Autoincrement filename after capture setting so that you can perform multiple captures during gameplay.
VirtualDub has customizable hotkeys available in the Capture->Preferences menu, but they do not function reliably when running a game. In order to use VirtualDub's capturing abilities without exiting your game, you need the global hotkey functionality of a program like HotkeyP.
Run HotkeyP and click the system tray icon to open the main window:
HotkeyP is an advanced program. While it is logically designed and has an informative helpfile, it can be confusing to learn. To simplify this guide I have included a copy of the hotkey configuration file that I use for game recording, which you should already have unzipped in your HotkeyP program folder. To load the file, select the File->Open hotkeys menu option, navigate to the HotkeyP program folder and select vdub.htk. The main window should now show the following hotkeys:
VirtualDub requires 1 key to start recording and 1 key to stop (defaults are F5 and Esc, respectively). I have found the Numpad+ and Numpad- keys to work well in HotkeyP since I can hit them quickly with my thumb while gaming, but you can use any keys or combinations you want. Highlight one of the hotkey entries and click the Edit button to open this dialog:
Be sure both hotkeys have the correct path entered for VirtualDub. As you can see, my Program Files are on Drive F. Yours are probably different. To customize different hotkeys for recording, change the entry at the very top of this dialog window. I prefer the simplicity of a single keystroke, but you can configure just about any combination of keys using Ctrl, Alt, Shift, or even the Windows key.
Once HotkeyP is configured, you are ready to start recording. The recording process is actually rather simple: HotkeyP monitors your keyboard while you play a game, then sends VirtualDub 'start' and 'stop' recording commands when you press the associated keys, and VirtualDub records your videos to the folder you selected. Obviously there is no on-screen recording indicator, so you will need to mentally keep track of when you are recording (sometimes I randomly hit the 'stop' key, just in case I've forgotten that I started a recording earlier). When you are finished playing you'll have a collection of .avi files ready for playback or editing.
I recommend testing by recording your desktop first, then try some games. For HotkeyP to successfully send keystrokes to VirtualDub, the VirtualDub window must be open on your desktop and not minimized to the taskbar. Also remember to turn off VirtualDub's preview to save system resources. HotkeyP can be minimized to the system tray.
Windowed mode recording
Nearly all games render using hardware overlays/Direct3D surfaces. Because of this, general-purpose screen capturing applications like VirtualDub cannot successfully capture game images, but instead end up with a solid color image like black or purple (the color key used by the overlay). When I capture games in fullscreen on my system, the videos constantly flicker because the capture driver sometimes captures the overlay color instead of the game image, resulting in black frames peppered throughout the video. The first thing to do is try recording some of your games in fullscreen. If it works, awesome! Please post infomation about your results and system specs in the discussion thread. If your videos have flickering, read on.
There are a few ways to deal with this problem. With my old graphics card, I could successfully record most games in fullscreen just by enabling anti-aliasing. With other games, if I opened Media Player, played a video, then paused the video but left Media Player open, it would prevent the game from using the hardware overlay and I could thus record the game in fullscreen without issue. Games can obviously run without this feature--since it is not used in windowed mode--but I have not yet discovered how to universally disable hardware overlays so that games won't use them when running fullscreen. If anyone reading knows how to do this, please share the information!
If neither of the above tips works for you, then running your games in windowed mode will allow you to record game videos successfully. Almost every game can run in windowed mode by one means or another. Here is a list I have compiled of many popular games along with windowed mode instructions.
Scenario 1 - Games featuring a windowed mode that renders at fullscreen resolution with no window border. (best case)
Example games: Valve's Orange Box engine games (Team Fortress 2, Half-Life 2: Episode Two, Portal)
Games in this category essentially look and function like they are running fullscreen, even when windowed. For example, if you launch Team Fortress 2 with the command line parameters -windowed and -noborder, the game will run at maximum resolution, without a window border, perfectly aligned to the edges of your monitor. You can leave the capture driver set to 'Track screen' to record these games.
Scenario 2 - Games featuring a windowed mode that renders at fullscreen resolution with a window border.
Example games: Lord of the Rings: The Battle for Middle-Earth series, some games when run with D3DWindower
For games like this, I use ShellEnhancer's Alt+Lclick window dragging feature to center the game window so the window border is off-screen, allowing the game to fill 100% of the monitor as in Scenario 1. The 'Track screen' setting can be used here as well.
The only other option for these games is to run them windowed at less than your maximum screen resolution. In this case, refer to the instructions in Scenario 3 below.
**Note that running an RTS in windowed mode will likely disable the ability to pan the game camera by moving the mouse to a screen edge. Most RTS games also allow panning by arrow keys and/or click+drag, so the loss is minor.
**To disable the Taskbar's 'Always On Top' setting, right-click on it, choose 'Properties,' then un-check 'Keep the taskbar on top of other windows.'
Scenario 3 - Games featuring a windowed mode that includes an always-visible window border on screen.
Example games: most games fall in this category
There are 2 methods available, depending on your workflow preference:
1. Capture fullscreen, crop in post.
Example - Star Wars Battlefront 2
The game looks like this in windowed mode (set to my maximum resolution, 1440x900, in the game options menu):
Note: I use ShellEnhancer's auto-manage feature to lock the window position to X:0 Y:0 coordinates.
Notice the slightly visible window border at the top and bottom of the screen. Leaving the game running or paused, Alt+Tab to the VirtualDub window left open on the desktop, then access the capture driver via Video->Capture filter:
Select the Region Tool as shown above. A crosshair cursor will appear as well as a dialog box showing realtime pixel coordinates feedback. Selecting your capture region is a 2-click process: select one corner, then the corner diagonally opposite:
The capture driver dialog window will auto-fill the values you selected:
The Width and Height values are most important. Make sure these are even numbers, preferably powers-of-4. You can manually correct them in this dialog if needed. If you want to use the realtime resizing features of the driver, click the Settings buttons and enter your chosen values.
Now you can close the driver dialog, but before you go back to the game and begin recording you should always Alt+Tab to VirtualDub and hit Shift+F to verify the resolution values in VirtualDub match exactly what you chose in the capture driver. VirtualDub normally auto-detects the driver's settings, but a mismatch will likely cause a crash that you won't notice until you have exited your game (and thus you will have no recordings).
Some games behave differently than others in windowed mode, but you can extrapolate the steps here to accomodate most any recording situation. I hope this guide proves useful, and don't forget to share your experiences, questions, or tips.
Notes / Troubleshooting
For reference, this guide was created and tested primarily on a machine with the following specs:
Encoding application- VirtualDub
Source video- 1280x800, 30fps, 0:30 game recording
VirtualDub controls processing thread priority. All encodes were done at the default 'Normal' setting, resulting in 65-70% CPU usage across the board.
-Results-19fps- Lagarith lossless
23fps- Xvid (realtime capture profile)
28fps- MS MPEG4 V2 (used by GameCam)
32fps- HUFFYUV lossless
37fps- Xfire lossless
54fps- PicVideo MJPEG
|Copyright 2008-2011 genadmission.com.