§ ¶VirtualDub 1.8.1 released
I've finally pushed out 1.8.1 as a stable release, which promotes the new features added in 1.8.0 to stable status. If you are interested in features that were previously only in the 1.8.0 experimental version or are having trouble with the previous 1.7.8 stable version, I recommend checking out 1.8.1 to see if your issue has been fixed. This version was supposed to go out last week, but I was stymied by a login problem in the SourceForge upload system.
There are a couple of new features in this release, the main one being the distributed job support, which allows using different instances of VirtualDub to create and run jobs, and you can even use multiple machines if you get all the paths and plugins synchronized. It uses a filesystem-based sharing model, as it's designed for ease of setup with a few machines rather than a whole cluster farm. There is also now support for running the video compressor in a second thread for better dual-core performance, and now that I have a readily available 64-bit station again, I'm starting to bring the AMD64 build up to feature parity with the 32-bit version.
I've been really busy at work lately, so I'm afraid my progress on VirtualDub will be substantially slowed for the next few months (more so than usual), but I do have a few things already in flight for 1.8.2/1.8.3. Nothing big, but one of the things I've been doing is reversing some of my assembly-only features to C so I have an easily maintained baseline and so that the features can be enabled for AMD64. I've also been doing some prototypes here and there for the next big change, but nothing concrete's surfaced yet.
Changelist after the jump.
Build 29808 (1.8.1, stable): [June 15, 2008]
* The 'run as job' setting is now saved in the Save AVI dialog.
* Added distributed job queue mode.
* Added bob and non-interlaced field display modes to render preview.
* Added new test video mode: interlaced cube.
* Added option to run video compressor in a separate thread for better dual
* YCbCr resampler is now SSE4.1 optimized.
* Added command-line switches for minimizing/maximizing the window on
startup and setting process priority.
* AMD64: The threshold, grayscale, levels, logo, and brightness/contrast
video filters are now available.
* Plugins: Fixed bugs with and raised size limits for serialized input
plugin options data.
* Fixed cases where the crop/letterbox to aspect ratio options in the resize
filter were broken.
* Fixed another rare crash when exiting filter list dialog.
* Fixed Postpone and Delete buttons in job control dialog sometimes not
updating when a job state changes.
* Fixed swapping of AVI superindex and subindex settings.
* Fixed bugs with cropping in filter chain with YUY2 or UYVY formats.
* Mouse wheel scrolling with Shift held down (by key frame) now works
* Fixed infinite loop when attempting to convert a pal8 source to 4:2:0
* The initial load of AVI files is now faster over a network.
* TARGA files are no longer written all black when 32-bit RGB output is
* Added workaround for "image not in Y or YCbCr format" errors when reading
JPEGs from a RAZR V3 phone.
* Deleting a filter in the filter list no longer causes the checkboxes to
desync from the actual filter enable states.
* AVI: Files with truncated hierarchical AVI indices no longer result in
"missing 'movi' chunk" errors and can now be recovered.
* DV: Fixed decoding issue that resulted in some lost blocks.
* DDraw: Fixed occasional crash when another application forces full-screen
* Capture: Fixed crash in OpenGL screen capture mode related to occlusion
query based frame dropping.
* Capture: Fixed hang on shutdown when exiting with OpenGL screen capture
* Fixed crash when a script specifies arguments for a video filter that
doesn't take any.
* Data rate was reported incorrectly for the video stream in the status
* Fixed audio display.
* Fixed crashes and decompression errors with paletted video.
* Filter preview no longer shows bogus frames when previewing a filter chain
with edits on the timeline and no frame rate changing filters.
* Filters: Codec-friendly alignment works in resize filter again.
> Added option to run video compressor in a separate thread for better dual core/SMP performance.
Interesting. Would this adversely affect codecs that already distribute their own processing load, such as DivX? My gut feeling is that, unless processor affinity is set for the second thread, there would be just the overhead of that newly created thread (which would probably be minor). If processor affinity were set, that might free up one of the cores to do input decoding and running the filters.
I'm a nitwit when it comes to multithreading, so I can't predict what would happen in practice. May be interesting to find out.
Anyhow, great work on all the new features and fixes!
Steven - 15 06 08 - 20:21
The two settings are different. The setting in the codec tells the codec to try splitting its work internally across multiple threads, but it's constrained by the codec interface requiring one-frame-out-per-one-frame-in. The setting in VirtualDub is different because VirtualDub can restructure its pipeline to avoid the need to sync on the way in and out of the codec, increasing parallelism, but it's still limited by the back-to-back frame speed of the codec. In practice neither is 100% effective, so your best bet is to try both at the same time! Since 1.8.1 isn't able to run more than one codec thread -- which would be incompatible with delta frame compression anyway -- enabling both threading options lets you scale higher.
Setting the processor affinity on threads in general is a bad idea. There are other threads in the system, so setting affinities simply constrains the OS scheduler and introduces additional blocks into parallel operations. Kernel schedulers in general try not to hop threads unnecessarily between CPUs for cache efficiency reasons, so the only major time it makes sense to do so is when you are trying to avoid a memory bottleneck caused by two highly-talkative threads being on separate cache hierarchies (which strongly implies that your application threading model is inefficient anyway).
Phaeron - 15 06 08 - 20:36
Many thanks for 1.8.1!
One thing I already noticed in 1.7.x:
The "Job Control", "Create Test Video" and "Preferences" windows just have this standard program icon in their title bar. Can you add the proper VirtualDub icon in those title bars too? Would look a bit more "complete" ...
Dstruct - 16 06 08 - 03:18
Just been trying out the new features Ė fantastic! They don't seem to be documented at all though Ė had to do a bit of guesswork to work out how to use the shared job queue. For the record, to use a shared job queue:
1. File | Job control...
2. File | Use shared job list...
3. (Optional) select "Autostart" if you want it to start transcoding as soon as any jobs are available.
And to encode on a separate thread:
1. Options | Preferences...
2. Threading: 1 video compression thread.
Anyway, configuring Vdub to encode on a separate thread definitely improves throughput. Previously I was getting about 75% CPU utilisation (ffdshow-tryouts, XviD with 2 threads, TextSub to add subtitles). Now getting an average of about 93% utilisation, and a definite increase in FPS.
Secondly, the new shared queue is great. My normal setup is 3 dual-core machines on a gigabit network (I can add more single-core machines as well if I want). Previously I had to work out an appropriate number of jobs for each machine. Now I can set up 2 machines to auto-start the shared queue, and add jobs on the third (the start it when all jobs are added).
And thank you for having it remember that Iím adding to the job queue. When you're adding 20-30 jobs having to remember to click the box each time was painful.
A few comments/observations:
1. When using a shared queue, need to be careful to use UNC paths. Maybe Vdub should warn if a a non-UNC path is used for input files or output files if a shared queue is being used, including files used by filters such as subtitles. Although a mapped drive would work (if all machines have the same mapped drive) ...
2. Vdub always defaults to the local job queue when started up Ė it doesnít remember that a shared queue was selected. Iíd definitely prefer it to remember the shared queue.
3. Something I donít think you can do anything about Ė the number of encoder threads is propagated to all machines (configured in the codec). So if the job is added with a single encoding thread, whichever machine takes that job will only use a single encoding thread.
4. I would suggest defaulting the encoding and preview priority to "lower". So long as thereís not another task using copious amounts of CPU, throughput basically doesnít change, but responsiveness of applications is greatly increased.
5. Might it be possible to add the throughput (FPS) to the job control window? Maybe shorten the progress bar (to line up with the job list) and add the FPS under the buttons. I often find Iím manually opening the status window to get some idea of throughput, and of course it goes away when the next job starts ...
Tim - 16 06 08 - 09:43
6. With a shared queue list (with jobs handled by multiple machines), selecting "Edit | Delete completed jobs" only deleted some of the jobs. I think it deleted as many jobs as had been handled by the machine on which "Delete completed jobs" was selected - but it deleted jobs that had been completed on other machines. Executing 3 times cleared the list. So I suspect "Delete completed jobs" is working out how many jobs this machine completed, and just deleting that many from the list.
Tim - 16 06 08 - 09:51
Hmm, that's odd. I wonder why it's using the placeholder icon. Shouldn't be hard to fix.
Weird, I'll see if I can reproduce the delete bug. Whenever you manipulate the job list, the instance you're using has to sync to the version that's on disk, and there's probably a bug somewhere in that. (In retrospect, a journal may have made some parts of this easier, but I wasn't quite sure how to do it.)
Continuous status updates are possible, but may not be a good idea the way the job list works right now -- because the more often the machines update it, the more often they're likely to conflict on a file lock, and are then forced to retry.
There are a number of command-line switches that you can use to aid in multi-machine setup -- run vdub /? to get the list. The interesting ones are the /low, /master, and /slave switches.
Phaeron - 17 06 08 - 03:34
"Video Levels" and "Select Raw Audio format" windows (capture mode) also only have this placeholder icon in it's titlebar. Maybe there are even more Windows with this ...
Dstruct - 18 06 08 - 09:43
Ok, found another windows with this placeholder icon in itís titlebar: "Real-Time Profiler" (capture)
Dstruct - 19 06 08 - 04:43
I wasn't thinking of the throughput being recorded in the job queue (what a horrible thought - multiple machines *constantly* aquiring and releasing a write lock on a networked file). I was just thinking it would be useful to display the throughput of the job that is currently in progress on that machine i.e. the exact same FPS that is displayed in the status window while encoding. You can already get to the info, but you have to go through a number of additional steps, and it only lasts for the file currently in progress.
I think it's probably not as useful now as it would have been in 1.7 and earlier, as I used that infor to decide how to split up the job queue.
I think you've been confused by the username being placed at the bottom of the post here ;) Phaeron was the one who said he should be able to fix the placeholder icons ...
Tim - 19 06 08 - 18:10
It would seem you've introduced a bug with the 1.8.1 version. The scene next/prev Ctrl-Shift-Right/Left no longer works; it just advances two frames.
I use AVISynth to pass a WMV file to VirtualDub; the script has one line only and that is the DirectShowSource for the WMV file. I just download the 1.7.8 version and placed it into a seperate folder and the scene next/prev works (just like it did before I replaced that version with 1.8.1). Note I do run my env within Virtual PC; I've not tested 1.8.1 on the host side.
kudabird - 19 06 08 - 20:02
Just a follow up to my last an scene detection, if I load the WMV file directly into VirtualDub utilizing fccHandler WMV plugin (File Information -> Decompressor = WMV3) or modify the AVISynth script adding ConvertToYUY2 (File Information -> Decompressor = YUY2) then scene detection works. The AVISynth script without the conversion show File Information -> Decompressor = YV12. Again, this is for 1.8.1 (version 1.7.8 is OK). Script is simply the line:
Scene detection values default and same between the versions. Hope that helps to narrow down the problem.
kudabird - 19 06 08 - 20:23
My appologies - I meant to say that loading the file with the fccHandler WMV plugin does NOT fix the scene detection function.
kudabird - 19 06 08 - 20:31
LTRFTW. I like the "save to job list" checkbox remembering, don't know how I lived without it! I have a feature request that I hope isn't too much trouble: add "Run video analysis pass" to the job list. I ask because it seems a waste when batching 2-pass jobs to save the resulting AVI file when it won't get used.
Turkey Machine - 20 06 08 - 00:15
@Tim: yes, i already noticed that is was Phaeron ;)
Dstruct - 20 06 08 - 03:55
The scene detection problem is confirmed. What's happening is that the input plugin is operating in a YCbCr mode, and this breaks the scene detection code. This didn't happen in 1.7.8 because that version always forces RGB decompression for the timeline view, whereas 1.8.0/1.8.1 try to match the input color setting. The code doesn't do format checking and tries to scan a UYVY or YUY2 source as RGB16, which obviously doesn't work. You can work around it by forcing RGB for the input color mode (it's possible that an input plugin doesn't support this, but then it wouldn't have worked in 1.7.8). I'll fix in 1.8.2.
Also, I've found and fixed one of the distributed job queue bugs. Short answer: delete VirtualDub.jobs before using distributed job mode. Otherwise, the revision number in that file can screw up the sync/merge process on the distributed queue.
Phaeron - 22 06 08 - 19:10
Bug in the 1.8 version(s)!
I use Virtualdub (or exakter Cutassistant with virtualdub) to cut out commercials from TV Records by using cutlists. But after I use VD 1.8.1 nearly 50% of the Output Files have VBR Audiostream Errors (the Input File is clean). If I use VD 1.7.8, the Outpufile is clean. If I use the VD 1.8.0 Version, i get the VBR Error in the Output File.
I think, it must be a Bug in the 1.8x Code. Can you please fix it?
Georg Hoeren - 26 06 08 - 04:08
I just thought of a minor issue I have with VirtualDub all the time. I added it to the context menu for certain file types and I right-click them and select "Open in VirtualDub". All would be well if VirtualDub could be so kind to remember that it should open maximized (in other words to save window position or at least minimized/maximized/normal state). Is that already possible by any chance and if not why not and when could you add it?
Igor Levicki (link) - 05 07 08 - 21:35
@Igor: It already saves the window position and size properly here. Just maximized state isn't saved and restored. I agree, that this would be great!
Dstruct - 06 07 08 - 03:37
Request: in "VirtualDub Job Control" window, a button to "CLEAR ALL JOBS" or "CLEAR DONE JOBS". Thank you.
i386 - 12 07 08 - 14:28
@Igor: This is already possible thanks to a built-in Windows program. Add "START /MAX " (without the quotes) to the beginning of your VirtualDub commandline to start it maximized.
An example would look like:
start /max "C:\vdub\VirtualDub.exe" "%1"
Simon E. - 13 07 08 - 12:23
Maximized state will be saved in 1.8.2.
START /MAX works for all versions of VirtualDub, but note that in 1.8.1+ you can also just do VirtualDub /max as well.
Phaeron - 16 07 08 - 03:42