Monday, May 21, 2007

Melancholy of a Travelling Man









Low resolution 320x200 version further compressed by Youtube Sorenson codec.


Shot in PF24 mode mostly in TV 48 with custom effect. Shallow DOF in the opening scene was achieved using 28mm Nikon f/2.0 at full wide. The wild flower scene used the standard lens taking advantage of the shallow DOF inherent in the 1/2.7" sensor. Custom effect was Color Depth 0, Sharpness -1, Contrast -1, Brightness -1. The only shot with color correction was the city lights view from the air. Gamma was adjusted to hide the low light grain.

Monday, May 14, 2007

Things to consider for 24 frames per second

I've almost always shot progressive. It builds discipline. The higher frame rate of 60i has it uses but most of the time I still shoot as if I'm on 30p/24p. Here is a video to help illustrate some of the differences in style when using lower frame rates. 35mm users trying to emulate film style probably learn this as they go along.


Sorry, video unavailable due to Stage6 shutting down.

Original video is 720x400 which can be run fullscreen.
Apologies if DivX player doesn't agree with you.



The two basic rules are:
1) Avoid sudden movements
2) Try to shoot from a fixed position

All these shots were taken with a monopod. In the beginning of the above video I do a relatively slow pan. As you will notice, the fast moving objects like cars and people show a bit of the 24p stutter. Higher frame rates are better for higher movement. If you need to move fast or capture really fast objects look at 60i. Besides the subjects you have to consider your own movement. Panning takes patience and thinking ahead. You have to learn how fast you can twist the video head and appear smooth. And don't try to stop the head too quickly unless its an effect you mean to use. It can be jarring.

In the middle part I walk with the monopod to the middle of the center island. The shock of walking, especially with my flat footed step, can send the camcorder bobbing and yawing all over the place. The stuttering effect of 24p accents this. Unless you're trying to duplicate the beachhead scene from Saving Private Ryan you may not want to move around with 24p unless you have a well balanced base. I've driven in cars with 24p and gotten good video. But walking is always tricky.

In the later middle portion I show 24p sped up. Its a bit of a cheat. I would have used a sky scene but didn't have one. Sped up footage of nature scenes look nice when all the frames are progressive.

In the last portion I zoom in a bit. You have to realize that zooming means that any unintentional movement of the camera results in big movement of the image. So practice the steadiness of your hand. Try to keep a comfortable position so your nervous body movements don't transmit from the monopod/tripod to your camera.

From one amateur to another, have fun working in 24p.

Friday, May 11, 2007

Making true 24p sources out of HV20 m2t


24p inside 60i with pulldown

This happens on TV a lot so you may not even notice this with your HDV.

24p with pulldown removed

You would pretty much have to be using a computer or some progressive display to notice the difference.


Introduction
Hi there! You're probably reading this because you want to convert your Canon HV20 (or whatever with 24F/P in 60i) footage to true progressive format prior to editing. Whew. Let me begin by saying not everyone needs to do this. If you're adding lots of special effects and resizing your output to a smaller HD (eg. 1280x720) it may matter because of "combing". Or simply because you like to see your frames individually as clear as possible. Maybe you are a film enthusiast and your final output needs to be its progressive best. Doing Inverse Telecine/Pulldown Removal (we'll call it IVTC now) will not compensate for bad video. It just removes the blur/ghost produced by that one interlaced frame every few seconds. Most people will not even notice. I barely notice it.

This IVTC process won't save you hard drive space because you need to generate large files with a lossless codec if you really want to preserve the bit-by-bit look. If you compress with WMV9 or XViD you're doing lossy editing which may lose more detail if you add in text effects or overlays. It's up to you. Maybe your audience won't notice. Call it an artistic device!

Anyway, you can still work with your raw m2t or dvr-ms to produce fantastic looking output. DVR-MS used by Windows Movie Maker is stored with WMV9 compression. That format does not allow interlace therefore it is automatically deinterlaced for you! If you just need something simple to share with friends and relatives you can recompress to WMV. Even Mac users should have a codec to read those! As for Linux, they can figure it out.

This process is technically involved. The open source software hasn't been glued together enough to be easy to use. Get one thing wrong and it may not work at all. If you have no patience for this, stick with Microsoft's Movie Maker or stay with interlace. If you need to do this for a living then go spend the extra cash to buy Cineform's software. They cost a pretty penny but they simplify this process and do some extra high end stuff.

Have I knocked some sense into you? No? Oh well, then read on.

The Farnsworth plus Sillyman Process
Forgive me Farnsworth, whoever you really are, for I am about to immortalize your name with your own process. His original process is posted here. He listed down the tools you needed. They are all free, from extracting the m2t to performing IVTC and then rendering to an intermediate edit format. The only non-free part is where you actually start using an NLE. Unless you're using VirtualDub/Windows Movie Maker you probably had to buy an HDV capable editor.

Here's an overview of the workflow with the improvements I've made:

1) Extract m2t using HDVSplit (NLEs like Vegas can do this too)
2) Run DGIndex and fixavs.pl via go.bat to identify the progressive and interlaced frames and save it to a d2v file. Generate an MPA file for audio to go with it. Create an Avisynth (AVS) file with instructions to use the d2v and mpa with the correct audio delay.
3) Load the AVS file into VirtualDubMod and select your compression format. Then Save As to an AVI file.
4) Load your new AVI file to your choice of NLE for editing. Then render to your desired format.
5) Delete or archive the source files that you no longer need.

Getting Step 1-3 To Work
We'll only cover Steps 1 to 3 in this process. 4 to 5 is all yours.

A) Download the Software
ActivePERL 5.8
HDVSplit 0.77 Beta
VirtualDubMod 1.5.10.2 All inclusive
Avisynth 2.57
DGIndex 1.49 RC2
TIVTC v1.01 plugin ~search TIVTC
MPASource plugin ~search MPASource
MSU Lossless Codec
Lagarith 1.312 Codec
Huffyuv 2.2.0 Codec
Farnsworth plus Sillyman 24p scripts

B) Install HDVSplit 0.77 Beta (you don't have to if your NLE already makes m2t with scene splitting)
C) Create a directory called 24p (anywhere you'd like)
D) Decompress VirtualDubMod 1.5.102 into that 24p directory.
E) Decompress dgmpgdec149rc2.zip into that 24p directory; RENAME IT to dgindex
F) Go to the dgindex directory and look for DGDecode.dll; COPY THAT to c:\Program Files\Avisynth 2.5\plugins
G) Open TIVTCv101.zip and COPY the TIVTC.dll to c:\Program Files\Avisynth 2.5\plugins
H) Open mpasource_25_dll_20040109.zip and COPY mpasource.dll to c:\Program Files\Avisynth 2.5\plugins
I) Right-click msu-ls-codec.exe and Run as Administrator (For Vista -- XP you can run as normal) and follow the prompts to install
J) Decompress Lagarith_1312.zip and RIGHT-CLICK INSTALL lagrith.inf; you can delete the decompressed files aftewards
K) Decompress Huffyuv 2.2.0 and RIGHT-CLICK INSTALL huffyuv.inf; you can delete the decompressed files afterwards
L) COPY the go.bat, template.avs, and fixavsdelay.pl into the 24p directory
M) Double click on the activeperlxxx.msi file to install and follow the default options

Your 24p directory should look something like this:
D:\Capture\24p>dir
Volume in drive D is DATA

Directory of D:\Capture\24p

05/12/2007 08:48 AM DIR .
05/12/2007 08:48 AM DIR ..
05/11/2007 07:01 PM DIR dgindex
05/12/2007 07:03 AM 1,526 fixavsdelay.pl
05/11/2007 10:55 PM 173 go.bat
05/12/2007 08:48 AM 5,576 README-YouSillyMan.txt
05/11/2007 10:57 PM 117 template.avs
05/09/2007 09:35 AM DIR VirtualDubMod_1_5_10_2_All_inclusive


Your plugins directory should look something like this:
C:\Program Files\AviSynth 2.5\plugins>dir
Volume in drive C is OS

Directory of C:\Program Files\AviSynth 2.5\plugins

05/11/2007 07:07 PM DIR .
05/11/2007 07:07 PM DIR ..
07/05/2005 08:04 AM 7,129 colors_rgb.avsi
05/07/2007 10:53 PM 192,512 DGDecode.dll
12/24/2006 05:12 AM 24,576 DirectShowSource.dll
01/09/2004 08:55 PM 135,168 mpasource.dll
12/24/2006 05:12 AM 112,640 TCPDeliver.dll
02/19/2007 05:52 PM 456,192 TIVTC.dll


How To Use The Script

A) Run HDVSplit to extract all your m2t scenes (in this example we'll assume D:\Capture)
B) Start -> cmd , this will launch a command shell
C) Go to your 24p directory where go.bat is found (in this example we'll assume D:\Capture\24p)
D) in the command shell type "go D:\capture" -- USE ABSOLUTE PATHS; DGindex will launch once per m2t file

Sample filenames generated:

05/08/2007 06:11 PM 35,451,392 CaptureTemp_0023.m2t
05/12/2007 09:01 AM 492,626 CaptureTemp_0023.m2t_ MPA PID 814 DELAY -236ms.mpa
05/12/2007 09:01 AM 208 CaptureTemp_0023.m2t_.avs
05/12/2007 09:01 AM 1,800 CaptureTemp_0023.m2t_.d2v
05/12/2007 09:01 AM 214 CaptureTemp_0023.m2t_fixedaudio.avs


E) There will be two sets on avs file for each m2t. You want the one that says xxxx_fixedaudio.avs
F) Run VirtualDubMod and drag the xxxx_fixedaudio.avs into the open window
G) Video -> Compression (pick a lossless one like LSU or whatever you like)
F) File -> Save As (you can use fast recompress since or full processing, doesn't matter), give it a file name
G) open that file in your NLE and begin editing
H) When you're all done you can delete the files as you see fit

The procedure is still rough but its a good start. Here is some sample output.

UPDATE1
Some have had trouble following the instructions for various reasons. I've put together the pieces that don't explicitly disallow bundling. Hopefully this link is still valid when you find this.

http://www.sendspace.com/file/d71x3y

The contents have an installme.bat. Run that to create a c:\capture directory. Inside the 24p directory specified in E-H and J are done. You still have to do the rest manually but it will help get you started if you run into errors.

You may want to read the thread that spawned the above "patch" at http://www.dvinfo.net/conf/showthread.php?t=91500&page=2


UPDATE2
Steve Szudzik consolidated the above process and the improvements in update1. You can find his exe and procedure at http://www.szudzik.net/vegas/scripts/HV20Pulldown.html. Therefore the procedure outlined above is mostly for reference now. If you have problems with Steve's method look to the components that made it work.

fixavsdelay.pl+go.bat, this is replaced by Steve's HV20Pulldown.exe

#!c:\perl\bin\perl.exe
# You can install ActivePERL 5.8 to parse this for you
# This file is fixavsdelay.pl Version 1
# Parse AVS files in directory with proper delay
# by Mike Dulay -- http://yousillyman.blogspot.com
# This code is released under GPL to align with the spirit of the code this works with. It's free and open source, okay? Just use it and abuse it. But any mistakes from here on are you own! :-)

# My template.AVS:
#vid=MPEG2Source("__vid__")
#aud=MPASource("__aud__").delayaudio()
#audiodub(vid,aud)
#TFM(d2v="__vid__")
#tdecimate()

# My go.bat
#COPY template.avs %1
#FOR %%f IN (%1\*.m2t) DO dgindex\dgindex.exe -if=[%%f] -FO=0 -OF=[%%f_] -AT=[%1\template.avs] -EXIT
#FOR %%f IN (%1\*.m2t) DO perl fixavsdelay.pl %%f


sub mstos {
my $ms = $_[0];
#preserve the "-" sign if present
my $sign = "";
$sign = '-' if ($ms=~/^\-/);
$ms =~ s/^\-//;
my $secs = sprintf("%.3f", ($ms/1000));
return $sign.$secs;
}

my $filename = $ARGV[0];
open(AVSFILE,"<$filename\_\.avs");
open(OUTFILE,">$filename\_fixedaudio\.avs");

foreach my $line () {
if ($line =~ /DELAY /) {
my $subline = $'; # part of string after word DELAY
$subline =~ /ms\.mpa/; # part of string between delay and ms.mpa
my $delaystring = $`;
my $seconds=mstos($delaystring);
$line =~ s/delayaudio\(\)/delayaudio\($seconds\)/;
print OUTFILE $line;
} else {
print OUTFILE $line;
}
}
close(AVSFILE);
close(OUTFILE);

__END__



rem COPY template.avs %1
FOR %%f IN (%1\*.m2t) DO dgindex\dgindex.exe -if=[%%f] -FO=0 -OF=[%%f_] -AT=[template.avs] -EXIT
FOR %%f IN (%1\*.m2t) DO perl fixavsdelay.pl "%%f"


template.avs, this was my version

vid=MPEG2Source("__vid__")
aud=MPASource("__aud__").delayaudio()
audiodub(vid,aud)
TFM(d2v="__vid__")
tdecimate()


farnsworth.avs, this was Farnsworth's original which Steve also uses as a base

v=MPEG2Source("__vid__")
a=MPASource("__aud__")
audiodub(v,a)
TFM(d2v="__vid__")
tdecimate()
DelayAudio(-0.222)


UPDATE3:
Here's a reference to the HV20 forum where Farnsworth first posted his method.
http://www.hv20.com/showthread.php?t=218&page=13

Eugenia Loli-Queru also posted improvements to Steve's documentation (it's great how the improvements keep rolling).
http://eugenia.gnomefiles.org/2007/07/13/canon-hv20-24p-pulldown/

Wednesday, May 9, 2007

HV20 Non-LANC controller

Now that I've built a figrig its great for moving around. But starting or stopping the recording means having to let go with one hand. Unfortunately, there is no LANC available for the HV20. Not to be deterred I built one based on an earlier idea.






Parts
1 Canon WL-D85 Wireless Controller
1 Toslink optical cable
1 Pack of Sticky Back Velcro
1 Roll of Electrical Tape
1 Plastic fork/spoon/knife (flat handle)
1 Flat plastic/credit card
4 rubber bands
1 strip of aluminum foil

Tools
1 Pair of Scissors

Start by testing the principle. The HV20 sensor is that smoke colored plastic on the lower right side (left if you're looking directly at it) of the lens. Get a fiber optic cable like a Toslink (aka SPDIF optical) cable and line up the HV20's controller with one side of the pickup. Point the other end of the cable at the sensor. You can control the camcorder from behind! The trick is getting the controller and cable to line up. Same thing with the sensor and the other end.

The Controller Mount
I start by cutting up an old plastic card into 2/3 and 1/3 pieces lengthwise. The 1/3 piece is used as a spine and extends out the top by an inch to be a fastener for the cable. Wrap the pieces together with electrical tape.



Do a dry fit of the cable and the controller to see how best to line them up. This will determine where to place the sticky velcro. Attach two pieces of the loopy velcro pad to the back of the controller. Put two corresponding pieces on the mount. Dry fit the cable again. Based on where the body of the optical head lies, cut out two tabs on either end of the mount with your scissors. This will be where the rubber bands hold on to the cable.





Dry fit with the cable and some rubberbands. The objective is to keep the LED on the controller aligned with the cable head. Knowing its position, remove the rubber band and tape a hook-pad of velcro to the mount. Then wrap the loopy pad to the body of the cable head. Both ends should now line up nicely but the mount tends to flex up and down. You can get a handle from plastic spoon or fork for an extra spine to the mount. This will prevent the mount from flexing the head and LED into misalignment.

Now you have a working controller. Test it by pointing the other end to the camcorder's sensor. We're halfway there.

Mounting the Controller
We'll need to be more creative here depending on what's holding the HV20. A monopod/tripod will have to be setup a bit differently from my figrig type. Use your imagination.

There are a few simple rules to follow. Do not bend the cable sharply or the glass may break. It is flexible and sturdy but not if it gets snapped. Rubber bands make good fasteners. Twisties will do too. Plastic tie-downs will work but be careful when you need to cut them off.

Reflecting the Beam
Keeping the covers to the optical cable helps diffuse the beam. Otherwise it is so narrow you have to be aimed almost perfectly for the sensor to make it work. With the covers on, the sensor will pickup the signal up to 30-degrees off angle when stuck point-blank. Or on angle from about three inches away to the front. Having the cable face the camera may be inconvenient like with my figrig. So I routed mine from beside the camcorder.






I routed the cable through the handstrap which hangs it at about a 45 degree angle to the sensor. On the optical head I attach some shiny aluminum foil. Crumpled a bit to form a concave mirror it will scatter the beam toward the sensor. This seems to work quite well even if the head moves.


References:
http://www.dvinfo.net/conf/showthread.php?t=16934

Wednesday, May 2, 2007

The PVC figrig semi-complete

Its noticeably more stable than using one hand or one hand while moving a monopod. But some shock is still transferred by the casual step. If you pay attention to the nose it does move when you go up/down steps or move quickly.



As soon as I can jury rig a laser pointer I'll try various weights to balance it some more.

Mixing 60i and PF24 sources on a 35mm IVTC project

Since the HV20 had this wonderful PF24 mode I've been going made looking for the best way to generate 3:2 pulldown to make progressive source files. Then I realize that my other camera is a 60i only device. So was there a point to converting to 23.976 fps before editing?



If there is, I haven't discovered it yet. The time spent doing pulldown could have been done editing. With mixed sources there will be a deinterlace and frame rate conversion to be done anyway. By mixing it together the final output won't have such a jarring difference between cameras. Is this right? I don't know.