Twixtor intelligently slows down, or speeds up your image sequences.
Twixtor enables you to "time warp" your image sequences with visually stunning results. In order to achieve its unparalleled image quality, Twixtor synthesizes unique new frames by warping and interpolating frames of the original sequence... employing RE:Vision's proprietary tracking technology that calculates motion for each individual pixel.
New in Twixtor 5.0:
New in Twixtor 4.5:
Twixtor Features:
Before You Start, EXTREMELY IMPORTANT.
Common problems discussed here!
Twixtor 5.0
Twixtor comes in two modes: Desktop and GUI+Batch. The Desktop mode only has 6 values and only ask you for one reel. It can work extremely fast as it does not have any overhead in that mode. The parameters are described below.
DESKTOP MODE:
MODULE MODE: GUI+ Batch
When loading Twixtor you are asked to provide 3 reels, 1) The Color Source 2) The ALT motion 3) the FG-BG Matte. If you don't use either ALT Motion or FG_BG segmentation matte, then just select the same reels.Here is a quick parameter reference (complete with links to their descriptions).
Twixtor Control Page 1:
![]()
Display Mode : There are 4 output types supported. The Output Time mode is the normal mode. The Source Time is used to mark Segments (say to twixtor where there is a cut). The Background and Foreground only modes are used to split your workflow in multiple passes OR simply to see what's happening if you use a Matte as Foreground-Background segmentation.
Color Source : There are 3 types of sources to be aware of when using Twixtor. The Color Source specifies the clip that will be warped. It is the first clip you select (Front). You can also select a different second clip, Back. And a third clip Matte. If you don't use AlternateMotion source of Foreground/Background separation, then simply select the same clip 3 times.
The Front clip is the source for the color information used to create new frames. If another clip is specified as the Back Clip, and you press AlternateMotion, this becomes the source for tracking purposes, limiting the role of Color Source to be the RGB values that are warped.
This second clip, the Alternate Motion Source, allows you for example to enhance the colors used in the tracking process, but then use the colors from the original sequence for the `Twixtored` sequence." Or, the reverse, you've heavily filtered the original (which you've used as the color source for Twixtor), but you think you might get better tracking results by using the original footage as the imagery to be tracked. This also allows you to warp a matte image using the RGB clip as tracking source.
So, we've provided this feature to set an Alternate Motion Source: If there is no second clip input to the node, then then the Color Source is used for both tracking and the color of the Twixtor sequence. Of course, Twixtor assumes that both the Alt Motion Source and Color Source clips line up exactly in time.
We also provide a third clip option, "motion matte". This allows you to perform "Motion Masking" which we explain later on, but is a way to fix posssible problems. Note if you need a mask, but not an alternate tracking source, then select the same clip twice (for front and back). Also, we will repeat again, the Motion Mask specifies a region and has to be full-on or full-black, greyscale values are meant to indicate the anti-aliased edge of a motion matte.
Notes for Interlaced Video Usage Only: Handling Fields
Okay, you ask, what happens when there are fields in the input (interlaced video)?
1) Let Twixtor create frames from fields using field interpolation. This is accomplished by setting "Output Fields" in the Twixtor settings to "None" (Full Frame). When passing Twixtor footage with fields, each field in an input source becomes a frame to be interpolated. So 30i footage really looks like 60p internally to Twixtor. A standard algorithm is applied to make each frame (we interpolate the fields to create a whole frame). Input frame 1.0 is made from the first field (in time, which may be upper or lower depending on your source footage) of the first frame. Frame 1.5 is made from the second field in time of the first frame and so on.
2) You also have the option to strip a field prior to Twixtor and like that pass "full frames". This might be useful for instance in cases where the camera is sort of locked and there are fine horizontal lines in the picture.
For interlaced output, frames at 1/2 frame times are calculated (and warped and blended as you specify) and then interlaced. So be aware that as a result the amount of motion blur will be half of what it is in frame mode and to render with Fields Out will take near two times as long as there is more sub-frames to calculate.
Frame Interpolation : A menu option with 3 choices.
- Nearest: warps the nearest calculated input frame by the right amount to make the current output frame.
- Blend: Performs bi-directional warping and then blends the resulting two warped images.. The more you stretch a clip, the more this choice is recommended.
- Motion Weighted Blend: When a new frame is created from existing frames it is often the case that some softening is introduced. On large motion, this softening can more or less strong (depending on how far away in time the new frame is located from existing frames in the original sequence. This can result in a pulsing of what appears to be the Twixtored sequence coming in and out of focus. The Motion Weighted Blend option attempts to reduce this pulsing.
Motion Vector Quality : A menu option with 5 choices. With "High Quality" set:
For demonstration purposes, let's assume that we are scaling a clip's duration by a factor of 3. As such, output frame 25 would conceptually need input frame 8.33333. How is frame 8.333 calculated?
- No Motion Vectors: no motion vectors are calculated
- Sloppy: motion vectors are calculated at 1/4 resolution, then interpolated. Furthermore, a faster but less accurate method is used.
- Medium: motion vectors are calculated at 1/4 resolution, then interpolated.
- High: motion vectors are calculated at 1/2 resolution, then interpolated.
- Best: motion vectors are calculated at full resolution, then interpolated.
- For "Draft " mode we calculate motion vectors at half the resolution as specified above.
Note that if fewer frames are being generated for the output sequence, that is, the input sequence is being speed up (total time duration is shrunk), then motion blur may then be applied (see Motion Blur option below).
- Case 1: Interpolation: Nearest with Motion Vector Quality: No Motion Vectors. In this case, no warping can occur (there are no motion vectors!), so the closest input frame in time (input frame 8) is returned. This is equivalent to Time Stretching or Time Remapping in After Effects without frame blending.
- Case 2: Interpolation: Blend or Motion Weighted Blend with Motion Vector Quality: No Motion Vectors . In this case, no warping can occur (there are no motion vectors!). However, we've asked the module to blend frames, so we'll get a return frame that is a dissolve of frame 8 (at 67%) and frame 9 (33%). This is similar to After Effects with frame blending turned on.
- Case 3: Interpolation: Nearest with Motion Vector Quality set to anything other than No Motion Vectors . The motion is calculated from input frame 8 to input frame 9. Frame 8 is then warped using the motion calculated (scaled by 1/3) and returned.
- Case 4: Interpolation: Blend or Motion Weighted Blend with Motion Vector Quality set to anything other than No Motion Vectors. The motion is calculated from input frame 8 to input frame 9. Frame 8 is then warped using the motion calculated (scaled by 1/3) and returned. Frame 9 is warped using the motion calculated from frame 9 to frame 8 (scaled by 2/3). Finally, these two warped frames are blended as in Case 2.
The animatable MarkSegment option is designed to provide an override mechanism for the Motion Vectors calculation based on source material knowledge When you retime the footage you may notice that Twixtor inappropriately morphs across a hard cut edit in the source material. Or there may be a section of the footage, such as a dissolve, where you know frame blending
without any motion vectors would produce better results. Because of these situations we have provided the following setting.This setting allows you to specify information you know about the source material. When animating this option, make sure to do it when Display is set to Source Time otherwise it will be invisible. Since menus are not animatable in IFFFS, you will see below a dummy integer slider which will essentially keep your menu animation as a constant step animation. You set the menu and we set the animation in the Menu Anim slider. To Reset All in the Channel Editor, then use the Menu Anim slider, otherwise that slider has no input controls.
You will see a menu that provides these options.
Cut A [1]
Cut B [2]
Cut C [3]Twixtor will not interpolate between two segments marked with different Cut settings. For example, if one frame is marked as part of Cut A and the next frame is marked as part of Cut B, then Twixtor will not interpolate between the two when time remapping. In fact, let's say frame 10 is part of Cut A and frame 11 is part of Cut B. Let's say Twixtor is asked to slow the footage down: if Twixtor is asked to make frame 10.4, then instead of taking 60% of frame 10 and 40% of frame 11, Twixtor will simply return frame 10. If Twixtor is asked to make frame 10.6, then frame 11 is simply returned. In fact, when two adjacent frames are marked as part of different cuts, Twixtor simply returns the nearest frame in the source instead of blending between the two frames. However, when two source frames to be interpolated are marked with the same Cut setting, then the Motion Vectors and Frame Interp settings will be used to interpolate between the two frames to create the output frame.
No MV, Use Nearest [4]
No MV, Frame Blend [5]When the first source frame (of a pair of frames to be interpolated) is marked with either of these two settings, then any output Twixtored frame made will have the motion vector calculation turned off and the nearest frame (or frame blended version, as selected) will be returned. Think of this setting as overriding the Motion Vectors and Frame Interp settings... but unlike the other two settings which are specified in the output Twixtored timing, this is specified in the source material's timing. These two settings are useful for a dissolve, wipe or other transition in the source footage will "obviously" produce unwanted artifacts when interpolated using Twixtor's motion vector estimation. In essence this setting lets you turn of motion vectors in the source timing without having to find out where the Twixtored frames end up in the output timing.
Okay, now here's the meat of the plugin. How do I retime my footage?
Stretching Method pop-up menu has 2 options for the retiming method, as follows:
EXERCISE: If you are reading this manual for the first time, you should try taking a movie you have lying around and try to expand or contract it's duration using a constant speed value like 1.5 or 2.
- Keyframed Stretch
If Keyframed Stretch is chosen, then Twixtor knows you'll be keyframing your time resequencing. Twixtor looks at the value of the Frame slider to figure out what frame from the input sequence to compute, even if it's a fractional value. Note: the Frame parameter is used when you want to dynamically remap time.This section describes keyframing the remapping of time using frames.. Again, let's assume that the input frame rate is the same as the output frame rate.
If you wish to keyframe the timing using direct frame numbers specification, then make sure Stretch Method is set to Key-Framed Stretch. This parameter will be referred to as Frame for the rest of this document. The value of the Frame parameter tells Twixtor which input frame should be returned for the current time (fractional values are interpolated as described in the previous section).
To use the Frame parameter for retiming:Immediately after applying Twixtor to a clip, it is advisable that you:
First, make the Frame parameter animatable by turning AutoKey ON
Second, you are advised to make keyframes for the Frame parameter as follows:
For example: Let's say you have a sequence of 300 frames.
Set frame 1 at time 1 and frame 300 at time 300. This will give you an identity mapping (rame 47 will be returned at time 47, etc.). You can then safely animate the frame parameter curve.
- Speed
The other menu option lets you animate the speed of the input clip. To obtain the speed of the clip, Twixtor gets the value from the Speed slider. A value of 0.5 tells Twixtor to play the clip 1/2 as fast as the original, a value of 3 tells Twixtor to play the original clip at 3 times its normal rate (3 sources frames = 1 destination frame).If Stretch Method is set to Speed then the Frame parameter is ignored and hidden. Conversely, the Speed slider is ignored when key-framing destination frames.TIP: If you are doing a complex dynamic remapping of the video action, you can quickly set your key-frames by setting Motion Vector Quality to None and Frame Interpolation set to Blend. When done setting keyframes, you can go back and set the Motion Vector Quality to the desired quality.
Specifying fieldsWhat are fields?
1) Simply put, an interlaced video frame consists of 2 fields. One field consists of all even lines of a frame (starting at the top of the frame), and the other field consists of odd lines. A video device (monitor, TV, etc) has a beam that traces, or draws, one field, then goes back and draws the other. As such, one field is displayed first, then the other is displayed half of a frame time later.2) So interlaced video has two fields per frame. One field is displayed first in time, and then the other. If the top scan line is part of the field that is displayed first (that is, the field of odd scan lines is displayed first) then we say that the video is "Upper Field First." If the second-to-top scan line is part of the field that is displayed first (that is, the field of even scan lines is displayed first) then we say that the is "Lower Field First ".
The rule of thumb when specifying "field first" for your footage: if you choose lower or upper field first and the plugin produces material that is overly jittery, or jumps back and forth, then you probably chose the incorrect option.
Note that you cannot assume all of your source material will have the same field first-ness. Although historically PAL was Upper Field and NTSC Lower Field, this is not to be assumed. For example, PAL DV material is typically lower-field first.
Warping Method:
We provide 3 ways to warp your results.
1.Inverse: Inbetweened warped images are calculated using an inverse mapping method. This setting is much faster than the other two settings. It is often better to use this setting when portions of objects are not moving off the edge of the image (or onto the image from offscreen)
2.Inverse W/ Smart Blend: This setting takes more time than the Inverse setting, but is faster than the Forward setting. Smart Blend gives better results on a pan or zoom or other footage where objects (or parts of objects) are entering or exiting the frame. When this option is off on such shots, you might get pixels that smear at the edge of the frame. When turned on, this option can produce much better blending when objects are moving off screen. This option will increase the calculation time and is only necessary on certain shots (like shots with pans and zooms with greater than 5% ), so thats why its an option.
3.Forward: Inbetweened warped images are calculated using a forward mapping method. This method is slower than Inverse, but can produce much less ghosting. The forward method includes the Smart Blend method. It is sometimes advised to use this mode if you are using extreme slow motion (more than 3x slow down) and inbetween frames do not produce desirable results (ghosting or "sliding" of imagery can occur if you use the Inverse method).
I'm slowing footage down, do I really need to recalulate the motion vectors every frame?If you have plenty of memory, check the Cache? checkbox. This will cache the last motion vector calculation. This will only be helpful when slowing down footage. With Cache toggled On Twixtor can be much faster when run directly within the Module itself but will consume more memory.The answer is no!
The tracking is not working so well on my source footage, what can I do?
Tracking problems show up as motion not being properly tracked: either objects in the output sequence "gloop" together, or objects don't move enough causing what seems to be simple cross dissolves of the input images. (for what constitues problems for our tracker, see the "PROBLEMS IN INPUT SEQUENCES" section below).
Motion Sensitivity:
Okay, you might say "I don't have time to experiment with image processing my footage to specify an alternate motion source," or worse, "I try to color correct my footage and it does not help the tracking!" If objects still "gloop" together when they shouldn't, then you can try manipulating the Motion Sensitivity parameter. Motion Sensitivity limits how much pixels can move. A value of zero assures that pixels can't move very much, and a value of 100 allows pixels to move as much as the motion estimator can calculate. The default value might sometimes be a bit too "ambitious" for some material... by reducing the sensitivity you might be able to create a more interesting result (giving a more traditional frame blended look) where there is large motion rather then some sort of unappropriate swirl that Twixtor might introduce.Image Prep: A menu option with 2 choices
1.None
2.Contrast/Edge EnhanceThe default image preparation is to do nothing special with the source material that is used for tracking. If
your material is dark or has poorly defined edges then Twixtor might perform better if you choose
Contrast/Edge Enhance. Note that the Contrast/Edge Enhance setting may perform more poorly than using the None
setting on material that already has reasonable contrast or has many thin edges close together. This option is provided as it's the most common image correction performed to help Twixtor. You can create your own tracking source by using the ALT Motion Source.So, how can you enhance tracking using an Alt Motion Source? The following set of pictures seeks to explain:
![]()
Figure 1
![]()
Figure 2
![]()
Figure 3
![]()
Figure 4
Using an alternate motion source:
- Figure 1: Original low contrast sequence
- Figure 2: Inbetween frame, note that the rear arm is not tracked well because the arm is similar in intensity to the background (of course our eyes see the difference!)
- Figure 3: Image enhanced, but used only for the tracking portion of Twixtor by specifying it as the Alt Motion Source.
Figure 4: The original colors in Figure 1 are used, but the tracking is performed on a sequence represented by Figure 3. Note that the rear arm is now tracked nicely!
Twixtor Control Page 2:
![]()
Motion Blur Compensation:
When speeding up a clip, you are essentially telling Twixtor that there is "more action" happening per frame. In order to simulate what happens in a real camera, and to reduce flickering, Twixtor applies motion blur when speeding up footage in order to simulate what happens when more motion takes place in the same amount of (virtual) shutter opening of a camera. You can dial in how much motion blur is added with the Motion Blur Compensation slider. New in Twixtor 4: Twixtor also compensates for motion blur by removing motion blur when slowing down footage. Note that the removal of motion blur may be more or less effective depending on how much blur there is in the original sequence. The default setting for Motion Blur Compensation is 0.0 (that is, by default there is not motion blur compensation). A value of 0.5 is a good place to start if you wish Twixtor to automatically compensate for motion blur (this assumes a shutter angle of 180 degrees).
Note that the application of motion blur in Twixtor only occurs when footage is sped up or slowed down, to compensate for different virtual shutter speeds. This is different from our ReelSmart Motion Blur product, which adds motion blur based on the motion calculated in a sequence without speeding up or slowing down the footage.
- Note that there is a different Motion Blur tool, our ReelSmart Motion Blur product, which adds motion blur based on the motion calculated in a sequence without speeding up or slowing down the footage. It is sometimes advised to simply add Motion Blur as a post-process rather then within Twixtor. If you are field rendering on output the resulting output fields will have 1/2 as much motion blur (upon speedup) or twice as sharp (upon slowdown) as when viewing the frames interactively because the interlaced material will have twice as many time samples.
Better results through Foreground/background separation, or I have a matte for a foreground object and would like to improve Twixtor's tracking by using this matte" (or, Improved tracking Part Two)
Twixtor allows you to specify a mask so that you can track a foreground object separate from the background.This can help immensely with "gloopiness" in the 'Twixted' sequence. Except for specifying nice anti-aliased edges (and perhaps incorporating some motion blur) of the foreground, make sure that mask is full-on white. Be careful for example of not having tiny holes in the supplied mask. Remember this is not a compositing mask but a way to guide the estimator and tell it that this is a more or less related motion region. Therefore occasionally you might need to make the shape approximatively the shape of an object that disappears behind another one rather then a typical composting roto.
Please note that while using a foreground mask can dramatically improve results, Twixtor needs to calculate one or two additional motion estimations per frame (as we now generate motion for the foreground, and motion for the background), so it can take more than twice as long.
In order to describe the Inv Mask Shrink option, pictures are worth a thousand words! So let's say we have the following image, and matte for the foreground object:
- FG - Top Layer Motion Sensitivity: It is helpful to specify a foreground motion sensitivity that is separate from the main layer motion sensitivity (for example something moving over a stable background), so it is provided here (see above for the Motion Sensitivity description that is used for the background layer).
- FG- Top Layer Inv Mask Shrink:This allows you to specify (in pixels) the amount the inverse of the foreground matte should be shrunk when tracking and creating the main (background) layer. If you see a dark line or similar artifacts where you know the edges of your mask are, just raise this value.
![]()
Source Image
![]()
Matte for Foreground Object
Using the matte for the foreground and the inverse of the matte for the background we get the following two pictures:Note that while the matte specifies the foreground quite nicely, some of the pixels of the foreground spill into the background! By specifying how many pixels to shrink the inverse of the foreground matte (the FG Layer Inv Matte Shrink value), we can reduce or eliminate the spillage. By specifying a value of 1.5 we get the following picture:
![]()
Matte used to separate out foreground.
![]()
Inverse of matte to specify background.
![]() Shrinking the inverse of the matte to get a better background by specifying a Top Layer Inv Matte Shrink value of 1.5. |
Note you might ask, "won't that create a gap in my final output?" The answer is no! Through clever filling and compositing, we track the background separately from the foreground, and fill in missing information in the background as best we can!Using a foreground layer and memory usage.. Twixtor will use approximately 6 image buffers' worth of memory to run the foreground layer option. (image buffer's worth of memory is approximately 4*width*height bytes, or if in 16-bit, then 8*width*height bytes), in addition to whether or not Plenty Memory? is checked.
ReelSmart and Twixtor are trademarks of RE:Vision Effects, Inc.
sparks, inferno, flame, flint, fire and smoke are registered
trademarks of discreet.
Copyright 2000-2010 RE:Vision Effects, Inc.