Apple has won a patent (number 7,856,059) from the US Patent & Trademark Office involving determining the number of unidirectional and bidirectional motion compensated frames to be encoded for a video sequence and detecting scene cuts in the video sequence.
Methods for processing a set of successive video frames in two passes to determine the number of bidirectional (B) and unidirectional (P) motion compensated frames to be encoded in a video coding system. During the first pass, motion vectors and motion costs are computed for each frame and a derived cost value is computed based on the motion cost of at least one frame.
The derived cost value is used to determine the number (N.sub.B) of B-frames to be encoded in the set of successive frames. In the second pass, the set of successive frames are encoded where N.sub.B frames are encoded as B-frames and some or all motion vectors computed in the first pass are re-used in the second pass. A scene cut detection method is also provided where an impulse-like increase in a ratio of motion costs is monitored. The inventors are Barin Geoffry Haskell, Adriana Dumitras and Atul Puri.
Here’s Apple’s summary of the invention: “The present invention provides methods for encoding frames of a video sequence where the sequence is processed in two passes. During the first pass, motion vectors are computed for pixelblocks of each frame in a set of successive frames with reference to other specific frame or frames. In some embodiments, motion compensation errors (MCEs) for pixelblocks of each frame are also computed.
“A motion cost value for each frame is then determined, the motion cost value being related to the number of bits required to encode the motion vectors and/or the value of the MCEs of the pixelblocks of the frame. A derived cost value is then computed based on the motion cost value of at least one frame (e.g., the derived cost value can be the motion cost value of one frame, the average motion cost value of two or more frames, or the ratio of the motion cost value of a first frame and the motion cost value of a second frame).
“In addition, in the first pass, the derived cost value is used to determine the number (N.sub.B) of B-frames to be encoded in the set of successive frames. The number (N.sub.B) of B-frames to be encoded increases as long as the derived cost value is below a predetermined threshold value. In the second pass, frame N.sub.B+1 in the set of successive frames is encoded as a P-frame and frames 1 through N.sub.B are encoded as B-frames where some or all motion vectors computed in the first pass are re-used in the encoding process of the second pass.
“In some embodiments, during the first pass, motion vectors are computed for each pixelblock and for each frame in a set of successive frames with reference to an immediately preceding frame. In these embodiments, some of the motion vectors computed in the first pass are re-used in the encoding process of the second pass. In further embodiments, during the first pass, motion vectors are computed for each frame in a set of successive frames with reference to a same preceding frame (frame 0 in the set of successive frames). In these embodiments, all of the motion vectors computed in the first pass are re-used in the encoding process of the second pass.
“In some embodiments, the derived cost value is the average motion cost of a series of successive frames. In other embodiments, the derived cost value is the motion cost of a single frame. In further embodiments, the derived cost value is a ratio between the motion cost of a first frame and the motion cost of a second frame that immediately precedes the first frame. In these further embodiments, the ratio of motion costs is used to detect an impulse-like increase in the motion costs between two successive frames which typically indicates a scene cut between the two successive frames. As such, these further embodiments provide a scene cut detection method that is used in conjunction with the two pass encoding method of the present invention. In additional embodiments, the scene cut detection method is used independent from the two pass encoding method.”
— Dennis Sellers
