View Issue Details

IDProjectCategoryView StatusLast Update
0000147JVT JM H.264/AVC reference softwareencoderpublic2008-12-02 05:16
ReporterShyam Assigned To 
PrioritynormalSeveritymajorReproducibilityalways
Status newResolutionopen 
Product VersionJM 14.2 
Summary0000147: Encoder crashes while encoding Hierarchical B pictures in interlace mode
DescriptionHi,

I tried encoding a HD content with Hierarchical B pictures in interlace mode.
But I got the below error.
 >> get_mem2Dmp() failed.

Its basically not allocating memory as a calloc is returning NULL.
I compiled the code on MC VC++ 2008 express edition on WinXP.

I have attached the config file for the encoder settings I tried.
Please correct me if I am going wrong in setting the config file.

Regards,
Shyam
TagsNo tags attached.

Activities

2008-11-30 13:29

 

encoder_hd.cfg (42,949 bytes)   
# New Input File Format is as follows
# <ParameterName> = <ParameterValue> # Comment
#
# See configfile.h for a list of supported ParameterNames
#
# For bug reporting and known issues see:
# https://ipbt.hhi.de


##########################################################################################
# Files
##########################################################################################
InputFile             = "test_hd.yuv"       # Input sequence
InputHeaderLength     = 0      # If the inputfile has a header, state it's length in byte here
StartFrame            = 0      # Start frame for encoding. (0-N)
FramesToBeEncoded     = 35     # Number of frames to be coded
FrameRate             = 30.0   # Frame Rate per second (0.1-100.0)
SourceWidth           = 1920   # Source frame width
SourceHeight          = 1080  # Source frame height
SourceResize          = 0      # Resize source size for output
OutputWidth           = 176    # Output frame width
OutputHeight          = 144    # Output frame height

TraceFile             = "trace_enc.txt"      # Trace file
ReconFile             = "test_rec.yuv"       # Recontruction YUV file
OutputFile            = "test_hd.264"           # Bitstream
StatsFile             = "stats.dat"          # Coding statistics file

##########################################################################################
# Encoder Control
##########################################################################################
ProfileIDC            = 100 # Profile IDC (66=baseline, 77=main, 88=extended; FREXT Profiles: 100=High, 110=High 10, 122=High 4:2:2, 244=High 4:4:4, 44=CAVLC 4:4:4 Intra)
IntraProfile          = 0   # Activate Intra Profile for FRExt (0: false, 1: true)
                            # (e.g. ProfileIDC=110, IntraProfile=1  =>  High 10 Intra Profile)
LevelIDC              = 40  # Level IDC   (e.g. 20 = level 2.0)

IntraPeriod           = 0   # Period of I-pictures   (0=only first)
IDRPeriod             = 32  # Period of IDR pictures (0=only first)
AdaptiveIntraPeriod   = 0   # Adaptive intra period
AdaptiveIDRPeriod     = 0   # Adaptive IDR period
IntraDelay            = 0   # Intra (IDR) picture delay (i.e. coding structure of PPIPPP... )
EnableIDRGOP          = 0   # Support for IDR closed GOPs (0: disabled, 1: enabled)
EnableOpenGOP         = 0   # Support for open GOPs (0: disabled, 1: enabled)
QPISlice              = 28  # Quant. param for I Slices (0-51)
QPPSlice              = 28  # Quant. param for P Slices (0-51)
FrameSkip             = 3   # Number of frames to be skipped in input (e.g 2 will code every third frame)
ChromaQPOffset        = 0   # Chroma QP offset (-51..51)

DisableSubpelME       = 0   # Disable Subpixel Motion Estimation (0=off/default, 1=on)
SearchRange           = 96  # Max search range

MEDistortionFPel      = 0   # Select error metric for Full-Pel ME    (0: SAD, 1: SSE, 2: Hadamard SAD)
MEDistortionHPel      = 0   # Select error metric for Half-Pel ME    (0: SAD, 1: SSE, 2: Hadamard SAD)
MEDistortionQPel      = 0   # Select error metric for Quarter-Pel ME (0: SAD, 1: SSE, 2: Hadamard SAD)
MDDistortion          = 0   # Select error metric for Mode Decision  (0: SAD, 1: SSE, 2: Hadamard SAD)
SkipDeBlockNonRef     = 0   # Skip Deblocking (regardless of DFParametersFlag) for non-reference frames (0: off, 1: on)
ChromaMCBuffer        = 0   # Calculate Color component interpolated values in advance and store them.sss
                            # Provides a trade-off between memory and computational complexity
                            # (0: disabled/default, 1: enabled)
ChromaMEEnable        = 0   # Take into account Color component information during ME
                            # (0: only first component/default,
                            #  1: All Color components - Integer refinement only
                            #  2: All Color components - All refinements)
ChromaMEWeight        = 1   # Weighting for chroma components. This parameter should have a relationship with color format.

NumberReferenceFrames = 4   # Number of previous frames used for inter motion search (0-16)

PList0References      = 1   # P slice List 0 reference override (0 disable, N <= NumberReferenceFrames)
Log2MaxFNumMinus4     = -1   # Sets log2_max_frame_num_minus4 (-1 : based on FramesToBeEncoded/Auto, >=0 : Log2MaxFNumMinus4)
Log2MaxPOCLsbMinus4   = -1  # Sets log2_max_pic_order_cnt_lsb_minus4 (-1 : Auto, >=0 : Log2MaxPOCLsbMinus4)

GenerateMultiplePPS   = 0   # Transmit multiple parameter sets. Currently parameters basically enable all WP modes (0: diabled, 1: enabled)
ResendPPS             = 0   # Resend PPS (with pic_parameter_set_id 0) for every coded Frame/Field pair (0: disabled, 1: enabled)

MbLineIntraUpdate     = 0   # Error robustness(extra intra macro block updates)(0=off, N: One GOB every N frames are intra coded)
RandomIntraMBRefresh  = 0   # Forced intra MBs per picture
PSliceSkip            = 1   # P-Slice Skip mode consideration  (0=disable, 1=enable)
PSliceSearch16x16     = 1   # P-Slice Inter block search 16x16 (0=disable, 1=enable)
PSliceSearch16x8      = 1   # P-Slice Inter block search 16x8  (0=disable, 1=enable)
PSliceSearch8x16      = 1   # P-Slice Inter block search  8x16 (0=disable, 1=enable)
PSliceSearch8x8       = 1   # P-Slice Inter block search  8x8  (0=disable, 1=enable)
PSliceSearch8x4       = 0   # P-Slice Inter block search  8x4  (0=disable, 1=enable)
PSliceSearch4x8       = 0   # P-Slice Inter block search  4x8  (0=disable, 1=enable)
PSliceSearch4x4       = 0   # P-Slice Inter block search  4x4  (0=disable, 1=enable)
BSliceDirect          = 1   # B-Slice Skip mode consideration  (0=disable, 1=enable)
BSliceSearch16x16     = 1   # B-Slice Inter block search 16x16 (0=disable, 1=enable)
BSliceSearch16x8      = 1   # B-Slice Inter block search 16x8  (0=disable, 1=enable)
BSliceSearch8x16      = 1   # B-Slice Inter block search  8x16 (0=disable, 1=enable)
BSliceSearch8x8       = 1   # B-Slice Inter block search  8x8  (0=disable, 1=enable)
BSliceSearch8x4       = 0   # B-Slice Inter block search  8x4  (0=disable, 1=enable)
BSliceSearch4x8       = 0   # B-Slice Inter block search  4x8  (0=disable, 1=enable)
BSliceSearch4x4       = 0   # B-Slice Inter block search  4x4  (0=disable, 1=enable)

BiPredSearch16x16     = 1   # B-Slice Bi-prediction block search 16x16 (0=disable, 1=enable)
BiPredSearch16x8      = 1   # B-Slice Bi-prediction block search 16x8  (0=disable, 1=enable)
BiPredSearch8x16      = 1   # B-Slice Bi-prediction block search 8x16  (0=disable, 1=enable)
BiPredSearch8x8       = 1   # B-Slice Bi-prediction block search 8x8   (0=disable, 1=enable)

DisableIntra4x4        = 0  # Disable Intra 4x4 modes
DisableIntra16x16      = 0  # Disable Intra 16x16 modes
DisableIntraInInter    = 0  # Disable Intra modes for inter slices
IntraDisableInterOnly  = 0  # Apply Disabling Intra conditions only to Inter Slices (0:disable/default,1: enable)
Intra4x4ParDisable     = 0  # Disable Vertical & Horizontal 4x4
Intra4x4DiagDisable    = 0  # Disable Diagonal 45degree 4x4
Intra4x4DirDisable     = 0  # Disable Other Diagonal 4x4
Intra16x16ParDisable   = 0  # Disable Vertical & Horizontal 16x16
Intra16x16PlaneDisable = 0  # Disable Planar 16x16
ChromaIntraDisable     = 0  # Disable Intra Chroma modes other than DC
EnableIPCM             = 0  # Enable IPCM macroblock mode

DisposableP            = 0  # Enable Disposable P slices in the primary layer (0: disable/default, 1: enable)
DispPQPOffset          = 0  # Quantizer offset for disposable P slices (0: default)

##########################################################################################
# B Slices
##########################################################################################

NumberBFrames          = 3  # Number of B coded frames inserted (0=not used)
QPBSlice               = 30 # Quant. param for B slices (0-51)
BRefPicQPOffset        = -1 # Quantization offset for reference B coded pictures (-51..51)
DirectModeType         = 1  # Direct Mode Type (0:Temporal 1:Spatial)
DirectInferenceFlag    = 1  # Direct Inference Flag (0: Disable 1: Enable)
BList0References       = 1  # B slice List 0 reference override (0 disable, N <= NumberReferenceFrames)
BList1References       = 1  # B slice List 1 reference override (0 disable, N <= NumberReferenceFrames)
                            # 1 List1 reference is usually recommended for normal GOP Structures.
                            # A larger value is usually more appropriate if a more flexible
                            # structure is used (i.e. using HierarchicalCoding)

BReferencePictures    =  0  # Referenced B coded pictures (0=off, 1=B references for secondary layer, 2=B references for primary layer)

HierarchicalCoding      =  3  # B hierarchical coding (0= off, 1= 2 layers, 2= 2 full hierarchy, 3 = explicit)
HierarchyLevelQPEnable  =  1  # Adjust QP based on hierarchy level (in increments of 1). Overrides BRefPicQPOffset behavior.(0=off, 1=on)
ExplicitHierarchyFormat = "b1r4b0e5b2e5" # Explicit Enhancement GOP. Format is {FrameDisplay_orderReferenceQP}.
                          #"b3r4b1r5b5r5b0e6b2e6b4e6b6e6" # Explicit Enhancement GOP. Format is {FrameDisplay_orderReferenceQP}.
                                                          # Valid values for reference type is r:reference, e:non reference.
ReferenceReorder      =  0  # Reorder References according to Poc distance for HierarchicalCoding (0=off, 1=enable)
PocMemoryManagement   =  0  # Memory management based on Poc Distances for HierarchicalCoding (0=off, 1=on)

BiPredMotionEstimation = 1   # Enable Bipredictive based Motion Estimation (0:disabled, 1:enabled)
BiPredMERefinements    = 0   # Bipredictive ME extra refinements (0: single, N: N extra refinements (1 default)
BiPredMESearchRange    = 16  # Bipredictive ME Search range (8 default). Note that range is halved for every extra refinement.
BiPredMESubPel         = 0   # Bipredictive ME Subpixel Consideration (0: disabled, 1: single level, 2: dual level)


##########################################################################################
# SP Frames
##########################################################################################

SPPicturePeriodicity  = 0                  # SP-Picture Periodicity (0=not used)
QPSPSlice             = 36                 # Quant. param of SP-Slices for Prediction Error (0-51)
QPSP2Slice            = 35                 # Quant. param of SP-Slices for Predicted Blocks (0-51)
SI_FRAMES             = 0                  # SI frame encoding flag (0=not used, 1=used)
SP_output             = 0                  # Controls whether coefficients will be output to encode switching SP frames (0=no, 1=yes)
SP_output_name        = "low_quality.dat"  # Filename for SP output coefficients
SP2_FRAMES            = 0                  # switching SP frame encoding flag (0=not used, 1=used)
SP2_input_name1       = "high_quality.dat" # Filename for the first swithed bitstream coefficients
SP2_input_name2       = "low_quality.dat"  # Filename for the second switched bitstream coefficients

##########################################################################################
# Output Control, NALs
##########################################################################################

SymbolMode             =  1  # Symbol mode (Entropy coding method: 0=UVLC, 1=CABAC)
OutFileMode            =  0  # Output file mode, 0:Annex B, 1:RTP
PartitionMode          =  0  # Partition Mode, 0: no DP, 1: 3 Partitions per Slice

##########################################################################################
# CABAC context initialization
##########################################################################################

ContextInitMethod        =  0     # Context init (0: fixed, 1: adaptive)
FixedModelNumber         =  0     # model number for fixed decision for inter slices ( 0, 1, or 2 )

##########################################################################################
# Interlace Handling
#########################################################################################

PicInterlace             =  1     # Picture AFF    (0: frame coding, 1: field coding, 2:adaptive frame/field coding)
MbInterlace              =  0     # Macroblock AFF (0: frame coding, 1: field coding, 2:adaptive frame/field coding, 3: frame MB-only AFF)
IntraBottom              =  0     # Force Intra Bottom at GOP Period

##########################################################################################
# Weighted Prediction
#########################################################################################

WeightedPrediction       =  1     # P picture Weighted Prediction (0=off, 1=explicit mode)
WeightedBiprediction     =  1     # B picture Weighted Prediciton (0=off, 1=explicit mode,  2=implicit mode)
ChromaWeightSupport      =  0     # Enable consideration of weights for Chroma components
UseWeightedReferenceME   =  0     # Use weighted reference for ME (0=off, 1=on)
WPMethod                 =  0     # WP method (0: DC based, 1: LMS based)
EnhancedBWeightSupport   =  0     # Enhanced B Weight support (needs revisit if we wish to merge with WPMethod)
WPMCPrecision            =  0     # Improved Motion Compensation Precision using WP based methods.
                                  # Clones WP references with slightly modified rounding offsets (Requires RDPictureDecision and GenerateMultiplePPS) :
                                  # 0: disabled (default)
                                  # 1: Up to one additional coding pass. Ref0 is 0, ref1 is 0 with a -1 offset
                                  # 2: Up to two additional coding passes. (1) Ref0 is 0, ref1 is 0 with a -1 offset, (1) Ref0 is 0 with a -1 offset, ref1 is 0
WPMCPrecFullRef          =  0     # Increases the number of references in the reference picture lists to account
                                  # for the lost reference slot when reordering is used during a coding pass in WPMCPrecision for reference replication.
                                  # The number of references in non-reordered passes stays unchanged
WPMCPrecBSlice           =  0     # 2: Apply rounding on every B slice. This efectively disables the evaluation of alternative QPs during RDPictureDecision.
                                  # 1: Disable rounding for non-reference B slices. Non-reference B slices are evaluated for alternative QPs during RDPictureDecision.
                                  # 0: Disable rounding for B slices.

##########################################################################################
# Picture based Multi-pass encoding
#########################################################################################

RDPictureDecision        =  1     # Perform RD optimal decision between different coded picture versions.
                                  # If GenerateMultiplePPS is enabled then this will test different WP methods.
                                  # Otherwise it will test QP +-1 (0: disabled, 1: enabled)
RDPictureIntra           =  1     # Perform RD optimal decision also for intra coded pictures (0: disabled (default), 1: enabled).
RDPSliceWeightOnly       =  0     # Only consider Weighted Prediction for P slices in Picture RD decision. (0: disabled, 1: enabled (default))
RDBSliceWeightOnly       =  0     # Only consider Weighted Prediction for B slices in Picture RD decision. (0: disabled (default), 1: enabled )
RDPSliceBTest            =  0     # Perform Slice level RD decision between P and B slices.
                                  # Requires GenerateMultiplePPS = 1 and RDPSliceWeightOnly = 0. (0: disabled (default), 1: enabled )

##########################################################################################
# Deblocking filter parameters
##########################################################################################

DFParametersFlag         = 0      # Configure deblocking filter (0=parameters below ignored, 1=parameters sent)
                                                                                                                        # Note that for pictures with multiple slice types,
                                  # only the type of the first slice will be considered.
DFDisableRefISlice       = 0      # Disable deblocking filter in reference I coded pictures (0=Filter, 1=No Filter).
DFAlphaRefISlice         = 0      # Reference I coded pictures Alpha offset div. 2, {-6, -5, ... 0, +1, .. +6}
DFBetaRefISlice          = 0      # Reference I coded pictures Beta offset div. 2, {-6, -5, ... 0, +1, .. +6}
DFDisableNRefISlice      = 0      # Disable deblocking filter in non reference I coded pictures (0=Filter, 1=No Filter).
DFAlphaNRefISlice        = 0      # Non Reference I coded pictures Alpha offset div. 2, {-6, -5, ... 0, +1, .. +6}
DFBetaNRefISlice         = 0      # Non Reference I coded pictures Beta offset div. 2, {-6, -5, ... 0, +1, .. +6}
DFDisableRefPSlice       = 0      # Disable deblocking filter in reference P coded pictures (0=Filter, 1=No Filter).
DFAlphaRefPSlice         = 0      # Reference P coded pictures Alpha offset div. 2, {-6, -5, ... 0, +1, .. +6}
DFBetaRefPSlice          = 0      # Reference P coded pictures Beta offset div. 2, {-6, -5, ... 0, +1, .. +6}
DFDisableNRefPSlice      = 0      # Disable deblocking filter in non reference P coded pictures (0=Filter, 1=No Filter).
DFAlphaNRefPSlice        = 0      # Non Reference P coded pictures Alpha offset div. 2, {-6, -5, ... 0, +1, .. +6}
DFBetaNRefPSlice         = 0      # Non Reference P coded pictures Beta offset div. 2, {-6, -5, ... 0, +1, .. +6}
DFDisableRefBSlice       = 0      # Disable deblocking filter in reference B coded pictures (0=Filter, 1=No Filter).
DFAlphaRefBSlice         = 0      # Reference B coded pictures Alpha offset div. 2, {-6, -5, ... 0, +1, .. +6}
DFBetaRefBSlice          = 0      # Reference B coded pictures Beta offset div. 2, {-6, -5, ... 0, +1, .. +6}
DFDisableNRefBSlice      = 0      # Disable deblocking filter in non reference B coded pictures (0=Filter, 1=No Filter).
DFAlphaNRefBSlice        = 0      # Non Reference B coded pictures Alpha offset div. 2, {-6, -5, ... 0, +1, .. +6}
DFBetaNRefBSlice         = 0      # Non Reference B coded pictures Beta offset div. 2, {-6, -5, ... 0, +1, .. +6}

##########################################################################################
# Error Resilience / Slices
##########################################################################################

SliceMode             =  0   # Slice mode (0=off 1=fixed #mb in slice 2=fixed #bytes in slice 3=use callback)
SliceArgument         = 50   # Slice argument (Arguments to modes 1 and 2 above)

num_slice_groups_minus1 = 0  # Number of Slice Groups Minus 1, 0 == no FMO, 1 == two slice groups, etc.
slice_group_map_type    = 0  # 0:  Interleave, 1: Dispersed,    2: Foreground with left-over,
                             # 3:  Box-out,    4: Raster Scan   5: Wipe
                             # 6:  Explicit, slice_group_id read from SliceGroupConfigFileName
slice_group_change_direction_flag = 0    # 0: box-out clockwise, raster scan or wipe right,
                                         # 1: box-out counter clockwise, reverse raster scan or wipe left
slice_group_change_rate_minus1    = 85   #
SliceGroupConfigFileName          = "sg0conf.cfg"   # Used for slice_group_map_type 0, 2, 6

UseRedundantPicture   = 0    # 0: not used, 1: enabled
NumRedundantHierarchy = 1    # 0-4
PrimaryGOPLength      = 10   # GOP length for redundant allocation (1-16)
                             # NumberReferenceFrames must be no less than PrimaryGOPLength when redundant slice enabled
NumRefPrimary         = 1    # Actually used number of references for primary slices (1-16)

##########################################################################################
# Search Range Restriction / RD Optimization
##########################################################################################

RestrictSearchRange    =  2  # restriction for (0: blocks and ref, 1: ref, 2: no restrictions)
RDOptimization         =  0  # rd-optimized mode decision
                             # 0: RD-off (Low complexity mode)
                             # 1: RD-on (High complexity mode)
                             # 2: RD-on (Fast high complexity mode - not work in FREX Profiles)
                             # 3: with losses
DistortionSSIM         =  0  # Compute SSIM distortion. (0: disabled/default, 1: enabled)
SSIMOverlapSize        =  8  # Overlap size to calculate SSIM distortion (1: pixel by pixel, 8: no overlap)
DistortionYUVtoRGB     =  0  # Calculate distortion in RGB domain after conversion from YCbCr (0:off, 1:on)
CtxAdptLagrangeMult    =  0  # Context Adaptive Lagrange Multiplier
                             # 0: disabled (default)
                             # 1: enabled (works best when RDOptimization=0)
FastCrIntraDecision    =  1  # Fast Chroma intra mode decision (0:off, 1:on)
DisableThresholding    =  0  # Disable Thresholding of Transform Coefficients (0:off, 1:on)
DisableBSkipRDO        =  0  # Disable B Skip Mode consideration from RDO Mode decision (0:off, 1:on)
SkipIntraInInterSlices =  0  # Skips Intra mode checking in inter slices if certain mode decisions are satisfied (0: off, 1: on)
WeightY                =  1  # Luma weight for RDO
WeightCb               =  1  # Cb weight for RDO
WeightCr               =  1  # Cr weight for RDO

##########################################################################################
# Explicit Lambda Usage
##########################################################################################
UseExplicitLambdaParams  =  0    # Use explicit lambda scaling parameters (0:disabled, 1:enable lambda weight, 2: use explicit lambda value)
UpdateLambdaChromaME     =  0    # Update lambda given Chroma ME consideration
FixedLambdaIslice        =  0.1  # Fixed Lambda value for I slices
FixedLambdaPslice        =  0.1  # Fixed Lambda value for P slices
FixedLambdaBslice        =  0.1  # Fixed Lambda value for B slices
FixedLambdaRefBslice     =  0.1  # Fixed Lambda value for Referenced B slices
FixedLambdaSPslice       =  0.1  # Fixed Lambda value for SP slices
FixedLambdaSIslice       =  0.1  # Fixed Lambda value for SI slices

LambdaWeightIslice       =  0.65 # scaling param for I slices. This will be used as a multiplier i.e. lambda=LambdaWeightISlice * 2^((QP-12)/3)
LambdaWeightPslice       =  0.68 # scaling param for P slices. This will be used as a multiplier i.e. lambda=LambdaWeightPSlice * 2^((QP-12)/3)
LambdaWeightBslice       =  0.68 # scaling param for B slices. This will be used as a multiplier i.e. lambda=LambdaWeightBSlice * 2^((QP-12)/3)
LambdaWeightRefBslice    =  0.68 # scaling param for Referenced B slices. This will be used as a multiplier i.e. lambda=LambdaWeightRefBSlice * 2^((QP-12)/3)
LambdaWeightSPslice      =  0.68 # scaling param for SP slices. This will be used as a multiplier i.e. lambda=LambdaWeightSPSlice * 2^((QP-12)/3)
LambdaWeightSIslice      =  0.65 # scaling param for SI slices. This will be used as a multiplier i.e. lambda=LambdaWeightSISlice * 2^((QP-12)/3)


LossRateA                =  5  # expected packet loss rate of the channel for the first partition, only valid if RDOptimization = 3
LossRateB                =  0  # expected packet loss rate of the channel for the second partition, only valid if RDOptimization = 3
LossRateC                =  0  # expected packet loss rate of the channel for the third partition, only valid if RDOptimization = 3
NumberOfDecoders         = 30  # Numbers of decoders used to simulate the channel, only valid if RDOptimization = 3
RestrictRefFrames        =  0  # Doesnt allow reference to areas that have been intra updated in a later frame.

##########################################################################################
# Additional Stuff
#########################################################################################

UseConstrainedIntraPred  =  0  # If 1, Inter pixels are not used for Intra macroblock prediction.
LastFrameNumber          =  0  # Last frame number that have to be coded (0: no effect)
ChangeQPI                = 16  # QP (I-slices)  for second part of sequence (0-51)
ChangeQPP                = 16  # QP (P-slices)  for second part of sequence (0-51)
ChangeQPB                = 18  # QP (B-slices)  for second part of sequence (0-51)
ChangeQPBSRefOffset      =  2  # QP offset (stored B-slices)  for second part of sequence (-51..51)
ChangeQPStart            =  0  # Frame no. for second part of sequence (0: no second part)

NumberofLeakyBuckets     =  8                      # Number of Leaky Bucket values
LeakyBucketRateFile      =  "leakybucketrate.cfg"  # File from which encoder derives rate values
LeakyBucketParamFile     =  "leakybucketparam.cfg" # File where encoder stores leakybucketparams

NumFramesInELayerSubSeq  = 0  # number of frames in the Enhanced Scalability Layer(0: no Enhanced Layer)

SparePictureOption        =  0   # (0: no spare picture info, 1: spare picture available)
SparePictureDetectionThr  =  6   # Threshold for spare reference pictures detection
SparePicturePercentageThr = 92   # Threshold for the spare macroblock percentage

PicOrderCntType           = 0    # (0: POC mode 0, 1: POC mode 1, 2: POC mode 2)

########################################################################################
#Rate control
########################################################################################

RateControlEnable       = 1     # 0 Disable, 1 Enable
Bitrate                 = 15728640 # Bitrate(bps)
InitialQP               = 0     # Initial Quantization Parameter for the first I frame
                                # InitialQp depends on two values: Bits Per Picture,
                                # and the GOP length
BasicUnit               = 120    # Number of MBs in the basic unit
                                # should be a fraction of the total number
                                # of MBs in a frame
ChannelType             = 0     # type of channel( 1=time varying channel; 0=Constant channel)
RCUpdateMode            = 2     # Rate Control type. Modes supported :
                                # 0 = original JM rate control,
                                # 1 = rate control that is applied to all frames regardless of the slice type,
                                # 2 = original plus intelligent QP selection for I and B slices (including Hierarchical),
                                # 3 = original + hybrid quadratic rate control for I and B slice using bit rate statistics
                                #
RCISliceBitRatio        = 1.0   # target ratio of bits for I-coded pictures compared to P-coded Pictures (for RCUpdateMode=3)
RCBSliceBitRatio0       = 0.5   # target ratio of bits for B-coded pictures compared to P-coded Pictures - temporal level 0 (for RCUpdateMode=3)
RCBSliceBitRatio1       = 0.25  # target ratio of bits for B-coded pictures compared to P-coded Pictures - temporal level 1 (for RCUpdateMode=3)
RCBSliceBitRatio2       = 0.25  # target ratio of bits for B-coded pictures compared to P-coded Pictures - temporal level 2 (for RCUpdateMode=3)
RCBSliceBitRatio3       = 0.25  # target ratio of bits for B-coded pictures compared to P-coded Pictures - temporal level 3 (for RCUpdateMode=3)
RCBSliceBitRatio4       = 0.25  # target ratio of bits for B-coded pictures compared to P-coded Pictures - temporal level 4 (for RCUpdateMode=3)
RCBoverPRatio           = 0.45  # ratio of bit rate usage of a B-coded picture over a P-coded picture for the SAME QP (for RCUpdateMode=3)
RCIoverPRatio           = 3.80  # ratio of bit rate usage of an I-coded picture over a P-coded picture for the SAME QP (for RCUpdateMode=3)
RCMinQPPSlice           =  8    # minimum P Slice QP value for rate control
RCMaxQPPSlice           = 44    # maximum P Slice QP value for rate control
RCMinQPBSlice           =  8    # minimum B Slice QP value for rate control
RCMaxQPBSlice           = 46    # maximum B Slice QP value for rate control
RCMinQPISlice           =  8    # minimum I Slice QP value for rate control
RCMaxQPISlice           = 36    # maximum I Slice QP value for rate control
RCMinQPSPSlice          =  8    # minimum SP Slice QP value for rate control
RCMaxQPSPSlice          = 40    # maximum SP Slice QP value for rate control
RCMinQPSISlice          =  8    # minimum SI Slice QP value for rate control
RCMaxQPSISlice          = 36    # maximum SI Slice QP value for rate control
RCMaxQPChange           =  4    # maximum QP change for frames of the base layer

########################################################################################
#Fast Mode Decision
########################################################################################
EarlySkipEnable         = 0     # Early skip detection (0: Disable 1: Enable)
SelectiveIntraEnable    = 0     # Selective Intra mode decision (0: Disable 1: Enable)

########################################################################################
#FREXT stuff
########################################################################################

YUVFormat               = 1     # YUV format (0=4:0:0, 1=4:2:0, 2=4:2:2, 3=4:4:4)
RGBInput                = 0     # 1=RGB input, 0=GBR or YUV input
SeparateColourPlane     = 0     # 4:4:4 coding: 0=Common mode, 1=Independent mode
SourceBitDepthLuma      = 8     # Source Bit Depth for Luma color component (8...14 bits)
SourceBitDepthChroma    = 8     # Source Bit Depth for Chroma color components (8...14 bits)
SourceBitDepthRescale   = 0     # Rescale bit depth of source for output (0: Disable 1: Enable)
OutputBitDepthLuma      = 8     # Output Bit Depth for Luma color component (8...14 bits)
OutputBitDepthChroma    = 8     # Output Bit Depth for Chroma color components (8...14 bits)

CbQPOffset              = 0     # Chroma QP offset for Cb-part (-51..51)
CrQPOffset              = 0     # Chroma QP offset for Cr-part (-51..51)
Transform8x8Mode        = 1     # (0: only 4x4 transform, 1: allow using 8x8 transform additionally, 2: only 8x8 transform)
ReportFrameStats        = 0     # (0:Disable Frame Statistics 1: Enable)
DisplayEncParams        = 0     # (0:Disable Display of Encoder Params 1: Enable)
Verbose                 = 1     # level of display verboseness (0:short, 1:normal, 2:detailed)

########################################################################################
#Q-Matrix (FREXT)
########################################################################################
QmatrixFile              = "q_matrix.cfg"

ScalingMatrixPresentFlag = 0    # Enable Q_Matrix  (0 Not present, 1 Present in SPS, 2 Present in PPS, 3 Present in both SPS & PPS)
ScalingListPresentFlag0  = 3    # Intra4x4_Luma    (0 Not present, 1 Present in SPS, 2 Present in PPS, 3 Present in both SPS & PPS)
ScalingListPresentFlag1  = 3    # Intra4x4_ChromaU (0 Not present, 1 Present in SPS, 2 Present in PPS, 3 Present in both SPS & PPS)
ScalingListPresentFlag2  = 3    # Intra4x4_chromaV (0 Not present, 1 Present in SPS, 2 Present in PPS, 3 Present in both SPS & PPS)
ScalingListPresentFlag3  = 3    # Inter4x4_Luma    (0 Not present, 1 Present in SPS, 2 Present in PPS, 3 Present in both SPS & PPS)
ScalingListPresentFlag4  = 3    # Inter4x4_ChromaU (0 Not present, 1 Present in SPS, 2 Present in PPS, 3 Present in both SPS & PPS)
ScalingListPresentFlag5  = 3    # Inter4x4_ChromaV (0 Not present, 1 Present in SPS, 2 Present in PPS, 3 Present in both SPS & PPS)
ScalingListPresentFlag6  = 3    # Intra8x8_Luma    (0 Not present, 1 Present in SPS, 2 Present in PPS, 3 Present in both SPS & PPS)
ScalingListPresentFlag7  = 3    # Inter8x8_Luma    (0 Not present, 1 Present in SPS, 2 Present in PPS, 3 Present in both SPS & PPS)
ScalingListPresentFlag8  = 1    # Intra8x8_ChromaU for 4:4:4 (0 Not present, 1 Present in SPS, 2 Present in PPS, 3 Present in both SPS & PPS)
ScalingListPresentFlag9  = 3    # Inter8x8_ChromaU for 4:4:4 (0 Not present, 1 Present in SPS, 2 Present in PPS, 3 Present in both SPS & PPS)
ScalingListPresentFlag10 = 2    # Intra8x8_ChromaV for 4:4:4 (0 Not present, 1 Present in SPS, 2 Present in PPS, 3 Present in both SPS & PPS)
ScalingListPresentFlag11 = 3    # Inter8x8_ChromaV for 4:4:4 (0 Not present, 1 Present in SPS, 2 Present in PPS, 3 Present in both SPS & PPS)

########################################################################################
#Rounding Offset control
########################################################################################

OffsetMatrixPresentFlag  = 1    # Enable Explicit Offset Quantization Matrices  (0: disable 1: enable)
QOffsetMatrixFile        = "q_offset.cfg" # Explicit Quantization Matrices file

AdaptiveRounding         = 1    # Enable Adaptive Rounding based on JVT-N011 (0: disable, 1: enable)
AdaptRoundingFixed       = 1    # Enable Global Adaptive rounding for all qps (0: disable, 1: enable - default/old)
AdaptRndPeriod           = 16   # Period in terms of MBs for updating rounding offsets.
                                # 0 performs update at the picture level. Default is 16. 1 is as in JVT-N011.
AdaptRndChroma           = 1    # Enables coefficient rounding adaptation for chroma

AdaptRndWFactorIRef      = 4    # Adaptive Rounding Weight for I/SI slices in reference pictures /4096
AdaptRndWFactorPRef      = 4    # Adaptive Rounding Weight for P/SP slices in reference pictures /4096
AdaptRndWFactorBRef      = 4    # Adaptive Rounding Weight for B slices in reference pictures /4096
AdaptRndWFactorINRef     = 4    # Adaptive Rounding Weight for I/SI slices in non reference pictures /4096
AdaptRndWFactorPNRef     = 4    # Adaptive Rounding Weight for P/SP slices in non reference pictures /4096
AdaptRndWFactorBNRef     = 4    # Adaptive Rounding Weight for B slices in non reference pictures /4096

AdaptRndCrWFactorIRef    = 4    # Chroma Adaptive Rounding Weight for I/SI slices in reference pictures /4096
AdaptRndCrWFactorPRef    = 4    # Chroma Adaptive Rounding Weight for P/SP slices in reference pictures /4096
AdaptRndCrWFactorBRef    = 4    # Chroma Adaptive Rounding Weight for B slices in reference pictures /4096
AdaptRndCrWFactorINRef   = 4    # Chroma Adaptive Rounding Weight for I/SI slices in non reference pictures /4096
AdaptRndCrWFactorPNRef   = 4    # Chroma Adaptive Rounding Weight for P/SP slices in non reference pictures /4096
AdaptRndCrWFactorBNRef   = 4    # Chroma Adaptive Rounding Weight for B slices in non reference pictures /4096

################################################################
# Rate Distortion Optimized Quantization
################################################################
UseRDOQuant              =  0 # Use Rate Distortion Optimized Quantization (0=disable, 1=enable)
RDOQ_CR                  =  0 # Enable Rate Distortion Optimized Quantization for Chroma components (0=disable, 1=enable)
RDOQ_QP_Num              =  1 # 1-5: Number of QP tested in RDO_Q (I/P/B slice)
RDOQ_CP_Mode             =  1 # copy Mode from first QP tested
RDOQ_CP_MV               =  0 # copy MV from first QP tested
RDOQ_Fast                =  0 # Fast RDOQ decision method for multiple QPs

########################################################################################
#Lossless Coding (FREXT)
########################################################################################

QPPrimeYZeroTransformBypassFlag = 0    # Enable lossless coding when qpprime_y is zero (0 Disabled, 1 Enabled)

########################################################################################
#Fast Motion Estimation Control Parameters
########################################################################################

SearchMode               = 2    # Motion estimation mode
                                # -1 = Full Search
                                #  0 = Fast Full Search (default)
                                #  1 = UMHexagon Search
                                #  2 = Simplified UMHexagon Search
                                #  3 = Enhanced Predictive Zonal Search (EPZS)
UMHexDSR                 = 1    # Use Search Range Prediction. Only for UMHexagonS method
                                # (0:disable, 1:enabled/default)
UMHexScale               = 5    # Use Scale_factor for different image sizes. Only for UMHexagonS method
                                # (0:disable, 3:/default)
                                # Increasing value can speed up Motion Search.

EPZSPattern              = 2    # Select EPZS primary refinement pattern.
                                # (0: small diamond, 1: square, 2: extended diamond/default,
                                # 3: large diamond, 4: SBP Large Diamond,
                                # 5: PMVFAST )
EPZSDualRefinement       = 3    # Enables secondary refinement pattern.
                                # (0:disabled, 1: small diamond, 2: square,
                                # 3: extended diamond/default, 4: large diamond,
                                # 5: SBP Large Diamond, 6: PMVFAST )
EPZSFixedPredictors      = 2    # Enables Window based predictors
                                # (0:disabled, 1: P only, 2: P and B/default)
EPZSTemporal             = 1    # Enables temporal predictors
                                # (0: disabled, 1: enabled/default)
EPZSSpatialMem           = 1    # Enables spatial memory predictors
                                # (0: disabled, 1: enabled/default)
EPZSBlockType            = 1    # Enables block type Predictors
                                # (0: disabled, 1: enabled/default)
EPZSMinThresScale        = 0    # Scaler for EPZS minimum threshold (0 default).
                                # Increasing value can speed up encoding.
EPZSMedThresScale        = 1    # Scaler for EPZS median threshold (1 default).
                                # Increasing value can speed up encoding.
EPZSMaxThresScale        = 2    # Scaler for EPZS maximum threshold (1 default).
                                # Increasing value can speed up encoding.
EPZSSubPelME             = 1    # EPZS Subpel ME consideration
EPZSSubPelMEBiPred       = 1    # EPZS Subpel ME consideration for BiPred partitions
EPZSSubPelThresScale     = 2    # EPZS Subpel ME Threshold scaler
EPZSSubPelGrid           = 1    # Perform EPZS using a subpixel grid

########################################################################################
# SEI Parameters
########################################################################################

ToneMappingSEIPresentFlag = 0    # Enable Tone mapping SEI  (0 Not present, 1 Present)
ToneMappingFile           = "ToneMapping.cfg"

GenerateSEIMessage        = 0                    # Generate an SEI Text Message
SEIMessageText            = "H.264/AVC Encoder"  # Text SEI Message

########################################################################################
# VUI Parameters
########################################################################################
# the variables below do not affect encoding and decoding
# (many are dummy variables but others can be useful when supported by the decoder)

EnableVUISupport                                = 0      # Enable VUI Parameters

# display parameters
VUI_aspect_ratio_info_present_flag              = 0
VUI_aspect_ratio_idc                            = 1
VUI_sar_width                                   = 0
VUI_sar_height                                  = 0
VUI_overscan_info_present_flag                  = 0
VUI_overscan_appropriate_flag                   = 0
VUI_video_signal_type_present_flag              = 0
VUI_video_format                                = 5
VUI_video_full_range_flag                       = 0
VUI_colour_description_present_flag             = 0
VUI_colour_primaries                            = 2
VUI_transfer_characteristics                    = 2
VUI_matrix_coefficients                         = 2
VUI_chroma_location_info_present_flag           = 0
VUI_chroma_sample_loc_type_top_field            = 0
VUI_chroma_sample_loc_type_bottom_field         = 0
VUI_timing_info_present_flag                    = 0
VUI_num_units_in_tick                           = 1000
VUI_time_scale                                  = 60000
VUI_fixed_frame_rate_flag                       = 0

# nal hrd parameters
VUI_nal_hrd_parameters_present_flag             = 0
VUI_nal_cpb_cnt_minus1                          = 0
VUI_nal_bit_rate_scale                          = 0
VUI_nal_cpb_size_scale                          = 0
VUI_nal_bit_rate_value_minus1                   = 0
VUI_nal_cpb_size_value_minus1                   = 0
VUI_nal_vbr_cbr_flag                            = 0
VUI_nal_initial_cpb_removal_delay_length_minus1 = 23
VUI_nal_cpb_removal_delay_length_minus1         = 23
VUI_nal_dpb_output_delay_length_minus1          = 23
VUI_nal_time_offset_length                      = 24

# vlc hrd parameters
VUI_vcl_hrd_parameters_present_flag             = 0
VUI_vcl_cpb_cnt_minus1                          = 0
VUI_vcl_bit_rate_scale                          = 0
VUI_vcl_cpb_size_scale                          = 0
VUI_vcl_bit_rate_value_minus1                   = 0
VUI_vcl_cpb_size_value_minus1                   = 0
VUI_vcl_vbr_cbr_flag                            = 0
VUI_vcl_initial_cpb_removal_delay_length_minus1 = 23
VUI_vcl_cpb_removal_delay_length_minus1         = 23
VUI_vcl_dpb_output_delay_length_minus1          = 23
VUI_vcl_time_offset_length                      = 24
VUI_low_delay_hrd_flag                          = 0

# other params (i.e. bitsream restrictions)
VUI_pic_struct_present_flag                     = 0
VUI_bitstream_restriction_flag                  = 0
VUI_motion_vectors_over_pic_boundaries_flag     = 1
VUI_max_bytes_per_pic_denom                     = 0
VUI_max_bits_per_mb_denom                       = 0
VUI_log2_max_mv_length_vertical                 = 16
VUI_log2_max_mv_length_horizontal               = 16
VUI_num_reorder_frames                          = 16
VUI_max_dec_frame_buffering                     = 16

encoder_hd.cfg (42,949 bytes)   

Alexis Michael Tourapis

2008-12-02 05:16

developer   ~0000253

Although there are other problems as well, your system is basically running out of memory given also the likelihood of your system using a 32 bit OS (I consider it unlikely that you are using a 64 bit version of Win XP). You can resolve some of the problems by recompiling your code after changing #define IMGTYPE to 0, but also by not using RDPictureDecision which requires even more memory. I would also suggest setting SkipGlobalStats to 1 to avoid a known issue in the encoder (especially the version you are running), but to also disable bipredsearch modes. Note that these also do not work that well with any of the UMHEX algorithms, plus (even though I am heavily biased) EPZS is a better scheme.

Issue History

Date Modified Username Field Change
2008-11-30 13:29 Shyam New Issue
2008-11-30 13:29 Shyam File Added: encoder_hd.cfg
2008-12-02 05:16 Alexis Michael Tourapis Note Added: 0000253