转换时间过慢问题

[UFOyyds]

Description of the bug | 错误描述

一篇简短的PDF to markdown 等待时间有点久，不知什么原因？感谢给予指导

How to reproduce the bug | 如何复现

JSON文件中默认"max_time": 400，请问这个可以适度调节吗？

Operating system | 操作系统

Windows

Python version | Python 版本

3.10

Software version | 软件版本 (magic-pdf --version)

0.6.x

Device mode | 设备模式

cpu

[myhloli]

能上传一些解析过程中的日志看看具体是具体哪个步骤耗时过长吗

[2257396011]

cpu是会比较慢的，你可以试试gpu环境，目前我也用的cpu，模型加载1分钟左右。不算模型加载的话使用ocr处理，一页大概要处理30-45秒，不过因为设备限制我也不确定gpu能快多少。

[myhloli]

cpu是会比较慢的，你可以试试gpu环境，目前我也用的cpu，模型加载1分钟左右。不算模型加载的话使用ocr处理，一页大概要处理30-45秒，不过因为设备限制我也不确定gpu能快多少。

gpu正常情况一页1～2秒左右

[UFOyyds]

能上传一些解析过程中的日志看看具体是具体哪个步骤耗时过长吗

`(detectron2_env) C:\Users\XXXXX>magic-pdf pdf-command --pdf "pdf_path" --inside_model true
2024-08-05 14:10:24.153 | WARNING | magic_pdf.cli.magicpdf:get_model_json:312 - not found json "pdf_path".json existed
2024-08-05 14:10:24.154 | WARNING | magic_pdf.libs.config_reader:get_local_dir:64 - 'temp-output-dir' not found in magic-pdf.json, use '/tmp' as default
2024-08-05 14:10:24.378 | INFO | magic_pdf.libs.pdf_check:detect_invalid_chars:57 - cid_count: 0, text_len: 2, cid_chars_radio: 0.0
2024-08-05 14:10:24.378 | WARNING | magic_pdf.filter.pdf_classify_by_type:classify:334 - pdf is not classified by area and text_len, by_image_area: False, by_text: False, by_avg_words: False, by_img_num: True, by_text_layout: False, by_img_narrow_strips: True, by_invalid_chars: True
INFO:datasets:PyTorch version 2.3.1 available.
2024-08-05 14:10:32.531 | INFO | magic_pdf.model.pdf_extract_kit:init:99 - DocAnalysis init, this may take some times. apply_layout: True, apply_formula: True, apply_ocr: True
2024-08-05 14:10:32.532 | INFO | magic_pdf.model.pdf_extract_kit:init:107 - using device: cpu
2024-08-05 14:10:32.532 | INFO | magic_pdf.model.pdf_extract_kit:init:109 - using models_dir: E:\PDF-Extract-Kit\models
CustomVisionEncoderDecoderModel init
CustomMBartForCausalLM init
CustomMBartDecoder init
[08/05 14:10:55 detectron2]: Rank of current process: 0. World size: 1
[08/05 14:10:56 detectron2]: Environment info:

---

sys.platform win32
Python 3.10.14 | packaged by Anaconda, Inc. | (main, May 6 2024, 19:44:50) [MSC v.1916 64 bit (AMD64)]
numpy 1.26.4
detectron2 0.6 @d:\anaconda\envs\detectron2_env\lib\site-packages\detectron2
detectron2._C not built correctly: DLL load failed while importing _C: 找不到指定的程序。
DETECTRON2_ENV_MODULE
PyTorch 2.3.1+cpu @d:\anaconda\envs\detectron2_env\lib\site-packages\torch
PyTorch debug build False
torch._C._GLIBCXX_USE_CXX11_ABI False
GPU available No: torch.cuda.is_available() == False
Pillow 10.4.0
torchvision 0.18.1+cpu @d:\anaconda\envs\detectron2_env\lib\site-packages\torchvision
fvcore 0.1.5.post20221221
iopath 0.1.9
cv2 4.6.0

---

PyTorch built with:

• C++ Version: 201703
• MSVC 192930154
• Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v3.3.6 (Git Hash 86e6af5974177e513fd3fee58425e1063e7f1361)
• OpenMP 2019
• LAPACK is enabled (usually provided by MKL)
• CPU capability usage: AVX2
• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CXX_COMPILER=C:/actions-runner/_work/pytorch/pytorch/builder/windows/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /Zc:__cplusplus /bigobj /FS /utf-8 -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE /wd4624 /wd4068 /wd4067 /wd4267 /wd4661 /wd4717 /wd4244 /wd4804 /wd4273, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.3.1, USE_CUDA=0, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF,

[08/05 14:10:56 detectron2]: Command line arguments: {'config_file': 'D:\anaconda\envs\detectron2_env\lib\site-packages\magic_pdf\resources\model_config\layoutlmv3\layoutlmv3_base_inference.yaml', 'resume': False, 'eval_only': False, 'num_gpus': 1, 'num_machines': 1, 'machine_rank': 0, 'dist_url': 'tcp://127.0.0.1:57823', 'opts': ['MODEL.WEIGHTS', 'E:\PDF-Extract-Kit\models\Layout/model_final.pth']}
[08/05 14:10:56 detectron2]: Contents of args.config_file=D:\anaconda\envs\detectron2_env\lib\site-packages\magic_pdf\resources\model_config\layoutlmv3\layoutlmv3_base_inference.yaml:
AUG:
DETR: true
CACHE_DIR: ~/cache/huggingface
CUDNN_BENCHMARK: false
DATALOADER:
ASPECT_RATIO_GROUPING: true
FILTER_EMPTY_ANNOTATIONS: false
NUM_WORKERS: 4
REPEAT_THRESHOLD: 0.0
SAMPLER_TRAIN: TrainingSampler
DATASETS:
PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000
PRECOMPUTED_PROPOSAL_TOPK_TRAIN: 2000
PROPOSAL_FILES_TEST: []
PROPOSAL_FILES_TRAIN: []
TEST:

• scihub_train
  TRAIN:
• scihub_train
  GLOBAL:
  HACK: 1.0
  ICDAR_DATA_DIR_TEST: ''
  ICDAR_DATA_DIR_TRAIN: ''
  INPUT:
  CROP:
  ENABLED: true
  SIZE:
  • 384
  • 600
    TYPE: absolute_range
    FORMAT: RGB
    MASK_FORMAT: polygon
    MAX_SIZE_TEST: 1333
    MAX_SIZE_TRAIN: 1333
    MIN_SIZE_TEST: 800
    MIN_SIZE_TRAIN:
• 480
• 512
• 544
• 576
• 608
• 640
• 672
• 704
• 736
• 768
• 800
  MIN_SIZE_TRAIN_SAMPLING: choice
  RANDOM_FLIP: horizontal
  MODEL:
  ANCHOR_GENERATOR:
  ANGLES:
  • • -90
    • 0
    • 90
      ASPECT_RATIOS:
  • • 0.5
    • 1.0
    • 2.0
      NAME: DefaultAnchorGenerator
      OFFSET: 0.0
      SIZES:
  • • 32
  • • 64
  • • 128
  • • 256
  • • 512
      BACKBONE:
      FREEZE_AT: 2
      NAME: build_vit_fpn_backbone
      CONFIG_PATH: ''
      DEVICE: cuda
      FPN:
      FUSE_TYPE: sum
      IN_FEATURES:
  • layer3
  • layer5
  • layer7
  • layer11
    NORM: ''
    OUT_CHANNELS: 256
    IMAGE_ONLY: true
    KEYPOINT_ON: false
    LOAD_PROPOSALS: false
    MASK_ON: true
    META_ARCHITECTURE: VLGeneralizedRCNN
    PANOPTIC_FPN:
    COMBINE:
    ENABLED: true
    INSTANCES_CONFIDENCE_THRESH: 0.5
    OVERLAP_THRESH: 0.5
    STUFF_AREA_LIMIT: 4096
    INSTANCE_LOSS_WEIGHT: 1.0
    PIXEL_MEAN:
• 127.5
• 127.5
• 127.5
  PIXEL_STD:
• 127.5
• 127.5
• 127.5
  PROPOSAL_GENERATOR:
  MIN_SIZE: 0
  NAME: RPN
  RESNETS:
  DEFORM_MODULATED: false
  DEFORM_NUM_GROUPS: 1
  DEFORM_ON_PER_STAGE:
  • false
  • false
  • false
  • false
    DEPTH: 50
    NORM: FrozenBN
    NUM_GROUPS: 1
    OUT_FEATURES:
  • res4
    RES2_OUT_CHANNELS: 256
    RES5_DILATION: 1
    STEM_OUT_CHANNELS: 64
    STRIDE_IN_1X1: true
    WIDTH_PER_GROUP: 64
    RETINANET:
    BBOX_REG_LOSS_TYPE: smooth_l1
    BBOX_REG_WEIGHTS:
  • 1.0
  • 1.0
  • 1.0
  • 1.0
    FOCAL_LOSS_ALPHA: 0.25
    FOCAL_LOSS_GAMMA: 2.0
    IN_FEATURES:
  • p3
  • p4
  • p5
  • p6
  • p7
    IOU_LABELS:
  • 0
  • -1
  • 1
    IOU_THRESHOLDS:
  • 0.4
  • 0.5
    NMS_THRESH_TEST: 0.5
    NORM: ''
    NUM_CLASSES: 10
    NUM_CONVS: 4
    PRIOR_PROB: 0.01
    SCORE_THRESH_TEST: 0.05
    SMOOTH_L1_LOSS_BETA: 0.1
    TOPK_CANDIDATES_TEST: 1000
    ROI_BOX_CASCADE_HEAD:
    BBOX_REG_WEIGHTS:
  • • 10.0
    • 10.0
    • 5.0
    • 5.0
  • • 20.0
    • 20.0
    • 10.0
    • 10.0
  • • 30.0
    • 30.0
    • 15.0
    • 15.0
      IOUS:
  • 0.5
  • 0.6
  • 0.7
    ROI_BOX_HEAD:
    BBOX_REG_LOSS_TYPE: smooth_l1
    BBOX_REG_LOSS_WEIGHT: 1.0
    BBOX_REG_WEIGHTS:
  • 10.0
  • 10.0
  • 5.0
  • 5.0
    CLS_AGNOSTIC_BBOX_REG: true
    CONV_DIM: 256
    FC_DIM: 1024
    NAME: FastRCNNConvFCHead
    NORM: ''
    NUM_CONV: 0
    NUM_FC: 2
    POOLER_RESOLUTION: 7
    POOLER_SAMPLING_RATIO: 0
    POOLER_TYPE: ROIAlignV2
    SMOOTH_L1_BETA: 0.0
    TRAIN_ON_PRED_BOXES: false
    ROI_HEADS:
    BATCH_SIZE_PER_IMAGE: 512
    IN_FEATURES:
  • p2
  • p3
  • p4
  • p5
    IOU_LABELS:
  • 0
  • 1
    IOU_THRESHOLDS:
  • 0.5
    NAME: CascadeROIHeads
    NMS_THRESH_TEST: 0.5
    NUM_CLASSES: 10
    POSITIVE_FRACTION: 0.25
    PROPOSAL_APPEND_GT: true
    SCORE_THRESH_TEST: 0.05
    ROI_KEYPOINT_HEAD:
    CONV_DIMS:
  • 512
  • 512
  • 512
  • 512
  • 512
  • 512
  • 512
  • 512
    LOSS_WEIGHT: 1.0
    MIN_KEYPOINTS_PER_IMAGE: 1
    NAME: KRCNNConvDeconvUpsampleHead
    NORMALIZE_LOSS_BY_VISIBLE_KEYPOINTS: true
    NUM_KEYPOINTS: 17
    POOLER_RESOLUTION: 14
    POOLER_SAMPLING_RATIO: 0
    POOLER_TYPE: ROIAlignV2
    ROI_MASK_HEAD:
    CLS_AGNOSTIC_MASK: false
    CONV_DIM: 256
    NAME: MaskRCNNConvUpsampleHead
    NORM: ''
    NUM_CONV: 4
    POOLER_RESOLUTION: 14
    POOLER_SAMPLING_RATIO: 0
    POOLER_TYPE: ROIAlignV2
    RPN:
    BATCH_SIZE_PER_IMAGE: 256
    BBOX_REG_LOSS_TYPE: smooth_l1
    BBOX_REG_LOSS_WEIGHT: 1.0
    BBOX_REG_WEIGHTS:
  • 1.0
  • 1.0
  • 1.0
  • 1.0
    BOUNDARY_THRESH: -1
    CONV_DIMS:
  • -1
    HEAD_NAME: StandardRPNHead
    IN_FEATURES:
  • p2
  • p3
  • p4
  • p5
  • p6
    IOU_LABELS:
  • 0
  • -1
  • 1
    IOU_THRESHOLDS:
  • 0.3
  • 0.7
    LOSS_WEIGHT: 1.0
    NMS_THRESH: 0.7
    POSITIVE_FRACTION: 0.5
    POST_NMS_TOPK_TEST: 1000
    POST_NMS_TOPK_TRAIN: 2000
    PRE_NMS_TOPK_TEST: 1000
    PRE_NMS_TOPK_TRAIN: 2000
    SMOOTH_L1_BETA: 0.0
    SEM_SEG_HEAD:
    COMMON_STRIDE: 4
    CONVS_DIM: 128
    IGNORE_VALUE: 255
    IN_FEATURES:
  • p2
  • p3
  • p4
  • p5
    LOSS_WEIGHT: 1.0
    NAME: SemSegFPNHead
    NORM: GN
    NUM_CLASSES: 10
    VIT:
    DROP_PATH: 0.1
    IMG_SIZE:
  • 224
  • 224
    NAME: layoutlmv3_base
    OUT_FEATURES:
  • layer3
  • layer5
  • layer7
  • layer11
    POS_TYPE: abs
    WEIGHTS:
    OUTPUT_DIR:
    SCIHUB_DATA_DIR_TRAIN: ~/publaynet/layout_scihub/train
    SEED: 42
    SOLVER:
    AMP:
    ENABLED: true
    BACKBONE_MULTIPLIER: 1.0
    BASE_LR: 0.0002
    BIAS_LR_FACTOR: 1.0
    CHECKPOINT_PERIOD: 2000
    CLIP_GRADIENTS:
    CLIP_TYPE: full_model
    CLIP_VALUE: 1.0
    ENABLED: true
    NORM_TYPE: 2.0
    GAMMA: 0.1
    GRADIENT_ACCUMULATION_STEPS: 1
    IMS_PER_BATCH: 32
    LR_SCHEDULER_NAME: WarmupCosineLR
    MAX_ITER: 20000
    MOMENTUM: 0.9
    NESTEROV: false
    OPTIMIZER: ADAMW
    REFERENCE_WORLD_SIZE: 0
    STEPS:
• 10000
  WARMUP_FACTOR: 0.01
  WARMUP_ITERS: 333
  WARMUP_METHOD: linear
  WEIGHT_DECAY: 0.05
  WEIGHT_DECAY_BIAS: null
  WEIGHT_DECAY_NORM: 0.0
  TEST:
  AUG:
  ENABLED: false
  FLIP: true
  MAX_SIZE: 4000
  MIN_SIZES:
  • 400
  • 500
  • 600
  • 700
  • 800
  • 900
  • 1000
  • 1100
  • 1200
    DETECTIONS_PER_IMAGE: 100
    EVAL_PERIOD: 1000
    EXPECTED_RESULTS: []
    KEYPOINT_OKS_SIGMAS: []
    PRECISE_BN:
    ENABLED: false
    NUM_ITER: 200
    VERSION: 2
    VIS_PERIOD: 0

[08/05 14:10:58 d2.checkpoint.detection_checkpoint]: [DetectionCheckpointer] Loading from E:\PDF-Extract-Kit\models\Layout/model_final.pth ...
[08/05 14:10:58 fvcore.common.checkpoint]: [Checkpointer] Loading from e:\PDF-Extract-Kit\models\Layout/model_final.pth ...
2024-08-05 14:11:00.031 | INFO | magic_pdf.model.pdf_extract_kit:init:132 - DocAnalysis init done!
2024-08-05 14:11:00.032 | INFO | magic_pdf.model.doc_analyze_by_custom_model:custom_model_init:92 - model init cost: 35.65274977684021
2024-08-05 14:11:35.095 | INFO | magic_pdf.model.pdf_extract_kit:call:143 - layout detection cost: 34.73

0: 1888x1344 (no detections), 5070.7ms
Speed: 27.2ms preprocess, 5070.7ms inference, 1.0ms postprocess per image at shape (1, 3, 1888, 1344)
2024-08-05 14:11:42.570 | INFO | magic_pdf.model.pdf_extract_kit:call:173 - formula nums: 0, mfr time: 0.0
2024-08-05 14:11:50.457 | INFO | magic_pdf.model.pdf_extract_kit:call:250 - ocr cost: 7.89
2024-08-05 14:12:24.253 | INFO | magic_pdf.model.pdf_extract_kit:call:143 - layout detection cost: 33.79

0: 1888x1344 (no detections), 4877.3ms
Speed: 26.4ms preprocess, 4877.3ms inference, 0.0ms postprocess per image at shape (1, 3, 1888, 1344)
2024-08-05 14:12:29.161 | INFO | magic_pdf.model.pdf_extract_kit:call:173 - formula nums: 0, mfr time: 0.0
2024-08-05 14:12:30.193 | INFO | magic_pdf.model.pdf_extract_kit:call:250 - ocr cost: 1.03
2024-08-05 14:12:30.194 | INFO | magic_pdf.model.doc_analyze_by_custom_model:doc_analyze:118 - doc analyze cost: 89.82563972473145
2024-08-05 14:12:30.260 | INFO | magic_pdf.pdf_parse_union_core:pdf_parse_union:221 - page_id: 0, last_page_cost_time: 0.0
2024-08-05 14:12:30.480 | INFO | magic_pdf.pdf_parse_union_core:pdf_parse_union:221 - page_id: 1, last_page_cost_time: 0.22
2024-08-05 14:12:30.802 | INFO | magic_pdf.para.para_split_v2:__connect_middle_align_text:682 - 1.0
2024-08-05 14:12:30.848 | INFO | magic_pdf.pipe.UNIPipe:pipe_mk_markdown:48 - uni_pipe mk mm_markdown finished
2024-08-05 14:12:30.865 | INFO | magic_pdf.pipe.UNIPipe:pipe_mk_uni_format:43 - uni_pipe mk content list finished
2024-08-05 14:12:30.867 | INFO | magic_pdf.cli.magicpdf:do_parse:165 - local output dir is '/tmp\magic-pdf\单个文件名\auto', you can found the result in it.`

[myhloli]

@UFOyyds 看了下你的log，一个2页的扫描版pdf，除去模型加载34秒，解析用时89秒，其中约70秒是在layout检测上，这个时间是符合预期的。使用纯cpu解析时，解析效率与cpu的核心数量和主频相关性极高。一般4核cpu单页layout时间30秒左右是正常的，8核cpu可以把这个时间缩短到15秒左右。
cpu解析方案只是为了兼容更多设备，为没有gpu设备的用户提供的一种快速体验方案，不适合在生产环境长时间和大量处理文档使用，这一点在readme中也已经做了说明。
如果需要较快的解析速度，一张大显存显卡是不可缺少的。具体显卡型号选择，可以参考

https://github.com/opendatalab/MinerU/blob/master/README_zh-CN_v2.md

中【安装前必看——软硬件环境支持说明】的表格。

[UFOyyds]

@UFOyyds 看了下你的log，一个2页的扫描版pdf，除去模型加载34秒，解析用时89秒，其中约70秒是在layout检测上，这个时间是符合预期的。使用纯cpu解析时，解析效率与cpu的核心数量和主频相关性极高。一般4核cpu单页layout时间30秒左右是正常的，8核cpu可以把这个时间缩短到15秒左右。 cpu解析方案只是为了兼容更多设备，为没有gpu设备的用户提供的一种快速体验方案，不适合在生产环境长时间和大量处理文档使用，这一点在readme中也已经做了说明。 如果需要较快的解析速度，一张大显存显卡是不可缺少的。具体显卡型号选择，可以参考

https://github.com/opendatalab/MinerU/blob/master/README_zh-CN_v2.md

中【安装前必看——软硬件环境支持说明】的表格。

感谢解答！

[papayalove]

JSON文件中默认"max_time": 400，请问这个可以适度调节吗？

这个是表格识别的超时时间，单位：s

[hlisheng]

我使用GPU解析，最少都需要4分钟，如何减少时间
下面是解析日志
2025-12-24 10:22:56.385 | WARNING | mineru.utils.pdf_page_id:get_end_page_id:8 - end_page_id is out of range, use images length
2025-12-24 10:23:02.096 | INFO | mineru.backend.pipeline.pipeline_analyze:doc_analyze:128 - Batch 1/1: 93 pages/93 pages
2025-12-24 10:23:02.098 | INFO | mineru.backend.pipeline.pipeline_analyze:batch_image_analyze:186 - GPU Memory: 8 GB, Batch Ratio: 4. You can set MINERU_VIRTUAL_VRAM_SIZE environment variable to adjust GPU memory allocation.
Layout Predict: 100%|██████████████████████████████████████████████████████████████████| 93/93 [00:07<00:00, 13.15it/s]
MFD Predict: 100%|█████████████████████████████████████████████████████████████████████| 93/93 [00:11<00:00, 7.95it/s]
MFR Predict: 100%|███████████████████████████████████████████████████████████████████| 578/578 [00:15<00:00, 36.42it/s]
Table-ocr det: 100%|█████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 46.78it/s]
Table-ocr rec ch: 100%|███████████████████████████████████████████████████████████████| 22/22 [00:00<00:00, 285.57it/s]
Table-wireless Predict: 100%|████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 15.56it/s]
Table-wired Predict: 0%| | 0/1 [00:00<?, ?it/s]C:\hls\comda\conda\envs\mineru\Lib\site-packages\mineru\model\table\rec\unet_table\utils_table_line_rec.py:298: FutureWarning: RegionProperties.bbox_area is deprecated starting in version 0.26 and will be removed in version 2.0. Use RegionProperties.area_bbox instead.
if region.bbox_area > H * W * 3 / 4: # 过滤大的单元格
Table-wired Predict: 100%|███████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 6.52it/s]
OCR-det Predict: 100%|█████████████████████████████████████████████████████████████████| 93/93 [00:11<00:00, 8.12it/s]
OCR-rec Predict: 100%|█████████████████████████████████████████████████████████████| 4426/4426 [03:41<00:00, 19.98it/s]
Processing pages: 100%|████████████████████████████████████████████████████████████████| 93/93 [00:04<00:00, 21.45it/s]
2025-12-24 10:27:36.201 | INFO | mineru.cli.common:_process_output:167 - local output dir is ./output\f23edd51-7472-44c4-aff9-b651a0664db5\6770555\auto