1 Release published by 1 person

• v2.3.0 NVIDIA Neural Modules 2.3.0

  published May 8, 2025

32 Pull requests merged by 15 people

• Remove adapter_path from base AutoResume and refactor PEFT checkpoint handling

  #12565 merged May 11, 2025

• Add fp8_param argument back to mixed precision plugin for backward compatibility

  #13522 merged May 10, 2025

• add memory profile interface to perf scripts

  #13413 merged May 10, 2025

• Refactor Distillation PP support to use new MCore API

  #13065 merged May 10, 2025

• Add Llama4 GHA

  #13442 merged May 9, 2025

• Add Resume_path to llama_nemotron models

  #13515 merged May 9, 2025

• Improve error message when HF checkpoint cannot be loaded

  #13513 merged May 9, 2025

• chore(🤖): Bump NVIDIA/Megatron-LM to bcbede5... (2025-05-09)

  #13510 merged May 9, 2025

• ci: Add more files to filter

  #13517 merged May 9, 2025

• add extra hyena tests

  #13097 merged May 9, 2025

• Autodetect model_type and dtype for deployment using TRT-LLM backend

  #13209 merged May 9, 2025

• Cherry pick Adding more export tests (13410) into r2.3.0

  #13450 merged May 9, 2025

• Cherry pick Adding additional unit tests for the deploy module (13411) into r2.3.0

  #13449 merged May 9, 2025

• Cherry pick new perf configs (13110) into r2.3.0

  #13431 merged May 9, 2025

• Cherry pick Handle boolean args for performance scripts and log received config (13291) into r2.3.0

  #13416 merged May 9, 2025

• Cherry pick Allow fp8 param gather when using FSDP (13267) into r2.3.0

  #13383 merged May 9, 2025

• Cherry pick Fix skipme handling (13244) into r2.3.0

  #13376 merged May 9, 2025

• Cherry-pick Add recipe and ci scripts for qwen2vl to r2.3.0

  #13336 merged May 9, 2025

• Cherry pick [automodel] bump liger-kernel to 0.5.8 + fallback (13260) into r2.3.0

  #13308 merged May 9, 2025

• Cherry pick build: MAMBA_TAG=2e16fc3062cdcd4ebef27a9aa4442676e1c7edf4 (13173) into r2.3.0

  #13263 merged May 9, 2025

• Cherry pick Use explicitly cached canary-1b-flash in CI tests (13237) into r2.3.0

  #13508 merged May 9, 2025

• Bump to 2.3.1

  #13507 merged May 9, 2025

• Updating the long context performance number for B200

  #13468 merged May 8, 2025

• Publish 2.3.0

  #13506 merged May 8, 2025

• Cherry pick Update changelog for r2.3.0 (13501) into r2.3.0

  #13502 merged May 8, 2025

• Fix changelog formatting

  #13505 merged May 8, 2025

• Enabling flash decode for float16 precision only

  #13471 merged May 8, 2025

• Update 2.3.0 changelog

  #13504 merged May 8, 2025

• ci: Remove trt-llm breakpoint

  #13499 merged May 8, 2025

• Update 2.3.0 changelog

  #13503 merged May 8, 2025

• Update changelog for r2.3.0

  #13501 merged May 8, 2025

• [automodel] fallback FP8 + LCE -> FP8 + CE

  #13349 merged May 8, 2025

18 Pull requests opened by 13 people

• [NeMo 2.0] Nemotron 49B Super HF export Bugfix

  #13495 opened May 8, 2025

• ci: Enable codecov checks

  #13497 opened May 8, 2025

• Update vLLMExporter to use vLLM V1

  #13498 opened May 8, 2025

• Tdt buffered inference fix

  #13500 opened May 8, 2025

• remove blocks unused to increase coverage

  #13511 opened May 9, 2025

• remove the restriction of load_model_state_dict to cfsdp

  #13512 opened May 9, 2025

• Fixing #13509

  #13514 opened May 9, 2025

• HF export in nemo.export

  #13516 opened May 9, 2025

• Hyena SE B2B Kernel integration

  #13518 opened May 9, 2025

• Enable changing quant_cfg via command line in the ptq.py script

  #13519 opened May 9, 2025

• Add use_sharp and use user buffer registration args in perf scripts

  #13521 opened May 9, 2025

• Reconfigure 'limit_<train|val>_batches'

  #13523 opened May 9, 2025

• chore(🤖): Bump `NVIDIA/Megatron-LM` to `fa5eec6...` (2025-05-10)

  #13524 opened May 10, 2025

• chore(🤖): Bump `NVIDIA/Megatron-LM` to `f25dceb...` (2025-05-10)

  #13525 opened May 10, 2025

• Multi token bleu

  #13526 opened May 10, 2025

• Add NemotronH Performance Script

  #13528 opened May 10, 2025

• chore(🤖): Bump `NVIDIA/Megatron-LM` to `fa5eec6...` (2025-05-11)

  #13530 opened May 11, 2025

• chore(🤖): Bump `NVIDIA/Megatron-LM` to `16aeade...` (2025-05-11)

  #13531 opened May 11, 2025

5 Issues opened by 5 people

• Distributed support for preprocess_data_for_megatron.py

  #13529 opened May 10, 2025

• How do you train with speech_to_text_finetune.py and specify Cer instead of Wer

  #13527 opened May 10, 2025

• cannot import name 'L2Norm' from 'megatron.core.transformer.torch_norm'

  #13520 opened May 9, 2025

• UsUsing MegatronCommOverlapCallback(tp_comm_overlap=True) causes segfault.

  #13509 opened May 9, 2025

• How to perform knowledge distillation for Hyena architecture models using NeMo?

  #13496 opened May 8, 2025

45 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• SpeechLM2 collection

  #12617 commented on May 11, 2025 • 18 new comments

• Add Parakeet Hybrid RNNT CTC BPE Model with target language support

  #13360 commented on May 8, 2025 • 9 new comments

• SFT Script with HuggingFace chat template support

  #13273 commented on May 9, 2025 • 7 new comments

• first commit

  #12477 commented on May 9, 2025 • 5 new comments

• Tests for evaluation with NVIDIA Evals Factory

  #12985 commented on May 9, 2025 • 5 new comments

• feat - GPTSFTChatDataset alignment with OpenAI Messages, compatibility with packed sequences

  #13367 commented on May 10, 2025 • 4 new comments

• Hugging face model deployment with Ray Serve

  #13395 commented on May 8, 2025 • 4 new comments

• NeMo2 -> vLLM export via HF

  #13304 commented on May 9, 2025 • 3 new comments

• [Automodel] Fix CP device_mesh issue, use PTL distsampler

  #13473 commented on May 10, 2025 • 3 new comments

• Punctuation Marks in Timestamps

  #13353 commented on May 8, 2025 • 2 new comments

• Update extra_requires and requirements

  #13359 commented on May 9, 2025 • 0 new comments

• [automodel] add find_unused_parameters=True for DDP

  #13366 commented on May 10, 2025 • 0 new comments

• fix eval_beamsearch_ngram_ctc

  #13388 commented on May 9, 2025 • 0 new comments

• Add Qwen2VL export_ckpt

  #13398 commented on May 8, 2025 • 0 new comments

• adding back in PlaceholderFilter

  #13402 commented on May 10, 2025 • 0 new comments

• Add CallbackGroup & Metadata factory function

  #13437 commented on May 9, 2025 • 0 new comments

• UCB r2.3.0

  #13441 commented on May 9, 2025 • 0 new comments

• perf scripts updates

  #13456 commented on May 9, 2025 • 0 new comments

• ci: Remove optional marker

  #13469 commented on May 9, 2025 • 0 new comments

• [Audio] TransformerUNet: predictive model support added

  #13470 commented on May 8, 2025 • 0 new comments

• Cherry pick `Add CI test for local checkpointing (#13012)` into `r2.3.0`

  #13472 commented on May 8, 2025 • 0 new comments

• use null tokenizer

  #13480 commented on May 8, 2025 • 0 new comments

• Alit/nmh4b

  #13481 commented on May 9, 2025 • 0 new comments

• ci: Bump dependencies (#12819)

  #13482 commented on May 9, 2025 • 0 new comments

• deepseek finetuning callback error change

  #13483 commented on May 10, 2025 • 0 new comments

• Possible to convert EncDecRNNTBPEModel into TensorRT and run them through Triton inference server?

  #13457 commented on May 8, 2025 • 0 new comments

• nemo2riva "TypeError: Can't instantiate abstract class ModelPT with abstract methods list_available_models, setup_training_data, setup_validation_data"

  #12751 commented on May 9, 2025 • 0 new comments

• speech_llm/modular_audio_gpt_train.py is not running while freeze_audio_encoder: False

  #12627 commented on May 9, 2025 • 0 new comments

• export_ckpt failed due to AssertionError: dtype mismatch between source and target state dicts

  #13455 commented on May 9, 2025 • 0 new comments

• why CTCDecodingConfig dont work?

  #13155 commented on May 11, 2025 • 0 new comments

• Batched streaming RNN-T and TDT: new buffered inference + support cache-aware

  #9106 commented on May 9, 2025 • 0 new comments

• [resiliency] Add in process integration for Nemo2

  #12589 commented on May 10, 2025 • 0 new comments

• Adding more doc-strings to megatron_parallel.py

  #12767 commented on May 10, 2025 • 0 new comments

• sft nemo2.0 checkin

  #12794 commented on May 11, 2025 • 0 new comments

• Intermediate-tensor distillation support

  #13069 commented on May 8, 2025 • 0 new comments

• Add option distributed_size to MegatronDistributedFusedAdam

  #13102 commented on May 9, 2025 • 0 new comments

• Update Deepseek config for performance + add guard for using HF model directly

  #13188 commented on May 9, 2025 • 0 new comments

• Added multi distopt knobs

  #13196 commented on May 9, 2025 • 0 new comments

• feat: add max_num_unfinalized_calls to AsyncFinalizerCallback

  #13221 commented on May 9, 2025 • 0 new comments

• fix adapter in/out_features size for share_expert when use overlap.

  #13228 commented on May 10, 2025 • 0 new comments

• [automodel][draft] Integrate Megatron Custom FSDP2 into NeMo Automodel.

  #13250 commented on May 9, 2025 • 0 new comments

• Draft: Debug runner

  #13255 commented on May 10, 2025 • 0 new comments

• [DRAFT] LLama4 + nemo.tron

  #13265 commented on May 8, 2025 • 0 new comments

• Test Hyena mixer CP equivalency

  #13330 commented on May 9, 2025 • 0 new comments

• Add MLlama export_ckpt

  #13346 commented on May 9, 2025 • 0 new comments