Nemotron-4 340BOpen Weight

Nemotron-4 340B, released June 14, 2024, was the point at which NVIDIA stopped being only the company every other lab trained on and started publishing frontier-scale models of its own. It was a 340-billion-parameter dense family — Base, Instruct and Reward — pre-trained on 9 trillion tokens and put out under the NVIDIA Open Model License, which permitted commercial use of the models and their outputs. At the time only Meta had published open weights anywhere near that size.

The pitch was unusual: NVIDIA framed the release around synthetic data generation rather than chat, noting that over 98% of the data used in the Instruct model's alignment had itself been generated synthetically, and pairing it with a Reward model that quickly became a standard judge for other labs' data pipelines. That posture — publish the machinery, sell the hardware it runs on — is the strategy every later Nemotron release repeated.