Files
sunhpc/var/pkgs/cuda/13.0/nvml/doc/nvml_changelog.txt
2026-05-05 02:12:33 +08:00

505 lines
42 KiB
Plaintext

/*! @page KnownIssues Known issues in the current version of NVML library
*
* This is a list of known NVML issues in the current driver:
* - NVML Field Values from #251 - #273 (Power Smoothing, Clock Event Reason, and Sync Power Balancing related field values) have changed between 13.0 and 13.0U1/v580TRD2.
* - Any application that is using these field IDs must be recompiled using the NVML header file from CUDA 13.0 Update 1 in order to continue working correctly with NVIDIA drivers v580 TRD2 and beyond.
* - On systems where GPUs are NUMA nodes, the accuracy of FB memory utilization provided by NVML depends on the memory accounting of the operating system.
* This is because FB memory is managed by the operating system instead of the NVIDIA GPU driver.
* Typically, pages allocated from FB memory are not released even after the process terminates to enhance performance. In scenarios where
* the operating system is under memory pressure, it may resort to utilizing FB memory. Such actions can result in discrepancies in the accuracy of memory reporting.
* - On Linux GPU Reset can't be triggered when there is pending GPU Operation Mode (GOM) change
* - On Linux GPU Reset may not successfully change pending ECC mode. A full reboot may be required to enable the mode change.
* - \ref nvmlAccountingStats supports only one process per GPU at a time (CUDA proxy server counts as one process).
* - \ref nvmlAccountingStats_t.time reports time and utilization values starting from cuInit till process termination. Next driver versions might change this behavior slightly and account process only from cuCtxCreate till cuCtxDestroy.
* - On GPUs from Fermi family current P0 clocks (reported by \ref nvmlDeviceGetClockInfo) can differ from max clocks by few MHz.
*/
/*! @page Changelog Change log of NVML library
* This chapter list changes in API and bug fixes that were introduced to the library
* \section changelog32 Changes between NVML v575 and v580 ===
* - Fixed bug with NVML_FI_PWR_SMOOTHING_* Field Value numbering, which was different than the v570 values.
* - Adjusted NVML_FI_DEV_CLOCKS_EVENT_REASON_* and NVML_FI_DEV_POWER_SYNC_BALANCING_* field value numbering to resolve overlap with NVML_FI_PWR_SMOOTHING_* field values.
*
* - Added \ref nvmlDeviceGetSramUniqueUncorrectedEccErrorCounts to get the counts of SRAM unique uncorrected ECC errors.
* - Deprecated Applications Clocks APIs, which will be removed in CUDA 14.0:
* - \ref nvmlDeviceSetApplicationsClocks
* - \ref nvmlDeviceGetApplicationsClock
* - \ref nvmlDeviceGetDefaultApplicationsClock
* - \ref nvmlDeviceResetApplicationsClocks
* - Deprecated \ref nvmlDeviceGetViolationStatus, which will be removed in CUDA 14.0
* - Added \ref nvmlDeviceGetNvLinkInfo to query device NVLINK info.
* - Added \ref nvmlDeviceGetPdi to retrieve the device GPU PDI.
* - Added Multi-GPU mode NVLINK Encryption \ref NVML_CC_SYSTEM_MULTIGPU_NVLE
* - Added V2 struct to \ref nvmlDeviceGetNvLinkInfo to query NVLINK Firmware info.
* - Added \ref nvmlDeviceReadWritePRM_v1 to retrieve GPU PRM register contents
* - Added \ref nvmlDeviceGetAddressingMode to retrieve the addressing mode for the device.
* - Added \ref nvmlDeviceGetRepairStatus to get ECC status info.
* - Added \ref nvmlDeviceGetGpuInstanceProfileInfoByIdV which allows for MIG GPU instance profile info to be queried with profileId instead of profile name.
* - Updated nvmlGpuFabricInfoV_t to v3 to include a new Health Summary field, and new Incorrect Configuration statuses.
- nvmlGpuFabricInfo_v2_t is deprecated and will be removed in a future release
* - Added \ref nvmlDeviceGetPowerMizerMode_v1 to query the current and supported power mizer modes on Maxwell and newer gpus. Power mizer mode provides a hint to the driver as to how to manage the performance of the GPU.
* - Added \ref nvmlDeviceSetPowerMizerMode_v1 to set the power mizer mode on Maxwell and newer gpus.
* - Added new Incorrect Configuration Statuses to nvmlGpuFabricInfoV_t
* - NVML_GPU_FABRIC_HEALTH_MASK_INCORRECT_CONFIGURATION_INCOMPATIBLE_GPU_FW
* - NVML_GPU_FABRIC_HEALTH_MASK_INCORRECT_CONFIGURATION_INVALID_LOCATION
* - Added \ref nvmlDeviceSetHostname_v1 and \ref nvmlDeviceGetHostname_v1 to allow custom GPU hostname configuration.
* \section changelog31 Changes between NVML v570 and v575 ===
*
* - Added \ref nvmlSystemEventSetCreate to create a system event set.
* - Added \ref nvmlSystemEventSetFree to free a system event set.
* - Added \ref nvmlSystemRegisterEvents to register system events on a system event set.
* - Added \ref nvmlSystemEventSetWait to wait for system event notification and obtain system event data.
* - Added \ref nvmlGpuInstanceGetCreatableVgpus to query the currently creatable vGPU types on the user provided GPU Instance
* - Added \ref nvmlVgpuTypeGetMaxInstancesPerGpuInstance to query the maximum number of vGPU instances per GPU Instance for the given vGPU type
* - Added \ref nvmlGpuInstanceSetVgpuSchedulerState to set the vGPU scheduler state for the given GPU Instance
* - Added \ref nvmlGpuInstanceGetActiveVgpus to query the currently active vGPU instances on the user provided GPU Instance
* - Added \ref nvmlGpuInstanceGetVgpuSchedulerState to query the vGPU software scheduler state for the given GPU Instance.
* - Added \ref nvmlGpuInstanceGetVgpuSchedulerLog to query the vGPU software scheduler logs for the given GPU Instance.
* - Added \ref nvmlGpuInstanceGetVgpuTypeCreatablePlacements to query the creatable vGPU placement IDs of the vGPU type within a GPU instance
* - Added \ref nvmlGpuInstanceSetVgpuHeterogeneousMode to enable or disable vGPU heterogenous mode for the GPU Instance.
* - Added \ref nvmlGpuInstanceGetVgpuHeterogeneousMode to query the vGPU heterogenous mode for the GPU Instance.
* - Updated \ref nvmlDeviceGetVgpuCapabilities to report whether GPU supports timesliced vGPU on MIG and whether MIG timesliced mode is enabled or not vGPU capabilities.
* - Updated \ref nvmlDeviceSetVgpuCapabilities to set the MIG timesliced mode vGPU capability of a device.
* - Updated \ref nvmlDeviceSetVgpuHeterogeneousMode to return \ref NVML_ERROR_NOT_SUPPORTED when in MIG mode.
* - Updated \ref nvmlDeviceGetVgpuHeterogeneousMode to return \ref NVML_ERROR_NOT_SUPPORTED when in MIG mode.
* - Updated \ref nvmlDeviceGetVgpuTypeCreatablePlacements to return \ref NVML_ERROR_NOT_SUPPORTED when in MIG mode.
* - Updated \ref nvmlDeviceGetVgpuSchedulerLog to return \ref NVML_ERROR_NOT_SUPPORTED when in MIG mode.
* - Updated \ref nvmlDeviceSetVgpuSchedulerState to return \ref NVML_ERROR_NOT_SUPPORTED when in MIG mode.
* - Updated \ref nvmlDeviceGetVgpuSchedulerState to return \ref NVML_ERROR_NOT_SUPPORTED when in MIG mode.
* - Added 3 new NVML_FI_DEV_C2C_LINK_ERROR fieldIds
* - \ref NVML_FI_DEV_C2C_LINK_ERROR_INTR
* - \ref NVML_FI_DEV_C2C_LINK_ERROR_REPLAY
* - \ref NVML_FI_DEV_C2C_LINK_ERROR_REPLAY_B2B
* - Added new NVML_FI_DEV_C2C_LINK_POWER_STATE fieldId
* - Added new CTXSW GPM Metrics
* - Added \ref nvmlDeviceGetHandleByUUIDV that supports both the ASCII and binary format UUID to retrieve the device handle.
* - Added 2 new NVML_FI_DEV_POWER_SYNC_BALANCING fieldIds
* - \ref NVML_FI_DEV_POWER_SYNC_BALANCING_FREQ
* - \ref NVML_FI_DEV_POWER_SYNC_BALANCING_AF
* - Added 5 new Clock Event Reason Counters fieldIds
* - \ref NVML_FI_DEV_CLOCKS_EVENT_REASON_SW_POWER_CAP
* - \ref NVML_FI_DEV_CLOCKS_EVENT_REASON_SYNC_BOOST
* - \ref NVML_FI_DEV_CLOCKS_EVENT_REASON_SW_THERM_SLOWDOWN
* - \ref NVML_FI_DEV_CLOCKS_EVENT_REASON_HW_THERM_SLOWDOWN
* - \ref NVML_FI_DEV_CLOCKS_EVENT_REASON_HW_POWER_BRAKE_SLOWDOWN
* - Updated \ref nvmlDeviceGetMemoryErrorCounter to better account for transient vs. permanent errors
* - Added MIG profiles that can allocate all or none of Decoder, Encoder, JPEG and OFA engines.
*
* \section changelog30 Changes between NVML v565 Update and v570 ===
* - Revert the fix for the issue where PCIe throughput (reported via \ref nvmlDeviceGetPcieThroughput and nvidia-smi -q) is 1000 times bigger than its actual value
* - Added field values for data related to Power Smoothing
* - Added \ref nvmlDevicePowerSmoothingActivatePresetProfile to activate a specific Preset Profile for Power Smoothing
* - Added \ref nvmlDevicePowerSmoothingSetState to enable/disable the Power Smoothing feature
* - Added \ref nvmlDevicePowerSmoothingUpdatePresetProfileParam to update parameters to preset profiles for Power Smoothing
* - Added new enums for fieldId NVML_FI_DEV_NVLINK_GET_STATE to expose INACTIVE, ACTIVE, and SLEEP state for a link
* - Added \ref nvmlDeviceGetMarginTemperature to retrieve the thermal margin temperature (distance to nearest slowdown threshold).
* - Added \ref nvmlDeviceGetNvlinkSupportedBwModes to get all supported Nvlink Bandwidth modes
* - Added \ref nvmlDeviceGetNvlinkBwMode to get the current Nvlink Bandwidth mode
* - Added \ref nvmlDeviceSetNvlinkBwMode to set the Nvlink Bandwidth mode
* - Added MIG profiles with support for graphics.
* - Added support for new recovery action - NVML_GPU_RECOVERY_ACTION_DRAIN_AND_RESET
* - Deprecated nvml fieldIds NVML_FI_DEV_RESET_STATUS and NVML_FI_DEV_DRAIN_AND_RESET_STATUS. Usee NVML_FI_DEV_GET_GPU_RECOVERY_ACTION instead
* - Added \ref nvmlDeviceGetDramEncryptionMode and \ref nvmlDeviceSetDramEncryptionMode to query and configure DRAM Encryption Mode
* - Added 3 new flags to GPU Fabric Health Mask
* - NVML_GPU_FABRIC_HEALTH_MASK_SHIFT_ROUTE_RECOVERY
* - NVML_GPU_FABRIC_HEALTH_MASK_SHIFT_ROUTE_UNHEALTHY
* - NVML_GPU_FABRIC_HEALTH_MASK_SHIFT_ACCESS_TIMEOUT_RECOVERY
* - Added 4 new GPM metrics
* - NVML_GPM_METRIC_NVENC_0_UTIL
* - NVML_GPM_METRIC_NVENC_1_UTIL
* - NVML_GPM_METRIC_NVENC_2_UTIL
* - NVML_GPM_METRIC_NVENC_3_UTIL
* - Added new counters for Nvlink5
* - \ref NVML_FI_DEV_NVLINK_COUNT_EFFECTIVE_ERRORS to get sum of the number of errors in each Nvlink packet
* - \ref NVML_FI_DEV_NVLINK_COUNT_EFFECTIVE_BER to get Effective BER for effective errors
* - \ref NVML_FI_DEV_NVLINK_COUNT_FEC_HISTORY_0 to 15 to get count of symbol errors that are corrected
* - Swapped the values of field IDs \ref NVML_FI_DEV_IS_MIG_MODE_INDEPENDENT_MIG_QUERY_CAPABLE and \ref NVML_FI_DEV_NVLINK_GET_POWER_THRESHOLD_MAX to fix backwards compatibility with v550.
* - New revision of nvmlPlatformInfo_t -- nvmlPlatformInfo_v2 has been added. In this version the following fields from v1 have been renamed
* - rackGuid to chassisSerialNumber
* - chassisPhysicalSlotNumber to slotNumber
* - computeSlotIndex to trayIndex
* - nodeIndex to hostId
* - nvmlPlatformInfo_v1 is deprecated and will be removed in subsequent releases
*
* \section changelog29 Changes between NVML v560 Update and v565 ===
* - Fixed the ECC error count mismatch between nvidia-smi query output and NVML APIs, \ref nvmlDeviceGetMemoryErrorCounter and \ref nvmlDeviceGetFieldValues.
* - Added new value NVML_CC_SYSTEM_CPU_CAPS_AMD_SNP_VTOM for CC CPU capability reporting
* - Added \ref nvmlDeviceGetCoolerInfo to retrieve a cooler's control signal characteristics and target that cooler cools.
* - Added new value NVML_CC_SYSTEM_CPU_CAPS_AMD_SEV_SNP for CC CPU capability reporting
* - Added \ref nvmlDeviceGetFanSpeedRPM to report the intended operating speed in rotations per minute (RPM) of the device's specified fan.
* - Added \ref nvmlDeviceGetPerformanceModes to retrieve a performance modes string with all the performance modes defined for this device along with their associated GPU Clock and Memory Clock values.
* - Added \ref nvmlDeviceGetCurrentClockFreqs to retrieve a string with the associated GPU Clock and Memory Clock values for the current pstate.
* - Added \ref nvmlNvlinkVersion_t enum to define NvLink Version
* - Added \ref nvmlDeviceGetPlatformInfo to retrieve the platform information of a device
* - Added new event type nvmlEventTypeGpuUnavailableError
* - Removed support for \p nvmlDeviceGetNvLinkCrcLaneErrorCounter \p nvmlDeviceGetNvLinkEccLaneErrorCounter \p nvmlDeviceGetNvLinkErrorCounter on Blackwell
* - Removed support for fieldIds \ref NVML_FI_DEV_NVLINK_ERROR_DL_REPLAY \ref NVML_FI_DEV_NVLINK_ERROR_DL_RECOVERY \ref NVML_FI_DEV_NVLINK_ERROR_DL_CRC on Blackwell
* - Added \ref nvmlVgpuInstanceGetRuntimeStateSize to get the vGPU runtime state size
* - Updated nvmlDeviceGetVgpuTypeSupportedPlacements function to report both Heterogeneous and Homogeneous vGPU placements.
* - Updated nvmlDeviceGetVgpuCapabilities to report the Homogeneous vGPU capability.
* - Added \ref nvmlDeviceWorkloadPowerProfileGetProfilesInfo to retrieve Workload Power Profile Info
* - Added \ref nvmlDeviceWorkloadPowerProfileGetCurrentProfiles to retrieve current Requested and Enforced Workload Power Profiles
* - Added \ref nvmlDeviceWorkloadPowerProfileSetRequestedProfiles to set Requested Workload Power Profiles
* - Added \ref nvmlDeviceWorkloadPowerProfileClearRequestedProfiles to clear Requested Performance Profiles
* - Added new event type nvmlEventTypeGpuRecoveryAction
* - Added new fieldId to query gpu recovery action NVML_FI_DEV_GET_GPU_RECOVERY_ACTION
* - Deprecated fieldIds
* - \ref NVML_FI_DEV_NVLINK_COUNT_VL15_DROPPED to get Number of VL15 MADs dropped on a link in NVLink5
* - \ref NVML_FI_DEV_NVLINK_COUNT_RAW_BER_LANE0 to get BER per lane for lane 0
* - \ref NVML_FI_DEV_NVLINK_COUNT_RAW_BER_LANE1 to get BER per lane for lane 1
* - \ref NVML_FI_DEV_NVLINK_COUNT_RAW_BER to get BER per link. Sum of all the raw errors per lane/Bits received per link
* - \ref NVML_FI_DEV_NVLINK_COUNT_EFFECTIVE_ERRORS to get Sum of the number of errors in each Nvlink packet
* - \ref NVML_FI_DEV_NVLINK_COUNT_EFFECTIVE_BER to get Effective BER for effective errors
*
* \section changelog28 Changes between NVML v555 Update and v560 ===
*
* - Added field values NVML_FI_DEV_PCIE_OUTBOUND_ATOMICS_MASK and NVML_FI_DEV_PCIE_INBOUND_ATOMICS_MASK for nvmlDeviceGetFieldValues.
* - Added field ids NVML_FI_DEV_RESET_STATUS and NVML_FI_DEV_DRAIN_AND_RESET_STATUS which correspond to the nvidia-smi output.
* - Added NVML_DEVICE_ARCH_T23X architecture type.
* - Added \ref nvmlVgpuTypeGetBAR1Info to query the BAR1 information of a vGPU type.
* - Added new event types, nvmlEventTypeSingleBitEccErrorStorm, nvmlEventTypeDramRetirementEvent, nvmlEventTypeDramRetirementFailure, nvmlEventTypeNonFatalPoisonError and nvmlEventTypeFatalPoisonError.
* - Added \ref nvmlSystemGetDriverBranch to query the driver branch information.
*
* \section changelog27 Changes between NVML v550 Update and v555 ===
*
* - Added \ref nvmlDeviceGetClockOffsets to query min, max and current clock offset value on a Maxwell and later GPU for a specified clock.
* - Added \ref nvmlDeviceSetClockOffsets to control clock offset value on a Maxwell and later GPU for a specified clock.
* - Added new fieldIds for Nvlink5 telemetry on Blackwell
* - \ref NVML_FI_DEV_NVLINK_COUNT_XMIT_PACKETS to get Total Tx packets on the link in NVLink5
* - \ref NVML_FI_DEV_NVLINK_COUNT_XMIT_BYTES to get Total Tx bytes on the link in NVLink5
* - \ref NVML_FI_DEV_NVLINK_COUNT_RCV_PACKETS to get Total Rx packets on the link in NVLink5
* - \ref NVML_FI_DEV_NVLINK_COUNT_RCV_BYTES to get Total Rx bytes on the link in NVLink5
* - \ref NVML_FI_DEV_NVLINK_COUNT_VL15_DROPPED to get Number of VL15 MADs dropped on a link in NVLink5
* - \ref NVML_FI_DEV_NVLINK_COUNT_MALFORMED_PACKET_ERRORS to get Number of packets Rx on a link where packets are malformed
* - \ref NVML_FI_DEV_NVLINK_COUNT_BUFFER_OVERRUN_ERRORS to get Number of packets that were discarded on Rx due to buffer overrun
* - \ref NVML_FI_DEV_NVLINK_COUNT_RCV_ERRORS to get Total number of packets with errors Rx on a link
* - \ref NVML_FI_DEV_NVLINK_COUNT_RCV_REMOTE_ERRORS to get Total number of packets Rx - stomp/EBP marker
* - \ref NVML_FI_DEV_NVLINK_COUNT_RCV_GENERAL_ERRORS to get Total number of packets Rx with header mismatch
* - \ref NVML_FI_DEV_NVLINK_COUNT_LOCAL_LINK_INTEGRITY_ERRORS to get Total number of times that the count of local errors exceeded a threshold
* - \ref NVML_FI_DEV_NVLINK_COUNT_XMIT_DISCARDS to get Total number of tx error packets that were discarded
* - \ref NVML_FI_DEV_NVLINK_COUNT_LINK_RECOVERY_SUCCESSFUL_EVENTS to get Number of times link went from Up to recovery, succeeded and link came back up
* - \ref NVML_FI_DEV_NVLINK_COUNT_LINK_RECOVERY_FAILED_EVENTS to get Number of times link went from Up to recovery, failed and link was declared down
* - \ref NVML_FI_DEV_NVLINK_COUNT_LINK_RECOVERY_EVENTS to get Number of times link went from Up to recovery, irrespective of the result
* - \ref NVML_FI_DEV_NVLINK_COUNT_RAW_BER_LANE0 to get BER per lane for lane 0
* - \ref NVML_FI_DEV_NVLINK_COUNT_RAW_BER_LANE1 to get BER per lane for lane 1
* - \ref NVML_FI_DEV_NVLINK_COUNT_RAW_BER to get BER per link. Sum of all the raw errors per lane/Bits received per link
* - \ref NVML_FI_DEV_NVLINK_COUNT_EFFECTIVE_ERRORS to get Sum of the number of errors in each Nvlink packet
* - \ref NVML_FI_DEV_NVLINK_COUNT_EFFECTIVE_BER to get Effective BER for effective errors
* - \ref NVML_FI_DEV_NVLINK_COUNT_SYMBOL_ERRORS to get Number of errors in rx symbols
* - \ref NVML_FI_DEV_NVLINK_COUNT_SYMBOL_BER to get BER for symbol errors
* - Added two new field ids NVML_FI_DEV_PCIE_COUNT_TX_BYTES and NVML_FI_DEV_PCIE_COUNT_RX_BYTES for nvmlDeviceGetFieldValues.
* - Added new API nvmlDeviceGetCapabilities with the first capability bit NVML_DEV_CAP_EGM for Extended GPU Memory (EGM) capability.
* - Added multiGpuMode display on CC enabled system via new API nvmlSystemGetConfComputeSettings or "nvidia-smi conf-compute --get-multigpu-mode" or "nvidia-smi conf-compute -mgm".
* - Added new field ID \ref NVML_FI_DEV_NVLINK_GET_POWER_THRESHOLD_MAX to get the Max Nvlink Power Threshold for a device.
* - Deprecated \ref nvmlDeviceGetTemperature and replaced with a new API \ref nvmlDeviceGetTemperatureV to retrieve device temperature.
* - Added new field ID \ref NVML_VGPU_DRIVER_CAP_WARM_UPDATE and NVML_DEVICE_VGPU_CAP_WARM_UPDATE to query whether the driver and the device supports FSR and warm update of vGPU host driver without terminating the running guest VM respectively.
* - Added new field ID \ref NVML_FI_DEV_NVLINK_GET_POWER_THRESHOLD_MIN to get the Min Nvlink Power Threshold for a device.
* - Added new field ID \ref NVML_FI_DEV_NVLINK_GET_POWER_THRESHOLD_UNITS to get the Units of the Nvlink Power Threshold for a device.
* - Added new field ID \ref NVML_FI_DEV_NVLINK_GET_POWER_THRESHOLD_SUPPORTED to get if Nvlink Power Threshold is supported for a device.
*
*
* \section changelog26 Changes between NVML v545 Update and v550 ===
*
* - Added \ref nvmlDeviceGetNumaNodeId to query the NUMA node of a GPU.
* - Fix the issue where PCIe throughput (reported via nvmlDeviceGetPcieThroughput and nvidia-smi -q) is 1000 times bigger than its actual value.
* - Added new GPM metric Id NVML_GPM_METRIC_NVOFA_1_UTIL to \ref nvmlGpmMetricId_t.
* - Added new field ID \ref NVML_FI_DEV_IS_MIG_MODE_INDEPENDENT_MIG_QUERY_CAPABLE, to check MIG query capable device irrespective of MIG mode.
* - Deprecated NVML_P2P_CAPS_INDEX_PROP and added NVML_P2P_CAPS_INDEX_PCI to reflect the same P2P capability.
* - Added \ref nvmlDeviceGetProcessesUtilizationInfo to retrieve the recent utilization and process ID for all running processes.
* - Added new struct \ref nvmlProcessesUtilizationInfo_v1_t, which includes the new utilization of NVJPG and NVOFA.
* - Added \ref nvmlDeviceGetVgpuInstancesUtilizationInfo to retrieve the recent utilization for vGPU instances running on a physical GPU.
* - Added \ref nvmlDeviceGetVgpuProcessesUtilizationInfo to retrieve the recent utilization for processes running on vGPU instances on a physical GPU.
* - Added \ref nvmlDeviceSetVgpuHeterogeneousMode to enable or disable vGPU heterogenous mode for the device.
* - Added \ref nvmlDeviceGetVgpuHeterogeneousMode to query the vGPU heterogenous mode for the device.
* - Added \ref nvmlVgpuInstanceGetPlacementId to query placement ID of the active vGPU instance.
* - Added \ref nvmlDeviceGetVgpuTypeSupportedPlacements to query the supported vGPU placement IDs of a vGPU type.
* - Added \ref nvmlDeviceGetVgpuTypeCreatablePlacements to query the creatable vGPU placement IDs of a vGPU type.
* - Added support to display confidential compute protected memory along with fb & bar1 in nvidia-smi pmon & dmon commands.
* - Added \ref nvmlDeviceGetGpuFabricInfoV to query Gpu Fabric Probe Info for the device.
* - Deprecated \ref nvmlDeviceGetGpuFabricInfo. This function should not be used, and will be removed in a future release. Use \ref nvmlDeviceGetGpuFabricInfoV instead.
* - Modified \ref nvmlDeviceGetGpuInstanceProfileInfo and \ref nvmlDeviceGetGpuInstancePossiblePlacements_v2 to no longer require MIG being enabled
* - Added \ref nvmlSystemSetConfComputeKeyRotationThresholdInfo to set confidential compute key rotation threshold.
* - Added \ref nvmlSystemGetConfComputeKeyRotationThresholdInfo to query confidential compute key rotation threshold detail.
* - Added \ref nvmlDeviceSetVgpuCapabilities to set the desirable vGPU capability of a device.
*
*
* \section changelog25 Changes between NVML v535 Update and v545 ===
*
* - Added a new error code \ref NVML_ERROR_GPU_NOT_FOUND to be returned if no supported GPUS are found during initialization.
* - In \ref nvmlGpuFabricInfo_t \p partitionId has been renamed to \p cliqueId.
* - Added new versioned structs \ref nvmlGpuInstanceProfileInfo_v3_t and \ref nvmlComputeInstanceProfileInfo_v3_t.
* - Added \ref nvmlDeviceGetLastBBXFlushTime for retrieving the timestamp and duration of the latest flush of the BBX object to the inforom storage.
* - Added \ref NVML_POWER_SCOPE_MEMORY to report out power usage for GPU Memory.
* - Added \ref nvmlDeviceGetPciInfoExt which expands \ref nvmlDeviceGetPciInfo_v3 to also report PCI base and sub classcodes.
* - Added new struct \ref nvmlPciInfoExt_v1_t, which is used in \ref nvmlDeviceGetPciInfoExt.
* - Added \ref nvmlDeviceGetRunningProcessDetailList api to get information about Compute, Graphics or MPS-Compute processes running on a GPU with protected memory usage info.
*
*
* \section changelog24 Changes between NVML v530 Update and v535 ===
*
* - Fixed \ref nvmlDeviceGetMemoryErrorCounter and \ref nvmlDeviceGetFieldValues to return correct SRAM volatile total error counts.
* - Added \ref nvmlDeviceGetSramEccErrorStatus to query SRAM ECC error status for the device.
* - Added \ref nvmlDeviceGetModuleId for getting device module id
* - Updated \ref nvmlDeviceGetPowerSource API to report undersized power source.
* - Added \ref nvmlDeviceGetJpgUtilization and \ref nvmlDeviceGetOfaUtilization APIs
* - Added \ref nvmlSystemGetNvlinkBwMode and \ref nvmlSystemSetNvlinkBwMode APIs
* - Added \ref nvmlDeviceSetVgpuSchedulerState to set the vGPU scheduler state.
* - Added new field ID \ref NVML_FI_DEV_IS_RESETLESS_MIG_SUPPORTED for device's resetless MIG capability
* - Added \ref nvmlDeviceGetComputeRunningProcesses_v3 to get information about Compute processes running on a GPU.
* - Added \ref nvmlDeviceGetGraphicsRunningProcesses_v3 to get information about Graphics processes running on a GPU.
* - Added \ref nvmlDeviceGetMPSComputeRunningProcesses_v3 to get information about MPS-Compute processes running on a GPU.
* - Added \ref nvmlDeviceGetRunningProcessDetailList to get information about Compute, Graphics or MPS-Compute processes running on a GPU with protected memory usage info. Currently returns NVML_ERROR_NOT_SUPPORTED. Functionality will be implemented in next release.
* - Added new field ID \ref NVML_FI_DEV_PCIE_COUNT_CORRECTABLE_ERRORS for PCIe correctable errors counter
* - Added new field ID \ref NVML_FI_DEV_PCIE_COUNT_NAKS_RECEIVED for PCIe NAK Receive counter
* - Added new field ID \ref NVML_FI_DEV_PCIE_COUNT_RECEIVER_ERROR for PCIe receiver error counter
* - Added new field ID \ref NVML_FI_DEV_PCIE_COUNT_BAD_TLP for PCIe bad TLP counter
* - Added new field ID \ref NVML_FI_DEV_PCIE_COUNT_NAKS_SENT for NAK Send counter
* - Added new field ID \ref NVML_FI_DEV_PCIE_COUNT_BAD_DLLP for PCIe bad DLLP counter
* - Added new field ID \ref NVML_FI_DEV_PCIE_COUNT_NON_FATAL_ERROR for PCIe non fatal error counter
* - Added new field ID \ref NVML_FI_DEV_PCIE_COUNT_FATAL_ERROR for PCIe fatal error counter
* - Added new field ID \ref NVML_FI_DEV_PCIE_COUNT_UNSUPPORTED_REQ for PCIe unsupported request counter
* - Added new field ID \ref NVML_FI_DEV_PCIE_COUNT_LCRC_ERROR for PCIe LCRC error counter
* - Added new field ID \ref NVML_FI_DEV_PCIE_COUNT_LANE_ERROR for per lane error counter with scope as PCIe lane number.
* - Added \ref nvmlDeviceGetPowerUsage to retrieve current power usage
* - Added \ref nvmlDeviceGetTotalEnergyConsumption to get current energy consumption
* - Added \ref nvmlDeviceSetPowerManagementLimit_v2 to set the power limit
* - Renamed nvmlDeviceCcuGetStreamState to nvmlGpmQueryIfStreamingEnabled and nvmlDeviceCcuSetStreamState to nvmlGpmSetStreamingEnabled.
*
*
* \section changelog23 Changes between NVML v525 Update and v530 ===
*
* - Fixed a typo in nvmlGpuP2PStatus_t: added a new enum entry for NVML_P2P_STATUS_CHIPSET_NOT_SUPPORTED with the same numeric value as the existing erroneous entry ("NVML_P2P_STATUS_CHIPSET_NOT_SUPPORED")
* - Added \ref nvmlDeviceGetVgpuSchedulerLog to fetch the vGPU software scheduler logs.
* - Added \ref nvmlDeviceGetVgpuSchedulerState to fetch the vGPU software scheduler state.
* - Added \ref nvmlDeviceGetVgpuSchedulerCapabilities to fetch the vGPU software scheduler capabilities.
*
*
* \section changelog22 Changes between NVML v520 Update and v525 ===
*
* - Added \ref nvmlDeviceSetNvLinkDeviceLowPowerThreshold to set the NvLink low power threshold.
* - Added \p nvmlDeviceGetPcieAtomicCaps to report PCIe atomic capabilities.
* - Added \p nvmlDeviceCcuGetStreamState API to report the counter collection unit stream state.
* - Added \p nvmlDeviceCcuSetStreamState API to set the counter collection unit stream state.
* - Removed support for NVML_FI_DEV_LINK_SPEED_MBPS_L{0..} field Ids in Hopper. Replaced with NVML_FI_DEV_NVLINK_GET_SPEED with scope as link Id.
* - Removed support for NVML_FI_DEV_NVLINK_CRC_FLIT_ERROR_COUNT{0..} field Ids in Hopper. Replaced with NVML_FI_DEV_NVLINK_ERROR_DL_CRC with scope as link Id.
* - Removed support for NVML_FI_DEV_NVLINK_REPLAY_ERROR_COUNT_L{0..} field Ids in Hopper. Replaced with NVML_FI_DEV_NVLINK_ERROR_DL_REPLAY with scope as link Id.
* - Removed support for NVML_FI_DEV_NVLINK_RECOVERY_ERROR_COUNT_{0..} field Ids in Hopper. Replaced with NVML_FI_DEV_NVLINK_ERROR_DL_RECOVERY with scope as link Id.
* - Added new field ID \ref NVML_FI_DEV_NVLINK_GET_STATE to get nvlink state
* - Added new field ID \ref NVML_FI_DEV_NVLINK_GET_VERSION to get nvlink version
* - Added new field ID \ref NVML_FI_DEV_C2C_LINK_COUNT to get C2C link count
* - Added new field ID \ref NVML_FI_DEV_C2C_LINK_GET_STATUS to get C2C link status
* - Added new field ID \ref NVML_FI_DEV_C2C_LINK_GET_MAX_BW to get C2C link bandwidth
*
*
* \section changelog21 Changes between NVML v515 Update and v520 ===
*
* - Added \ref nvmlDeviceGetMemClkVfOffset API to report the MemClk VF offset value.
* - Added \ref nvmlDeviceGetMemClkMinMaxVfOffset API to report the Memory clock min and max VF offset that user can set for a specified GPU.
* - Added \ref nvmlDeviceGetGpcClkMinMaxVfOffset API to report the Graphics clock min and max VF offset that user can set for a specified GPU.
* - Added \ref nvmlGpmMetricsGet to calculate GPM metrics from two GPM samples
* - Added \ref nvmlGpmSampleFree to free allocated GPM sample
* - Added \ref nvmlGpmSampleAlloc to allocate a GPM sample
* - Added \ref nvmlGpmSampleGet to retrieve a GPM snapshot
* - Added \ref nvmlGpmQueryDeviceSupport to query whether a device supports GPM
* - Added \ref nvmlDeviceGetFanControlPolicy_v2 API to report the control policy for a specified GPU fan.
* - Added \ref nvmlDeviceSetFanControlPolicy API to set the control policy for a specified GPU fan.
*
*
* \section changelog20 Changes between NVML v510 Update and v515 ===
*
* - Added \ref nvmlDeviceGetMinMaxClockOfPState API to report the min and max clocks of some clock domain for a given PState.
* - Added \ref nvmlDeviceGetSupportedPerformanceStates API to get all supported Performance States (P-States) for the GPU.
* - Added \ref nvmlDeviceGetGpcClkVfOffset API to report the GPCCLK VF offset value.
* - Added \ref nvmlDeviceGetMinMaxFanSpeed API to report the min and max fan speed that user can set for a specified GPU fan.
*
*
* \section changelog19 Changes between NVML v495 Update and v510 ===
*
* - Added \ref nvmlDeviceGetGpuInstanceProfileInfoV and \ref nvmlGpuInstanceGetComputeInstanceProfileInfoV APIs to include the profile name in their output.
* - Added \ref nvmlDeviceGetMemoryBusWidth API to report the GPU's Memory Bus Width.
* - Added \ref nvmlDeviceGetPcieLinkMaxSpeed API to report the GPU's PCIe Max Speed.
* - Added \ref nvmlDeviceGetPowerSource API to report the GPU's power source as AC or battery.
* - Added \ref nvmlDeviceGetNumFans API to report the GPU's number of fans.
* - Added \ref nvmlDeviceGetNumGpuCores API to report the GPU's number of cores.
* - Added \ref nvmlDeviceGetMemoryInfo_v2. The new version accounts separately for system-reserved memory, and includes it in the used memory amount. The previous version of the API reduced the total memory amount by the amount of system-reserved memory.
* - Added \ref nvmlDeviceGetAdaptiveClockInfoStatus API to report the status of adaptive clocking for the GPU.
*
*
* \section changelog18 Changes between NVML v465 Update and v470 ===
*
* - Added new MIG GPU instance profile NVML_GPU_INSTANCE_PROFILE_1_SLICE_REV1.
* - Added \ref nvmlDeviceGetGpuInstancePossiblePlacements_v2. The previous version of the API will not support the profiles with possible placements greater than its total capacity, such as NVML_GPU_INSTANCE_PROFILE_1_SLICE_REV1.
*
*
* \section changelog17 Changes between NVML v460 Update and v465 ===
*
* - Added new NVML_BRAND_* enumeration values for NVIDIA, NVIDIA_RTX, GEFORCE_RTX, QUADRO_RTX and TITAN_RTX
* - Updated \ref nvmlDeviceGetHandleByUUID to make it MIG-aware.
* - Updated \ref nvmlDeviceGetUUID to return MIG UUIDs in the canonical format, 'MIG-UUID'.
* - Updated \ref nvmlDeviceGetHandleByUUID to accept both UUID formats, 'MIG-UUID' and 'MIG-GPU UUID/GID/CID'.
* - The \ref nvmlDeviceSetAPIRestriction and \ref nvmlDeviceGetAPIRestriction APIs would no longer support the ability to toggle root-only requirement for \ref nvmlDeviceSetApplicationsClocks and \ref nvmlDeviceResetApplicationsClocks.
*
*
* \section changelog16 Changes between NVML v450 Update and v460 ===
*
* - Added \ref nvmlDeviceCreateGpuInstanceWithPlacement to allow placement specification when creating a new MIG GPU instance.
*
*
* \section changelog15 Changes between NVML v445 Update and v450 ===
*
* - Updated \ref nvmlDeviceGetFanSpeed and \ref nvmlDeviceGetFanSpeed_v2 for allowing fan speeds greater than 100% to be reported.
* - Added \ref nvmlDeviceGetCpuAffinityWithinScope to determine the closest processor(s) within a NUMA node or socket.
* - Added \ref nvmlDeviceGetMemoryAffinity to determine the closest NUMA node(s) within a NUMA node or socket.
* - Added support to query and disable MIG mode on Windows.
*
*
* \section changelog14 Changes between NVML v418 Update and v445 ===
*
* - Added support for NVIDIA Ampere architecture.
* - Added support for Multi Instance GPU management. Refer "Multi Instance GPU Management" section for details.
*
*
* \section changelog13 Changes between NVML v361 Update and v418
*
* - Support for Volta and Turing architectures, bug fixes, performance improvements, and new features
*
*
* \section changelog12 Changes between NVML v349 Update and v361
*
* - Added \ref nvmlDeviceGetBoardPartNumber to return GPU part numbers
* - Removed support for exclusive thread compute mode (Deprecated in 7.5)
* - Added NVML_CLOCK_VIDEO (encoder/decoder) clock type as a supported clock type for \ref nvmlDeviceGetClockInfo and \ref nvmlDeviceGetMaxClockInfo.
*
*
* \section changelog11 Changes between NVML v346 Update and v349
*
* The following new functionality is exposed on NVIDIA display drivers version 349 Production or later
* - Updated \ref nvmlDeviceGetMemoryInfo to report Used/Free memory under Windows WDDM mode
* - Added \ref nvmlDeviceGetTopologyCommonAncestor to find the common path between two devices
* - Added \ref nvmlDeviceGetTopologyNearestGpus to get a set of GPUs given a path level
* - Added \ref nvmlSystemGetTopologyGpuSet to retrieve a set of GPUs with a given CPU affinity
* - Updated \ref nvmlDeviceGetAccountingPids, \ref nvmlDeviceGetAccountingBufferSize and \ref nvmlDeviceGetAccountingStats to report accounting information for both active and terminated processes. The execution time field in \ref nvmlAccountingStats_t structure is populated only when the process is terminated.
*
*
* \section changelog10 Changes between NVML v340 Update and v346
*
* The following new functionality is exposed on NVIDIA display drivers version 346 Production or later
* - added the public APIs nvmlDeviceGetPcieReplayCounter and nvmlDeviceGetPcieThroughput
* - Discontinued Perl bindings support
* - Added \p nvmlDeviceGetGraphicsRunningProcesses_v2 to get information about Graphics processes running on a GPU.
*
*
* \section changelog9 Changes between NVML v331 Update and v340
*
* The following new functionality is exposed on NVIDIA display drivers version 340 Production or later
* - Added \ref nvmlDeviceGetSamples to get recent power, utilization and clock samples for the GPU.
* - Added \ref nvmlDeviceGetTemperatureThreshold to retrieve temperature threshold information.
* - Added \ref nvmlDeviceGetBrand to retrieve brand information (e.g. Tesla, Quadro, etc.)
* - Added support for K40d and K80
* - Added nvmlDeviceGetTopology internal API to retrieve path info between PCI devices (remove this for DITA)
* - Added \ref nvmlDeviceGetViolationStatus to get the duration of time during which the device was throttled (lower than requested clocks) due to thermal or power constraints.
* - Added \ref nvmlDeviceGetEncoderUtilization and \ref nvmlDeviceGetDecoderUtilization APIs
* - Added \ref nvmlDeviceGetCpuAffinity to determine the closest processor(s) affinity to a specific GPU
* - Added \ref nvmlDeviceSetCpuAffinity to bind a specific GPU to the closest processor
* - Added \ref nvmlDeviceClearCpuAffinity to unbind a specific GPU
* - Added \ref nvmlDeviceGetBoardId to get a unique boardId for the running system
* - Added \ref nvmlDeviceGetMultiGpuBoard to get whether the device is on a multiGPU board
* - Added \ref nvmlDeviceGetAutoBoostedClocksEnabled and nvmlDeviceSetAutoBoostedClocksEnabled for querying and setting the state of auto boosted clocks on supporting hardware.
* - Added \ref nvmlDeviceSetDefaultAutoBoostedClocksEnabled for setting the default state of auto boosted clocks on supporting hardware.
*
*
* \section changelog8 Changes between NVML v5.319 Update and v331
*
* The following new functionality is exposed on NVIDIA display drivers version 331 Production or later
* - Added \ref nvmlDeviceGetMinorNumber to get the minor number for the device.
* - Added \ref nvmlDeviceGetBAR1MemoryInfo to get BAR1 total, available and used memory size.
* - Added \ref nvmlDeviceGetBridgeChipInfo to get the information related to bridge chip firmware.
* - Added enforced power limit query API \ref nvmlDeviceGetEnforcedPowerLimit
* - Updated \ref nvmlEventSetWait_v2 to return xid event data in case of xid error event.
* - Added support for K8
*
* \section changelog7 Changes between NVML v5.319 RC and v5.319 Update
*
* The following new functionality is exposed on NVIDIA display drivers version 319 Update or later
*
* - Added \ref nvmlDeviceSetAPIRestriction and \ref nvmlDeviceGetAPIRestriction, with initial ability to toggle root-only requirement for \ref nvmlDeviceSetApplicationsClocks and \ref nvmlDeviceResetApplicationsClocks.
*
* \section changelog6 Changes between NVML v4.304 and v5.319 RC
*
* The following new functionality is exposed on NVIDIA display drivers version 319 Production or later
*
* - IMPORTANT: Added _v2 versions of \ref nvmlDeviceGetHandleByIndex_v2 and \ref nvmlDeviceGetCount_v2 that also count devices not accessible by current user
* - IMPORTANT: nvmlDeviceGetHandleByIndex_v2 (default) can also return NVML_ERROR_NO_PERMISSION
* - Added nvmlInit_v2 and nvmlDeviceGetHandleByIndex_v2 that is safer and thus recommended function for initializing the library
* - nvmlInit_v2 lazily initializes only requested devices (queried with nvmlDeviceGetHandle*)
* - nvml.h defines nvmlInit_v2 and nvmlDeviceGetHandleByIndex_v2 as default functions
* - Added \ref nvmlDeviceGetIndex
* - Added \ref NVML_ERROR_GPU_IS_LOST to report GPUs that have fallen off the bus.
* - Note: All NVML device APIs can return this error code, as a GPU can fall off the bus at any time.
* - Added new class of APIs for gathering process statistics (\ref nvmlAccountingStats)
* - Application Clocks are no longer supported on GPU's from Quadro product line
* - Added APIs to support dynamic page retirement. See \ref nvmlDeviceGetRetiredPages and
* \ref nvmlDeviceGetRetiredPagesPendingStatus
* - Renamed nvmlClocksThrottleReasonUserDefinedClocks to nvmlClocksThrottleReasonApplicationsClocksSetting. Old name is deprecated and can be removed in one of the next major releases.
* - Added \ref nvmlDeviceGetDisplayActive and updated documentation to clarify how it differs from \ref nvmlDeviceGetDisplayMode
*
* \section changelog5 Changes between NVML v4.304 RC and v4.304 Production
*
* The following new functionality is exposed on NVIDIA display drivers version 304 Production or later
*
* - Added \ref nvmlDeviceGetGpuOperationMode and \ref nvmlDeviceSetGpuOperationMode
*
* \section changelog4 Changes between NVML v3.295 and v4.304 RC
*
* The following new functionality is exposed on NVIDIA display drivers version 304 RC or later
*
* - Added \ref nvmlDeviceGetInforomConfigurationChecksum and \ref nvmlDeviceValidateInforom
* - Added new error return value for initialization failure due to kernel module not receiving interrupts
* - Added \ref nvmlDeviceSetApplicationsClocks, \ref nvmlDeviceGetApplicationsClock, \ref nvmlDeviceResetApplicationsClocks
* - Added \ref nvmlDeviceGetSupportedMemoryClocks and \ref nvmlDeviceGetSupportedGraphicsClocks
* - Added \ref nvmlDeviceGetPowerManagementLimitConstraints, \ref nvmlDeviceGetPowerManagementDefaultLimit and \ref nvmlDeviceSetPowerManagementLimit
* - Added \ref nvmlDeviceGetInforomImageVersion
* - Expanded \ref nvmlDeviceGetUUID to support all CUDA capable GPUs
* - Deprecated \ref nvmlDeviceGetDetailedEccErrors in favor of \ref nvmlDeviceGetMemoryErrorCounter
* - Added \ref NVML_MEMORY_LOCATION_TEXTURE_MEMORY to support reporting of texture memory error counters
* - Added \ref nvmlDeviceGetCurrentClocksThrottleReasons and \ref nvmlDeviceGetSupportedClocksThrottleReasons
* - \ref NVML_CLOCK_SM is now also reported on supported Kepler devices.
* - Dropped support for GT200 based Tesla brand GPUs: C1060, M1060, S1070
*
* \section changelog3 Changes between NVML v2.285 and v3.295
*
* The following new functionality is exposed on NVIDIA display drivers version 295 or later
*
* - deprecated \ref nvmlDeviceGetHandleBySerial in favor of newly added \ref nvmlDeviceGetHandleByUUID
* - Marked the input parameters of \ref nvmlDeviceGetHandleBySerial, \ref nvmlDeviceGetHandleByUUID and \ref nvmlDeviceGetHandleByPciBusId_v2 as const
* - Added \ref nvmlDeviceOnSameBoard
* - Added \ref nvmlConstants defines
* - Added \ref nvmlDeviceGetMaxPcieLinkGeneration, \ref nvmlDeviceGetMaxPcieLinkWidth, \ref nvmlDeviceGetCurrPcieLinkGeneration,\ref nvmlDeviceGetCurrPcieLinkWidth
* - Format change of \ref nvmlDeviceGetUUID output to match the UUID standard. This function will return a different value.
* - \ref nvmlDeviceGetDetailedEccErrors will report zero for unsupported ECC error counters when a subset of ECC error counters are supported
* \section changelog1 Changes between NVML v1.0 and v2.285
*
* The following new functionality is exposed on NVIDIA display drivers version 285 or later
*
* - Added possibility to query separately current and pending driver model with \p nvmlDeviceGetDriverModel
* - Added API \ref nvmlDeviceGetVbiosVersion function to report VBIOS version.
* - Added pciSubSystemId to \ref nvmlPciInfo_t struct
* - Added API \ref nvmlErrorString function to convert error code to string
* - Updated docs to indicate we support M2075 and C2075
* - Added API \ref nvmlSystemGetHicVersion function to report HIC firmware version
* - Added NVML versioning support
* - Functions that changed API and/or size of structs have appended versioning suffix
* (e.g. nvmlDeviceGetPciInfo_v2). Appropriate C defines have been
* added that map old function names to the newer version of the function
* - Added support for concurrent library usage by multiple libraries
* - Added API \ref nvmlDeviceGetMaxClockInfo function for reporting device's clock limits
* - Added new error code NVML_ERROR_DRIVER_NOT_LOADED used by \ref nvmlInit_v2
* - Extended \ref nvmlPciInfo_t struct with new field: sub system id
* - Added NVML support on Windows guest account
* - Changed format of pciBusId string (to XXXX:XX:XX.X) of \ref nvmlPciInfo_t
* - Parsing of busId in \ref nvmlDeviceGetHandleByPciBusId_v2 is less restrictive. You can pass 0:2:0.0 or 0000:02:00 and other variations
* - Added API for events waiting for GPU events (Linux only) see docs of \ref nvmlEvents
* - Added API \p nvmlDeviceGetComputeRunningProcesses_v2 and \ref nvmlSystemGetProcessName functions for looking up currently running compute applications
* - Deprecated \ref nvmlDeviceGetPowerState in favor of \ref nvmlDeviceGetPerformanceState.
* - Added \ref NVML_FI_DEV_POWER_REQUESTED_LIMIT to report out the power limit requested by the client.
*/