Skip to main content
←API Reference ←SwiftyNsdk

NsdkSemanticsSession

A session for semantic segmentation and environmental understanding....

Declaration​

final class NsdkSemanticsSession

Summary​

A session for semantic segmentation and environmental understanding. NsdkSemanticsSession provides capabilities for understanding the semantic structure of the environment by classifying pixels into different object categories. This enables applications to make intelligent decisions based on environmental context.

Overview​

Semantics features include:

  • Real-time semantic segmentation of camera images
  • Multiple semantic categories (sky, ground, buildings, people, etc.)
  • Confidence maps for semantic classifications
  • Packed channel data for efficient processing
  • Suppression masks for filtering unwanted areas

Usage Pattern​

// Create and configure semantics session
let semanticsSession = nsdkSession.createSemanticsSession()
let config = Configuration()
semanticsSession.configure(with: config)
semanticsSession.start()
// Get available semantic channels
let (error, channelNames) = semanticsSession.getChannelNames()
if error == .none, let names = channelNames {
    print("Available channels: \(names)")
}
// Get semantic confidence for a specific channel
let (status, result) = semanticsSession.getLatestConfidence(channelIndex: 0)
if status.isOk(), let semanticResult = result {
    // Process semantic data
    processSemanticData(semanticResult)
}

Methods​

NameTypeSummary
configurevoid
Configures the session with the specified settings.
- Attention: This method must be called while the session is stopped,
or else configuration will fail. In that case, while this function returns without
throwing, configuration will still fail asynchronously. Use featureStatus()
to check that configuration has not failed.
- Parameter config: An object that defines this session's behavior.
Only settings that differ from the defaults will be applied.
- Throws: NsdkError.invalidArgument if the configuration is invalid.
Check NSDK's C logs for more information.
featureStatusNsdkFeatureStatus
Gets the current status of the Semantics feature.
This method reports any errors or warnings that have occurred within the semantics system.
Check this periodically to monitor the health of semantic processing operations.
- Returns: Feature status flags indicating current state and any issues
## Example
let status = semanticsSession.getFeatureStatus()
if status.contains(.failed) {
    print("Semantics has encountered an error")
}
latestConfidenceNsdkAsyncState<SemanticsResult, AwarenessError>
Retrieves the latest confidence map for a specific semantic channel.
This method returns a confidence map where each pixel value represents the confidence
score (0.0-1.0) that the pixel belongs to the specified semantic category. Higher
confidence values indicate stronger belief in the semantic classification.
- Parameter channel: The name of the semantic channel to retrieve
- Returns: A tuple containing the operation status and semantic result if successful
## Example
// Get confidence for "sky" channel (assuming it's at index 0)
let (status, result) = semanticsSession.getLatestConfidence(channelIndex: 0)
if status.isOk(), let semanticResult = result {
    print("Sky confidence image size: \(semanticResult.image?.width ?? 0) x \(semanticResult.image?.height ?? 0)")
    // Process confidence data
    if let image = semanticResult.image {
        processConfidenceMap(image, forChannel: "sky")
    }
}
## Confidence Interpretation
Confidence values range from 0.0 to 1.0:
- 0.0: Definitely not the specified semantic category
- 0.5: Uncertain classification
- 1.0: Definitely the specified semantic category
Use confidence thresholds to filter results based on your application's needs.
latestImageParamsNsdkAsyncState<AwarenessImageParams, AwarenessError>
Retrieves the latest camera intrinsic parameters for semantic processing.
This method returns the camera intrinsic parameters that were used during semantic
processing. These parameters are essential for coordinate transformations between
image coordinates and 3D world coordinates.
- Returns: A tuple containing the operation status and intrinsics result
latestPackedChannelsNsdkAsyncState<SemanticsResult, AwarenessError>
Retrieves the latest packed semantic channels data.
This method returns a multi-channel image where each channel represents a different
semantic category. Packed channels provide an efficient way to access multiple
semantic classifications in a single image, reducing the need for multiple API calls.
- Returns: A tuple containing the operation status and semantic result if successful
## Example
let (status, result) = semanticsSession.getLatestPackedChannels()
if status.isOk(), let semanticResult = result {
    print("Packed channels image size: \(semanticResult.image?.width ?? 0) x \(semanticResult.image?.height ?? 0)")
    // Process packed semantic data
    if let image = semanticResult.image {
        processPackedSemanticChannels(image)
    }
}
## Packed Channels Format
The packed channels image contains multiple semantic categories encoded as separate
channels in a single image. Each channel corresponds to a semantic category, and
pixel values represent classification confidence or probability scores.
## Performance Benefits
Using packed channels is more efficient than calling getLatestConfidence multiple
times, as it reduces the number of API calls and data transfers required.
latestSuppressionMaskNsdkAsyncState<SemanticsResult, AwarenessError>
Retrieves the latest suppression mask for semantic processing.
This method returns a binary mask indicating areas that should be ignored or suppressed
during semantic processing. Suppression masks are useful for filtering out regions
that are not relevant for semantic understanding, such as areas with poor image quality.
- Returns: A tuple containing the operation status and semantic result if successful
## Example
let (status, result) = semanticsSession.getLatestSuppressionMask()
if status.isOk(), let semanticResult = result {
    print("Suppression mask image size: \(semanticResult.image?.width ?? 0) x \(semanticResult.image?.height ?? 0)")
    // Apply suppression mask to filter semantic results
    if let mask = semanticResult.image {
        applySuppressionMask(mask, toSemanticResults: otherResults)
    }
}
## Suppression Mask Usage
Suppression masks are binary images where:
- 0: Areas to be suppressed (ignored in semantic processing)
- 1: Areas to be processed normally
Use suppression masks to improve semantic processing quality by excluding
problematic regions from analysis.
startvoid
Starts the Semantics system.
After starting, Semantics will begin processing incoming frame data for semantic segmentation.
The system must be configured before starting.
## Example
semanticsSession.configure(with: config)
semanticsSession.start()
stopvoid
Stops the Semantics system.
This halts all semantic processing. The session can be reconfigured and restarted after stopping.
unpackChannelsFromBitmaskSemanticsChannels
Unpacks semantic channels from a packed channel bitmask.
This method converts a bitmask value from a packed channel pixel into a SemanticsChannels
OptionSet, where each bit represents a semantic channel (bit 0 = Sky, bit 1 = Ground, etc.).
- Parameter bitmask: The value of a pixel from an image returned by latestPackedChannels()
- Returns: A SemanticsChannels OptionSet representing the channels present in the bitmask

Nested Types​

Structs​

NameTypeSummary
ConfigurationConfiguration
Configuration structure for the semantics session.

Enums​

NameTypeSummary
SemanticsModeSemanticsMode
Options for different semantics modes.
These trade off between performance and accuracy.

Relationships​

conforms to: NsdkFeatureSession
ON THIS PAGE