MPSKernel(3) MetalPerformanceShaders.framework MPSKernel(3)
NAME
MPSKernel
SYNOPSIS
#import <MPSKernel.h>
Inherits NSObject, <NSCopying>, and <NSSecureCoding>.
Inherited by MPSBinaryImageKernel, MPSCNNBinaryKernel, MPSCNNKernel, MPSImageCopyToMatrix, MPSImageFindKeypoints, MPSImageGuidedFilter,
MPSImageHistogram, MPSMatrixBinaryKernel, MPSMatrixCopy, MPSMatrixMultiplication, MPSMatrixSum, MPSMatrixUnaryKernel, MPSNNGraph,
MPSRNNMatrixInferenceLayer, and MPSUnaryImageKernel.
Instance Methods
(nonnull instancetype) - initWithDevice:
(nonnull instancetype) - copyWithZone:device:
(nullable instancetype) - initWithCoder:
(nullable instancetype) - initWithCoder:device:
() - MPSCopyAllocator
Public Attributes
const MTLRegion MPSRectNoClip
Properties
MPSKernelOptions options
id< MTLDevice > device
NSString * label
Detailed Description
MPSKernel.h MPSCore.framework
Copyright:
Copyright (c) 2015-2017 Apple Inc. All rights reserved.
MPSKernel objects encode tuned image processing operations into a MTLCommandBuffer.
This depends on Metal.framework The MPSKernel class is the base class for all MPS objects. It defines a standard interface for MPS
kernels. You should not use the MPSKernel class directly. Instead, a number of MPSKernel subclasses are available in
MetalPerformanceShaders.framework that define specific high-performance image processing operations.
The basic sequence for applying a MPSKernel to an image is as follows:
1. Create a MPSKernel corresponding to the operation you wish to perform:
MPSImageSobel *sobel = [[MPSImageSobel alloc] initWithDevice: mtlDevice];
2. Encode the filter into a command buffer:
sobel.offset = ...;
sobel.clipRect = ...;
sobel.options = ...;
[sobel encodeToCommandBuffer: commandBuffer
sourceTexture: inputImage
destinationTexture: resultImage ];
Encoding the kernel merely encodes the operation into a MTLCommandBuffer. It does not modify any pixels, yet. All MPSKernel state has been
copied to the command buffer. MPSKernels may be reused. If the texture was previously operated on by another command encoder (e.g.
MTLRenderCommandEncoder), you should call -endEncoding on the other encoder before encoding the filter.
Some MPS filters work in place (inputImage = resultImage) even in situations where Metal might not normally allow in place operation on
textures. If in-place operation is desired, you may attempt to call [MPSKernel encodeKernelInPlace...]. If the operation can not be
completed in place, then NO will be returned and you will have to create a new result texture and try again. To make an in-place image
filter reliable, pass a fallback MPSCopyAllocator to the method to create a new texture to write to in the event that a filter can not
operate in place.
(Repeat steps 2 for more filters, as desired.)
It should be self evident that step 2 may not be thread safe. That is, you can not have
multiple threads manipulating the same properties on the same MPSKernel object at the
same time and achieve coherent output. In common usage, the MPSKernel properties don't
often need to be changed from their default values, but if you need to apply the same
filter to multiple images on multiple threads with cropping / tiling, make additional
MPSKernel objects per thread. They are cheap. You can use multiple MPSKernel objects on
multiple threads, as long as only one thread is operating on any particular MPSKernel
object at a time.
3. After encoding any additional work to the command buffer using other encoders, submit the MTLCommandBuffer to your MTLCommandQueue,
using:
[mtlCommandBuffer commit];
A MPSKernel can be saved to disk / network using NSCoders such as NSKeyedArchiver. When decoding, the system default MTLDevice will be
chosen unless the NSCoder adopts the <MPSDeviceProvider> protocol. To accomplish this, subclass or extend your unarchiver to add this
method.
Method Documentation
- (nonnull instancetype) copyWithZone: (nullable NSZone *) zone(nullable id< MTLDevice >) device
Make a copy of this MPSKernel for a new device -copyWithZone: will call this API to make a copy of the MPSKernel on the same device. This
interface may also be called directly to make a copy of the MPSKernel on a new device. Typically, the same MPSKernels should not be used to
encode kernels on multiple command buffers from multiple threads. Many MPSKernels have mutable properties that might be changed by the
other thread while this one is trying to encode. If you need to use a MPSKernel from multiple threads make a copy of it for each additional
thread using -copyWithZone: or -copyWithZone:device:
Parameters:
zone The NSZone in which to allocate the object
device The device for the new MPSKernel. If nil, then use self.device.
Returns:
a pointer to a copy of this MPSKernel. This will fail, returning nil if the device is not supported. Devices must be
MTLFeatureSet_iOS_GPUFamily2_v1 or later.
Reimplemented in MPSRNNMatrixInferenceLayer, MPSRNNImageInferenceLayer, MPSMatrixNeuron, MPSMatrixFindTopK, MPSMatrixFullyConnected, and
MPSMatrixSoftMax.
- (nullable instancetype) initWithCoder: (NSCoder *__nonnull) aDecoder
Called by NSCoder to decode MPSKernels This isn't the right interface to decode a MPSKernel, but it is the one that NSCoder uses. To
enable your NSCoder (e.g. NSKeyedUnarchiver) to set which device to use extend the object to adopt the MPSDeviceProvider protocol.
Otherwise, the Metal system default device will be used.
- (nullable instancetype) initWithCoder: (NSCoder *__nonnull) aDecoder(nonnull id< MTLDevice >) device
NSSecureCoding compatability While the standard NSSecureCoding/NSCoding method -initWithCoder: should work, since the file can't know
which device your data is allocated on, we have to guess and may guess incorrectly. To avoid that problem, use initWithCoder:device
instead.
Parameters:
aDecoder The NSCoder subclass with your serialized MPSKernel
device The MTLDevice on which to make the MPSKernel
Returns:
A new MPSKernel object, or nil if failure.
Reimplemented in MPSCNNBinaryConvolution, MPSCNNBinaryFullyConnected, MPSCNNConvolutionTranspose, MPSCNNConvolutionGradient,
MPSCNNFullyConnected, MPSCNNFullyConnectedGradient, MPSCNNBinaryKernel, MPSCNNGradientKernel, MPSRNNMatrixInferenceLayer,
MPSCNNConvolution, MPSRNNImageInferenceLayer, MPSCNNPoolingAverageGradient, MPSCNNPoolingMaxGradient, MPSCNNPoolingL2NormGradient,
MPSCNNDilatedPoolingMaxGradient, MPSCNNSoftMaxGradient, MPSCNNLogSoftMaxGradient, MPSCNNCrossChannelNormalizationGradient, MPSCNNLoss,
MPSCNNPoolingGradient, MPSCNNCrossChannelNormalization, MPSImagePyramid, MPSBinaryImageKernel, MPSCNNLocalContrastNormalizationGradient,
MPSCNNDilatedPoolingMax, MPSImageSobel, MPSCNNNeuronGradient, MPSCNNPoolingAverage, MPSCNNPoolingL2Norm, MPSCNNLocalContrastNormalization,
MPSMatrixNeuron, MPSCNNKernel, MPSImageHistogramSpecification, MPSImageThresholdToZeroInverse, MPSCNNBatchNormalization,
MPSCNNBatchNormalizationStatistics, MPSCNNNeuron, MPSImageThresholdToZero, MPSImageHistogramEqualization, MPSCNNInstanceNormalization,
MPSCNNDropoutGradient, MPSUnaryImageKernel, MPSImageBox, MPSImageGaussianBlur, MPSCNNSpatialNormalizationGradient, MPSMatrixCopy,
MPSImageStatisticsMean, MPSImageThresholdBinary, MPSImageThresholdTruncate, MPSMatrixSum, MPSMatrixFindTopK, MPSImageDilate, MPSCNNDropout,
MPSMatrixFullyConnected, MPSImageScale, MPSImageLanczosScale, MPSImageBilinearScale, MPSImageStatisticsMeanAndVariance,
MPSImageConvolution, MPSMatrixSoftMax, MPSImageThresholdBinaryInverse, MPSImageHistogram, MPSImageGuidedFilter, MPSCNNSpatialNormalization,
MPSImageCopyToMatrix, MPSImageEuclideanDistanceTransform, MPSImageFindKeypoints, MPSImageStatisticsMinAndMax, MPSImageMedian,
MPSCNNPooling, MPSCNNPoolingMax, MPSImageAreaMax, and MPSNNGraph.
- (nonnull instancetype) initWithDevice: (nonnull id< MTLDevice >) device
Standard init with default properties per filter type
Parameters:
device The device that the filter will be used on. May not be NULL.
Returns:
a pointer to the newly initialized object. This will fail, returning nil if the device is not supported. Devices must be
MTLFeatureSet_iOS_GPUFamily2_v1 or later.
Reimplemented in MPSCNNBinaryConvolution, MPSCNNBinaryFullyConnected, MPSCNNConvolutionTranspose, MPSCNNConvolutionGradient,
MPSCNNFullyConnected, MPSCNNFullyConnectedGradient, MPSCNNGradientKernel, MPSRNNMatrixInferenceLayer, MPSCNNConvolution,
MPSRNNImageInferenceLayer, MPSCNNSoftMaxGradient, MPSCNNLogSoftMaxGradient, MPSCNNLoss, MPSCNNPoolingGradient,
MPSCNNCrossChannelNormalization, MPSBinaryImageKernel, MPSNNReduceFeatureChannelsAndWeightsSum, MPSNNReshape, MPSNNSlice, MPSImagePyramid,
MPSCNNArithmeticGradient, MPSCNNAddGradient, MPSCNNSubtractGradient, MPSCNNMultiplyGradient, MPSNNReduceBinary,
MPSNNReduceFeatureChannelsAndWeightsMean, MPSImageSobel, MPSCNNNeuronGradient, MPSCNNNeuronLinear, MPSCNNNeuronReLU, MPSCNNNeuronPReLU,
MPSCNNNeuronSigmoid, MPSCNNNeuronHardSigmoid, MPSCNNNeuronTanH, MPSCNNNeuronAbsolute, MPSCNNNeuronSoftPlus, MPSCNNNeuronSoftSign,
MPSCNNNeuronELU, MPSCNNNeuronReLUN, MPSCNNNeuronPower, MPSCNNNeuronExponential, MPSCNNNeuronLogarithm, MPSNNReduceFeatureChannelsSum,
MPSCNNLocalContrastNormalization, MPSMatrixNeuron, MPSMatrixSum, MPSCNNBinaryKernel, MPSImageThresholdToZeroInverse,
MPSCNNBatchNormalization, MPSCNNBatchNormalizationStatistics, MPSCNNNeuron, MPSImageThresholdToZero, MPSCNNInstanceNormalization,
MPSCNNKernel, MPSCNNDropoutGradient, MPSCNNUpsamplingGradient, MPSUnaryImageKernel, MPSImageBox, MPSImageGaussianBlur,
MPSImageStatisticsMean, MPSImageThresholdBinary, MPSImageThresholdTruncate, MPSCNNArithmetic, MPSCNNAdd, MPSCNNSubtract, MPSCNNMultiply,
MPSCNNDivide, MPSMatrixFindTopK, MPSImageDilate, MPSCNNDropout, MPSMatrixFullyConnected, MPSImageLanczosScale, MPSImageBilinearScale,
MPSImageArithmetic, MPSImageAdd, MPSImageSubtract, MPSImageMultiply, MPSImageDivide, MPSImageStatisticsMeanAndVariance,
MPSMatrixMultiplication, MPSMatrixVectorMultiplication, MPSMatrixSoftMax, MPSImageThresholdBinaryInverse, MPSImageGuidedFilter,
MPSCNNSpatialNormalization, MPSImageEuclideanDistanceTransform, MPSImageFindKeypoints, MPSCNNUpsampling, MPSNNReduceUnary,
MPSNNReduceRowMin, MPSNNReduceColumnMin, MPSNNReduceFeatureChannelsMin, MPSNNReduceRowMax, MPSNNReduceColumnMax,
MPSNNReduceFeatureChannelsMax, MPSNNReduceRowMean, MPSNNReduceColumnMean, MPSNNReduceFeatureChannelsMean, MPSNNReduceRowSum,
MPSNNReduceColumnSum, MPSImageReduceUnary, MPSImageReduceRowMin, MPSImageReduceColumnMin, MPSImageReduceRowMax, MPSImageReduceColumnMax,
MPSImageReduceRowMean, MPSImageReduceColumnMean, MPSImageReduceRowSum, MPSImageReduceColumnSum, MPSImageScale, MPSImageStatisticsMinAndMax,
MPSImageMedian, MPSCNNPooling, MPSImageAreaMax, and MPSMatrixCopy.
- MPSCopyAllocator
MPSImageKernel.h MetalPerformanceShaders.framework
Copyright:
Copyright (c) 2015 Apple Inc. All rights reserved. MetalPerformanceShaders filter base classes
A block to make a copy of sourceTexture for MPSKernels that can only execute out of place. Some MPSKernel objects may not be able to
operate in place. When that occurs, and in-place operation is requested, MPS will call back to this block to get a new texture to return
instead. To avoid spending long periods of time allocating pages to back the MTLTexture, the block should attempt to reuse textures. The
texture returned from the MPSCopyAllocator will be returned instead of the sourceTexture from the MPSKernel method on return.
// A MPSCopyAllocator to handle cases where in-place operation fails.
MPSCopyAllocator myAllocator = ^id <MTLTexture>( MPSKernel * __nonnull filter,
__nonnull id <MTLCommandBuffer> cmdBuf,
__nonnull id <MTLTexture> sourceTexture)
{
MTLPixelFormat format = sourceTexture.pixelFormat; // FIXME: is this format writable?
MTLTextureDescriptor *d = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat: format
width: sourceTexture.width
height: sourceTexture.height
mipmapped: NO];
d.usage = MTLTextureUsageShaderRead | MTLTextureUsageShaderWrite;
//FIXME: Allocating a new texture each time is slow. They take up to 1 ms each.
// There are not too many milliseconds in a video frame! You can recycle
// old textures (or MTLBuffers and make textures from them) and reuse
// the memory here.
id <MTLTexture> result = [cmdBuf.device newTextureWithDescriptor: d];
// FIXME: If there is any metadata associated with sourceTexture such as colorspace
// information, MTLResource.label, MTLResource.cpuCacheMode mode,
// MTLResource.MTLPurgeableState, etc., it may need to be similarly associated
// with the new texture to avoid losing your metadata.
// FIXME: If filter.clipRect doesn't cover the entire image, you may need to copy
// pixels from sourceTexture to the new texture or regions of the new texture
// will be uninitialized. You can make a MTLCommandEncoder to encode work on
// the MTLCommandBuffer here to do that work, if necessary. It will be
// scheduled to run immediately before the MPSKernel work. Do not call
// [MTLCommandBuffer enqueue/commit/waitUntilCompleted/waitUntilScheduled]
// in the MPSCopyAllocator block. Make sure to call -endEncoding on the
// MTLCommandEncoder so that the MTLCommandBuffer has no active encoder
// before returning.
// CAUTION: The next command placed on the MTLCommandBuffer after the MPSCopyAllocator
// returns is almost assuredly going to be encoded with a MTLComputeCommandEncoder.
// Creating any other type of encoder in the MPSCopyAllocator will probably cost
// an additional 0.5 ms of both CPU _AND_ GPU time (or more!) due to a double
// mode switch penalty.
// CAUTION: If other objects (in addition to the caller of -encodeToCommandBuffer:inPlaceTexture:...)
// own a reference to sourceTexture, they may need to be notified that
// sourceTexture has been replaced so that they can release that resource
// and adopt the new texture.
// The reference to sourceTexture owned by the caller of
// -encodeToCommandBuffer:inPlaceTexture... will be released by
// -encodeToCommandBuffer:inPlaceTexture:... after the kernel is encoded if
// and only if the MPSCopyAllocator is called, and the operation is successfully
// encoded out of place.
return result;
// d is autoreleased
};
If nil is returned by the allocator, NO will be returned by the calling function.
When the MPSCopyAllocator is called, no MTLCommandEncoder is active on the commandBuffer. You may create a MTLCommandEncoder in the block
to initialize the texture. Make sure to call -endEncoding on it before returning, if you do.
Parameters:
filter A valid pointer to the MPSKernel that is calling the MPSCopyAllocator. From it you can get the clipRect of the intended
operation.
commandBuffer A valid MTLCommandBuffer. It can be used to obtain the device against which to allocate the new texture. You may also
enqueue operations on the commandBuffer to initialize the texture on a encoder allocated in the block. You may not submit, enqueue or
wait for scheduling/completion of the command buffer.
sourceTexture The texture that is providing the source image for the filter. You may wish to use its size and MTLPixelFormat for the
new texture, but it is not requred.
Returns:
A new valid MTLTexture to use as the destination for the MPSKernel. If the calling function succeeds, its texture parameter will be
overwritten with a pointer to this texture. If the calling function fails (highly unlikely, except for user error) then the texture
will be released before the calling function returns.
Member Data Documentation
- (const MTLRegion) MPSRectNoClip
MPSRectNoClip This is a special constant to indicate no clipping is to be done. The entire image will be used. This is the default
clipping rectangle or the input extent for MPSKernels.
Property Documentation
- device [read], [nonatomic], [retain]
The device on which the kernel will be used
- label [read], [write], [atomic], [copy]
A string to help identify this object.
- options [read], [write], [nonatomic], [assign]
The set of options used to run the kernel. MPSKernelOptions
Author
Generated automatically by Doxygen for MetalPerformanceShaders.framework from the source code.
Version MetalPerformanceShaders-100 Thu Feb 8 2018 MPSKernel(3)