GitHub

Essentials

Query the system, select a GPU and wait for GPU work. See Tasks and synchronization for execution ordering.

Versions and support

Metal.jl generates code for the newest Metal Shading Language (MSL), AIR and metallib versions the host macOS supports. See Apple's Detecting GPU features and Metal software versions.

Metal.versioninfo — Function
Metal.versioninfo(io::IO=stdout)

Print the versions of macOS, the toolchain and the Metal shading language, AIR and metallib versions Metal.jl generates code for, the relevant Julia packages, Metal-related environment variables and preferences, and the available GPUs. Include this when reporting issues.

source

Devices and global state

Every Julia task has a current device and, per device, a command queue that its GPU work is submitted to.

Metal.device — Function
device()::MTLDevice

Return the Metal GPU device associated with the current Julia task.

Since all M-series systems currently only externally show a single GPU, this function effectively returns the only system GPU.

source
device(<:MtlArray)

Get the Metal device for an MtlArray.

source
Metal.MTL.MTLDevice — Type
MTLDevice

A Metal GPU (MTLDevice), as returned by Metal.device() and Metal.devices(). Its properties are those of Apple's MTLDevice, e.g.:

  • dev.name: the name of the GPU, e.g., "Apple M1".
  • dev.maxThreadsPerThreadgroup: the maximum threadgroup size, per dimension.
  • dev.maxThreadgroupMemoryLength: the maximum amount of threadgroup memory, in bytes.
  • dev.recommendedMaxWorkingSetSize: the memory the GPU can use without affecting its performance, in bytes (see Metal.total_memory).
  • dev.currentAllocatedSize: the memory currently allocated on the GPU, in bytes.

MTL.supports_family(dev, MTL.MTLGPUFamilyApple9) checks whether the GPU belongs to a GPU family.

source
Metal.global_queue — Function
global_queue(dev::MTLDevice)::BatchedCommandQueue

Return the BatchedCommandQueue associated with the current Julia task.

This is a batched queue: kernel launches and blit operations accumulate into a single command buffer and are submitted lazily, rather than one command buffer per operation. It is a drop-in for a raw MTLCommandQueue — using it as one (e.g. to derive a command buffer, or for MPS) preserves program order by draining pending batches when command buffers are enqueued or committed. Call synchronize to wait for submitted work to finish. See BatchedCommandQueue for the full draining semantics.

source
Metal.BatchedCommandQueue — Type
BatchedCommandQueue

A command queue that batches GPU work to amortize Metal's per-command-buffer submission latency. Kernel launches (@metal) and GPU-side blit operations (copyto!, fill!) are encoded into a single open command buffer and committed lazily, instead of one command buffer per operation.

It wraps, and is a drop-in for, the MTLCommandQueue it batches: properties that aren't its own (e.g. label) forward to the underlying queue.

How the work is encoded and submitted is up to its CommandBackend type parameter: Metal3Backend using the Metal 3 command API, or Metal4Backend using the Metal 4 one, as selected by the command_api preference.

The open batch is committed ("flushed") when any of the following happens:

  • synchronize is called on the queue (or on one of its command buffers);
  • a command buffer derived from the queue is enqueued or committed;
  • command batching is disabled with the command_batching = false preference;
  • the batch reaches command_batching_ops() operations or command_batching_bytes() of blit traffic;
  • a GPU profiler is attached, in which case batching is disabled (each operation gets its own command buffer) so per-operation GPU timing is preserved;
  • an immediate submission is requested via @metal ... submit=true, or Metal.flush! is called explicitly.

Program order is preserved across flushes: command buffers execute in commit order and dispatches within an encoder run serially. At most command_batching_inflight() command buffers are kept in flight; further submissions block until the GPU drains one. Obtain the current task's batched queue with global_queue.

BatchedCommandQueues are task-local, but are only mutated while holding a global lock, so that other tasks can flush them too. Sharing a raw MTLCommandQueue across tasks is unsupported. Once the owning task has finished, any task's synchronize flushes and waits for the work it left behind, so results of a task are visible after waiting for it. device_synchronize flushes the batches of all tasks, which supports @async work and the REPL synchronization hook.

source

Synchronization

GPU work executes asynchronously. These functions wait for it to finish, and throw the errors it reported.

Metal.synchronize — Function
synchronize(queue=global_queue(device()))

Wait for currently committed GPU work on queue to finish. This includes work left behind by tasks that have finished, so that their results are visible after waiting for them.

source
synchronize(cmdbuf::MTLCommandBufferLike)

Wait for cmdbuf (which must already have been committed) and all preceding work on the same queue to complete.

source
Metal.@sync — Macro
Metal.@sync ex

Evaluate ex, then wait for the GPU work of the current task to finish, with synchronize, and return the value of ex. Use it, e.g., to time GPU work with @time or @btime.

source
Metal.KernelException — Type
KernelException

An exception thrown by a kernel, e.g., a BoundsError from an out-of-bounds access. Kernels cannot throw to the host while executing, so the first exception is recorded, and thrown by the next synchronize or device_synchronize after the kernel finished.

How much is reported depends on the kernel's debug_level (see @metal): at level 1, the default, the type of exception and its reason; at level 2, also the position of the thread that threw and a device-side backtrace.

source
Metal.MTL.CommandBufferError — Type
CommandBufferError

An error reported by one or more Metal command buffers at a synchronization boundary. The errors field contains the original NSError domain, code, and description, plus the command-buffer and queue labels when available.

source

KernelAbstractions