GigaAM MLX port for Mac

The large_ctc version of GigaAM-Multilingual was ported from PyTorch to MLX for native inference on Apple Silicon. The port provides four weight variants: FP16, INT8, INT6, and INT4.

The comparison covers storage, memory use, transcription speed, and the places where lower precision changes the output. The first audio sample contains the only difference between INT8 and FP16 across 255 Russian recordings.

Key results

How weight precision changes the transcript

INT8 changed one of 255 transcripts relative to FP16, INT6 changed seven, and INT4 changed 31. These counts show differences from FP16, so they do not always indicate an error against the reference. The two examples below show both cases: a spelling change in a common word and different Cyrillic renderings of a band name.

What is inside the model

GigaAM-Multilingual supports Russian, Kazakh, Kyrgyz, Uzbek, and English. The published large_ctc variant was selected for the port. It has about 600 million parameters and 24 Conformer blocks. These blocks find relationships in the audio. CTC, or Connectionist Temporal Classification, then turns their output into a character sequence without requiring a predefined alignment between every sound and point in time.

MLX runs computations on the CPU and GPU of Apple Silicon. Both processors use the Mac’s unified memory. To preserve the behavior of the original model, the MLX port reproduces the complete signal path: feature extraction, multidimensional array layouts, position handling, masking, and conversion of model output into text.

A working port had to satisfy four conditions:

1. regular transcription requires only MLX and an audio-reading library;
2. FP32 and FP16 reproduce the original model within an acceptable numerical tolerance;
3. every variant runs on the same public evaluation inputs;
4. speed, file size, and memory are measured separately on identical recordings.

The work is based on the official ai-sage/GigaAM-Multilingual model and the salute-developers/GigaAM source code. The completed MLX version is published in ai-babai/gigaam-multilingual-mlx.

How parity with the original model was verified

The converter reads the official weight file and saves it in the safer safetensors format. Transcription itself runs on MLX. PyTorch is needed only for conversion and implementation comparisons.

Intermediate tensors, the multidimensional arrays inside the model, were compared first. Checks were placed after every major stage. This makes it possible to locate a divergence immediately, even when the final transcript happens to match.

Quality evaluation uses pinned public sample lists and one text-normalization procedure: lowercasing, punctuation removal, and the same remaining transformations for every model. The original model, FP16, and quantized variants receive identical audio and reference transcripts. The first Russian suite contains 1,533 recordings from FLEURS, Russian LibriSpeech, and SOVA. The final multilingual-v1 suite uses 1,000 pinned FLEURS samples for each of Russian, Kazakh, Kyrgyz, Uzbek, and English.

The benchmark ran on a MacBook Pro with M4 Pro and 48 GB of unified memory under macOS 15.7.7. The original PyTorch implementation used Apple’s MPS acceleration with FP16 autocast. A five-minute Russian WAV was processed after model loading and a short warm-up. Every variant ran in a separate process.

WER measures substituted, deleted, and inserted words. A value near 5% can be read as roughly five corrections per hundred words. The full report calls the average across Russian, Kazakh, Kyrgyz, and Uzbek Core macro WER.

The source JSON also records CER, or Character Error Rate, and RTF, or Real-Time Factor. RTF is processing time divided by audio duration. An RTF below 1 means the model runs faster than real time. Lower is better in every numeric column below.

How precision affects size and speed

The official pytorch_model.bin stores all 824 tensors in FP32 and occupies 2.342 GB. Conversion to MLX changes the weights to FP16 and reduces the file to 1.171 GB. INT8, INT6, and INT4 are produced from MLX FP16 and use fewer bits. The 2.342 GB value describes the source weight file on disk. The PyTorch/MPS speed measurement used FP16 autocast for supported computations.

During quantization, each group of 64 weights receives a shared scale used to approximate the original values. The g64 label means that the group contains 64 weights. Every published variant has its own weights, description, checksum, and pinned revision.

Peak RSS, or Resident Set Size, is the maximum amount of physical memory occupied by the process. It includes the model, libraries, and working buffers.

Chart and table data: complete benchmark JSON and compact CSV for the five variants.

Speed relative to real time

GigaAM performs ASR, or automatic speech recognition. ASR systems commonly report speed with RTF: processing time divided by audio duration. A lower RTF means a faster model. Its reciprocal shows how many times faster than real time the model runs.

Speed table data: JSON with complete measurements and CSV with RTF and processing speed.

GPU memory is tracked separately from RSS because the CPU and GPU on Apple Silicon share one unified memory pool.

The table points to three practical choices:

• FP16 had the lowest latency on the five-minute input;
• INT8 preserved nearly the same quality and reduced peak RSS by 82.7% relative to the original implementation;
• INT4 reduced the weight file by 80.9% relative to the original and used the least memory, while running slower than FP16 and INT8.

A smaller file does not guarantee higher speed. Packed integer values must be unpacked and converted into a suitable compute format before multiplication. On M4 Pro, this extra work made FP16 slightly faster than INT8, INT6, and INT4. Speed was therefore measured separately for every variant.

The public benchmark report contains the complete methodology, confidence intervals, and per-language results.

Where the time goes

Each stage was timed separately on 20 seconds of audio to find the bottleneck. The recording becomes 1,999 short spectrogram frames. Their count is then reduced to about 500, and the 24 Conformer blocks perform nearly all remaining work. Measurements used the same M4 Pro with MLX 0.32.0. After every stage, mx.eval() forced MLX to finish computation before the timer was read.

Audio preparation, the CTC output layer, and conversion to text together occupy too little time to provide a large end-to-end gain. The main computation is concentrated inside the 24 identical Conformer blocks.

The profile explains the optimization priorities. Of 585.33 million parameters, about 402.90 million are in the FFNs. Attention projections account for 17.2%, and Conformer convolution modules account for another 12.9%.

The standard attention calculation was replaced with the built-in mx.fast.scaled_dot_product_attention operation. The full pass became 3.8% faster in FP16 and 4.4% faster in INT8. Attention occupies about a quarter of one block’s time, so accelerating only this component produces a gain of several percent for the entire model.

Which optimizations actually made the model faster

Candidates were tested one at a time in separate processes. The published code and weights remained unchanged during the investigation.

In the first measurement, the model received 16 chunks of 20 seconds each. They were processed in two groups of eight. Total time fell from 1.91 to 1.72 seconds, so the model processed 10.9% more audio per unit of time. Per-file latency was not evaluated in this test.

The gains cannot be added arithmetically. They use some of the same resources and alter the shape of the computation graph.

Chunk length must be tuned for each weight variant. FP16 gained 8.2% when the chunk grew from 20 to 40 seconds. INT8 lost 10.2% with the same shape. Processing eight chunks at once accelerated FP16 by 10.9% and slowed INT8 by 1.9%. One shared setting would have made the default variant worse.

Tests of mx.compile() showed that the effect depends on input size. Compilation produced +0.8% for FP16 and +3.1% for INT8. The arbitrary-input-size mode (shapeless=True) reached a technical limitation in mx.as_strided during audio preparation.

Why parts of the model cannot simply be removed

Temporarily disabling or replacing part of a model is called an ablation. The experiment reveals how that component contributes to speed and quality. For example, the simpler ReLU function barely accelerated a complete feed-forward block: 1.51 ms instead of 1.52 ms with SiLU. Most time is spent multiplying large matrices, so replacing one small operation changes very little.

Diagnostic ablations were then run on 100 public FLEURS samples without saving new weights. The original configuration had a WER of 6.51% on this subset.

WER can exceed 100% when a model inserts many extra words. These tests used existing weights after a mechanical change to the model. Every accelerated variant severely degraded recognition.

Even replacing the activation function changed the signals flowing between layers. Removing half of the blocks accelerated computation by 1.50×, while recognition stopped working. Changes of this scale require retraining or distilling knowledge from the large model into a smaller one.

The limit of acceleration without training

Optimizations without training produced gains from +0.8% to +10.9%, depending on the mode. Removing model components reached 1.50×, while recognition quality fell sharply. Only variants that preserved quality were included in the final results.

The workflow

The work was divided into three consecutive tracks:

The finished result is available on GitHub, PyPI, and in a Hugging Face Collection. INT8 g64 is the default. FP16 provides the lowest measured latency, while INT4 provides the smallest size and memory footprint.

This text was prepared by the Codex AI agent from materials produced during the working session.

Reproducibility

• Code and release notes: ai-babai/gigaam-multilingual-mlx
• Weights on Hugging Face: FP16, INT8 g64, INT6 g64, INT4 g64, and the complete collection
• Public methodology: benchmarks/multilingual-v1/README.md
• Complete report: docs/benchmark-multilingual-v1.md
• Original model: ai-sage/GigaAM-Multilingual
• Original code: salute-developers/GigaAM
• MLX compilation: official documentation
• Fast SDPA in MLX: official documentation
• Fast Conformer: paper on arXiv

Every published benchmark input is publicly available. Private recordings, model weights, datasets, caches, and large raw results are not stored in the GitHub repository.