Sortblock: Similarity-Aware Feature Reuse for Diffusion Model

Abstract

Diffusion Transformers (DiTs) have demonstrated remarkable generative capabilities, particularly benefiting from Transformer architectures that enhance visual and artistic fidelity. However, their inherently sequential denoising process results in high inference latency, limiting their deployment in real-time scenarios. Existing training-free acceleration approaches typically reuse intermediate features at fixed timesteps or layers, overlooking the evolving semantic focus across denoising stages and Transformer blocks.To address this, we propose Sortblock, a training-free inference acceleration framework that dynamically caches block-wise features based on their similarity across adjacent timesteps. By ranking the evolution of residuals, Sortblock adaptively determines a recomputation ratio, selectively skipping redundant computations while preserving generation quality. Furthermore, we incorporate a lightweight linear prediction mechanism to reduce accumulated errors in skipped blocks.Extensive experiments across various tasks and DiT architectures demonstrate that Sortblock achieves over 200% inference speedup with minimal degradation in output quality, offering an effective and generalizable solution for accelerating diffusion-based generative models.

Similarity-Aware Feature Reuse (Sortblock)

Methodology

To leverage this sparsity, Sortblock selectively updates only a small subset of DiT blocks that exhibit the most significant changes between adjacent steps. The challenge lies in efficiently and accurately identifying these DiT blocks. As shown in Fig. 2, we compute the cosine similarity between the input-output changes of DiT blocks at step K and step K+1. By utilizing the cosine similarity between the input-output changes of each DiT block, we identify those DiT blocks with the most substantial changes. Only these selected DiT blocks undergo complete feature recomputation and cache updates.

Figure 2: The Sortblock pipeline. Block features are adaptively updated based on their similarity between fully computed and predicted counterparts. Specifically, blocks with low similarity are recomputed, while the others are efficiently approximated via prediction.

Evaluations

Quantitative Results

Visual Results

Figure 3: Qualitative Results. Sortblock delivers high fidelity and maintains consistency with original images.

Figure 4: Ablation study on the effect of K.

Figure 5: Effect of using Sortblock at different denoising stages.

Figure 6: Impact of different β.

Wan2.1, 86s

TeaCache, 44s

Sortblock, 38s

Wan2.1, 81-frame, 832x480.

HunyuanVideo, 53s

TeaCache, 22s

Sortblock, 22s

HunyuanVideo, 61-frame, 512x320.

BibTeX


      @misc{chen2025sortblocksimilarityawarefeaturereuse,
        title={Sortblock: Similarity-Aware Feature Reuse for Diffusion Model}, 
        author={Hanqi Chen and Xu Zhang and Xiaoliu Guan and Lielin Jiang and Guanzhong Wang and Zeyu Chen and Yi Liu},
        year={2025},
        eprint={2508.00412},
        archivePrefix={arXiv},
        primaryClass={cs.CV},
        url={https://arxiv.org/abs/2508.00412}, 
  }