Tootfinder

Adaptive Request Scheduling for CodeLLM Serving with SLA Guarantees
Shi Chang, Boyuan Chen, Kishanthan Thangarajah, Hanan Lutfiyya, Ahmed E. Hassan
https://arxiv.org/abs/2506.19677

Adaptive Request Scheduling for CodeLLM Serving with SLA Guarantees
Code Large Language Models (CodeLLMs) are increasingly integrated into modern software development workflows, yet efficiently serving them in resource-constrained, self-hosted environments remains a significant challenge. Existing LLM serving systems employs Continuous Batching for throughput improvement. However, they rely on static batch size configurations that cannot adapt to fluctuating request rates or heterogeneous workloads, leading to frequent SLA (Service Level Agreement) violations a…

On the Batch Size Selection in Stochastic Gradient Methods Using No-Replacement Sampling
Marco Boresta, Alberto De Santis, Stefano Lucidi
https://arxiv.org/abs/2506.08758

On the Batch Size Selection in Stochastic Gradient Methods Using No-Replacement Sampling
Recent stochastic gradient methods that have appeared in the literature base their efficiency and global convergence properties on a suitable control of the variance of the gradient batch estimate. This control is typically achieved by dynamically increasing the batch size during the iterations of the algorithm. However, in the existing methods the statistical analysis often relies on sampling with replacement. This particular batch selection appears unrealistic in practice. In this paper, we c…

This https://arxiv.org/abs/2505.14884 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

Polar Sparsity: High Throughput Batched LLM Inferencing with Scalable Contextual Sparsity
Accelerating large language model (LLM) inference is critical for real-world deployments requiring high throughput and low latency. Contextual sparsity, where each token dynamically activates only a small subset of the model parameters, shows promise but does not scale to large batch sizes due to union of active neurons quickly approaching dense computation. We introduce Polar Sparsity, highlighting a key shift in sparsity importance from MLP to Attention layers as we scale batch size and seque…

ASMOP: Additional sampling stochastic trust region method for multi-objective problems
Nata\v{s}a Krklec Jerinki\'c, Luka Rute\v{s}i\'c
https://arxiv.org/abs/2506.10976

ASMOP: Additional sampling stochastic trust region method for multi-objective problems
We consider an unconstrained multi-criteria optimization problem with finite sum objective functions. The proposed algorithm belongs to a non-monotone trust-region framework where additional sampling approach is used to govern the sample size and the acceptance of a candidate point. Depending on the problem, the method can result in a mini-batch or an increasing sample size approach. Therefore, this work can be viewed as an extension of additional sampling trust region method for scalar finite …

AI Accelerators for Large Language Model In-ference: Architecture Analysis and Scaling Strategies
Amit Sharma
https://arxiv.org/abs/2506.00008 https://

AI Accelerators for Large Language Model In-ference: Architecture Analysis and Scaling Strategies
The rapid growth of large-language models (LLMs) is driving a new wave of specialized hardware for inference. This paper presents the first workload-centric, cross-architectural performance study of commercial AI accelerators, spanning GPU-based chips, hybrid packages, and wafer-scale engines. We compare memory hierarchies, compute fabrics, and on-chip interconnects, and observe up to 3.7x performance variation across architectures as batch size and sequence length change. Four scaling techniqu…

This https://arxiv.org/abs/2401.06738 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum
Stochastic heavy ball momentum (SHB) is commonly used to train machine learning models, and often provides empirical improvements over stochastic gradient descent. By primarily focusing on strongly-convex quadratics, we aim to better understand the theoretical advantage of SHB and subsequently improve the method. For strongly-convex quadratics, Kidambi et al. (2018) show that SHB (with a mini-batch of size $1$) cannot attain accelerated convergence, and hence has no theoretical benefit over SGD…