Tootfinder

DSDE: Dynamic Speculative Decoding with KLD Stability for Real-World Serving
Mingyu Yang, Jae-Young Choi, Kihyo Moon, Minsung Jang, Eunjoo Joen
https://arxiv.org/abs/2509.01083 …

DSDE: Dynamic Speculative Decoding with KLD Stability for Real-World Serving
Speculative decoding accelerates large language model inference, but its reliance on a fixed speculation length is suboptimal in large-batch serving environments with diverse requests. This paper explores a new direction for dynamic adaptation by investigating a novel class of post-hoc, diagnostic signals. We propose Dynamic Speculative Decoding Engine (DSDE), a training-free framework built on two primary components: (1) a predictive signal based on the variance of the Kullback-Leibler (KLD) d…

AdaptCache: KV Cache Native Storage Hierarchy for Low-Delay and High-Quality Language Model Serving
Shaoting Feng, Hanchen Li, Kuntai Du, Zhuohan Gu, Yuhan Liu, Jiayi Yao, Siddhant Ray, Samuel Shen, Yihua Cheng, Ganesh Ananthanarayanan, Junchen Jiang
https://arxiv.org/abs/2509.00105

AdaptCache: KV Cache Native Storage Hierarchy for Low-Delay and High-Quality Language Model Serving
Large language model (LLM) applications often reuse previously processed context, such as chat history and documents, which introduces significant redundant computation. Existing LLM serving systems address such redundant computation by storing the KV caches of processed context and loading the corresponding KV cache when a new request reuses the context. Further, as these LLM applications scale, the total size of KV caches becomes excessively large and requires both DRAM and SSD for full stora…

The Senate confirms former Fox News host and prosecutor Jeanine Pirro as US attorney for DC; she had been serving in the role on an interim basis since May (Nnamdi Egwuonwu/NBC News)
https://www.nbcnews.com/politics/congress/

Senate confirms former Fox News host Jeanine Pirro as U.S. Attorney for the District of Columbia
The Senate voted on Saturday to confirm former Fox News host and prosecutor Jeanine Pirro as U.S.

35-year-old former U.S. Army sergeant, Bajun “Baji” Mavalwalla II,
faces up to six years in prison
for protesting against ICE deportations
in what legal experts are calling a test case for the
Trump administration’s attempts to criminalize and punish dissent.
Mavalwalla was arrested and charged with “conspiracy to impede or injure officers”
after he was identified in a video taken at the protest and shared on Instagram.
Mavalwalla, who survived a ro…

Army Vet Charged with “Conspiracy” for ICE Protest as Trump Expands War on Dissent
A 35-year-old former U.S. Army sergeant, Bajun “Baji” Mavalwalla II, faces up to six years in prison for protesting against ICE deportations in what legal experts are calling a test case for the Trump administration’s attempts to criminalize and punish dissent. Mavalwalla was arrested and charged with “conspiracy to impede or injure officers” after he was identified in a video taken at the protest and shared on Instagram. Mavalwalla, who survived a roadside bomb blast while serving in…

Two things are worth noticing right now:
1. The military brass *did not* respond well to Trump and Hegseth.
2. The deployment to #Portland keeps getting delayed.
The military will never say "no" to the president (unless he's literally ordering them to open fire on unarmed civilians or something equally obviously illegal). But there are ways to not comply that don't necessarily involve refusal. Brass showing that they aren't aligned with Trump may weaken his billionaire backers, who might be realizing now that weak dictators who can't lead their militaries tend to get toppled... and their oligarch-backers tend to end up against walls.
If folks being ordered to send troops to #PDX don't want to comply, delaying until the there's an initial response from the lawsuit would be basically impossible to detect. The deployment to LA went far too fast, running into logistical challenges like troops sleeping on the floor. The delays we've already seen could indicate either a more careful approach or quiet resistance.
Trump will continue to escalate at every chance he gets. I would be surprised if PDX didn't give him a fight. I doubt the troops will become more interested in serving a guy who's stabbed them in the back and wasted their time at every opportunity.
It is still possible troops just won't deploy. Trump will make something up about how just the threat of an intervention was enough to make things safe or something like that. If we see that, it's 100% the military telling him to kick rocks because he's not competent enough to know when to back down.
Honestly, I think Trump wants revenge for the resistance PDX put up at the end of his last term. Any backing down from that is absolutely a big loss for him.

GenCompositor: Generative Video Compositing with Diffusion Transformer
Shuzhou Yang, Xiaoyu Li, Xiaodong Cun, Guangzhi Wang, Lingen Li, Ying Shan, Jian Zhang
https://arxiv.org/abs/2509.02460

GenCompositor: Generative Video Compositing with Diffusion Transformer
Video compositing combines live-action footage to create video production, serving as a crucial technique in video creation and film production. Traditional pipelines require intensive labor efforts and expert collaboration, resulting in lengthy production cycles and high manpower costs. To address this issue, we automate this process with generative models, called generative video compositing. This new task strives to adaptively inject identity and motion information of foreground video to the…

These Republicans oppose DEI, but also cuts hitting Hispanic-serving colleges (Rachel Hatzipanagos/Washington Post)
https://www.washingtonpost.com/nation/2025/10/30/hispanic-serving-institutions-republican-support/
http://www.memeorandum.com/251030/p70#a251030p70

Due to the devastating harm being done to the USA by the current administration, and the administration of Texas I have donated again to the Texas Civil Rights Project https://www.txcivilrights.org (doubling my giving for 2025).
I will be doing the same for several other

Home | Texas Civil Rights Project
TCRP believe that by serving the rising social justice movement in Texas with excellent legal representation and bold strategies, we can respond to the needs of the communities we serve.

LiquidGEMM: Hardware-Efficient W4A8 GEMM Kernel for High-Performance LLM Serving
Huanqi Hu, Bowen Xiao, Shixuan Sun, Jianian Yin, Zhexi Zhang, Xiang Luo, Chengquan Jiang, Weiqi Xu, Xiaoying Jia, Xin Liu, Minyi Guo
https://arxiv.org/abs/2509.01229

LiquidGEMM: Hardware-Efficient W4A8 GEMM Kernel for High-Performance LLM Serving
Quantization is a critical technique for accelerating LLM inference by reducing memory footprint and improving computational efficiency. Among various schemes, 4-bit weight and 8-bit activation quantization (W4A8) offers a strong balance between accuracy and performance. However, existing W4A8 GEMM kernels fall short in practice due to inefficient dequantization on CUDA Cores, which cannot keep pace with the high throughput of Tensor Cores. In this paper, we present LiquidGEMM, a hardware-effic…

this is an underrated dees, its composition evokes dali and carrington landscapes in a way none of his other stuff does

Batch Query Processing and Optimization for Agentic Workflows
Junyi Shen, Noppanat Wadlom, Yao Lu
https://arxiv.org/abs/2509.02121 https://arxiv.org/pdf/25…

Batch Query Processing and Optimization for Agentic Workflows
Large Language Models (LLMs) in agentic workflows combine multi-step reasoning, tool use, and collaboration across multiple specialized agents. Existing LLM serving engines optimize indi- vidual calls in isolation, while multi-agent frameworks focus on orchestration without system-level performance planning. As a result, repeated prompts, overlapping contexts, and concurrent ex- ecutions create substantial redundancy and poor GPU utilization, especially in batch analytics scenarios. We introduc…

Loop Quantum Vector-Tensor Gravity and Its Spherically Symmetric Model
Shengzhi Li, Yongge Ma
https://arxiv.org/abs/2509.02056 https://arxiv.org/pdf/2509.0…

Loop Quantum Vector-Tensor Gravity and Its Spherically Symmetric Model
The Hamiltoinian analysis of the vector-tensor theory of gravity is performed. The resulting geometrical dynamics is reformulated into the connection dynamics, with the real SU(2)-connection serving as one of the configuration variables. This formulation allows us to extend the loop quantization scheme of general relativity to the vector-tensor theory, thereby rigorously constructing its quantum kinematical framework. The scalar constraint is promoted to a well-defined operator in the vertex Hi…

The headline here is that
(1) the increasingly authoritarian US government demanded that Google suppress videos serving the public interest because those videos are embarrassing to government officials, and
(2) Google readily over-complied with that request, deleting not just the videos but the entire channel.
https://www.theguardian.com/us-news/2025/aug/31/fda-official-youtube-videos

exaPD: A highly parallelizable workflow for multi-element phase diagram (PD) construction
Feng Zhang, Zhuo Ye, Maxim Moraru, Ying Wai Li, Weiyi Xia, Yongxin Yao, Ryan Richard, Cai-Zhuang Wang
https://arxiv.org/abs/2510.01400

exaPD: A highly parallelizable workflow for multi-element phase diagram (PD) construction
Phase diagrams (PDs) illustrate the relative stability of competing phases under varying conditions, serving as critical tools for synthesizing complex materials. Reliable phase diagrams rely on precise free energy calculations, which are computationally intensive. We introduce exaPD, a user-friendly workflow that enables simultaneous sampling of multiple phases across a fine mesh of temperature and composition for free energy calculations. The package employs standard molecular dynamics (MD) a…

Signal Drop in Magnification Profiles: Combining Lensing Simulations and Observations
David Crespo, Joaqu\'in Gonz\'alez-Nuevo, Laura Bonavera, Marcos M. Cueli, Hu Zou, Rebeca Fern\'andez-Fern\'andez, Jose M. Casas
https://arxiv.org/abs/2509.02213

Signal Drop in Magnification Profiles: Combining Lensing Simulations and Observations
Gravitational lensing magnification bias is a valuable tool for studying mass density profiles, with submillimetre galaxies (SMGs) serving as ideal background sources. The satellite distribution in galaxy clusters also provides insights into their mass distribution.This study aims to investigate the signal drop in mass density profiles from magnification bias measurements, assessing the role of satellite galaxies through observational data and lensing simulations. Using a stacking technique, we…

VarCoNet: A variability-aware self-supervised framework for functional connectome extraction from resting-state fMRI
Charalampos Lamprou, Aamna Alshehhi, Leontios J. Hadjileontiadis, Mohamed L. Seghier
https://arxiv.org/abs/2510.02120

VarCoNet: A variability-aware self-supervised framework for functional connectome extraction from resting-state fMRI
Accounting for inter-individual variability in brain function is key to precision medicine. Here, by considering functional inter-individual variability as meaningful data rather than noise, we introduce VarCoNet, an enhanced self-supervised framework for robust functional connectome (FC) extraction from resting-state fMRI (rs-fMRI) data. VarCoNet employs self-supervised contrastive learning to exploit inherent functional inter-individual variability, serving as a brain function encoder that ge…

FPGA-Based RoCEv2-RDMA Readout Electronics for the CTAO-LST Advanced Camera
F. Marini, M. Bellato, A. Bergnoli, D. Corti, A. Griggio, R. Isocrate, L. Modenese, M. Toffano, C. Arcaro, F. Di Pierro, M. Mariotti, M. Mi, P. Wang
https://arxiv.org/abs/2509.02285

FPGA-Based RoCEv2-RDMA Readout Electronics for the CTAO-LST Advanced Camera
CTAO's (Cherenkov Telescope Array Observatory) largest telescopes type, the LST (Large-Sized Telescope), are being installed at the northern site of the Cherenkov Telescope Array (CTA) at the Observatorio del Roque de los Muchachos on the Canary island of La Palma. Their aim is to capture the lowest-energy gamma rays of the observatory. The hereby proposed readout electronics architecture, serving as a proof-of-concept for its advanced camera upgrade, relies on a custom high-channel count fast …

Crosslisted article(s) found for cs.LG. https://arxiv.org/list/cs.LG/new
[1/7]:
- AdaptCache: KV Cache Native Storage Hierarchy for Low-Delay and High-Quality Language Model Serving
Feng, Li, Du, Gu, Liu, Yao, Ray, Shen, Cheng, Ananthanarayanan, Jiang

Comparative Analysis of Ant Colony Optimization and Google OR-Tools for Solving the Open Capacitated Vehicle Routing Problem in Logistics
Assem Omar, Youssef Omar, Marwa Solayman, Hesham Mansour
https://arxiv.org/abs/2509.26216

Comparative Analysis of Ant Colony Optimization and Google OR-Tools for Solving the Open Capacitated Vehicle Routing Problem in Logistics
In modern logistics management systems, route planning requires high efficiency. The Open Capacitated Vehicle Routing Problem (OCVRP) deals with finding optimal delivery routes for a fleet of vehicles serving geographically distributed customers, without requiring the vehicles to return to the depot after deliveries. The present study is comparative in nature and speaks of two algorithms for OCVRP solution: Ant Colony Optimization (ACO), a nature-inspired metaheuristic; and Google OR-Tools, an …

@… Yeah, I am not opposed to state varying across that level. Across applications, maintained by separate teams or even departments, of course! You don’t want to couple yourself to people who might have drastically different definitions of “success” to you.
But within a single application, serving a single purpose? Only if there’s no better way.…

Ukraine captures Kenyan serving in Russian army, who claims he was tricked into joining: https://benborges.xyz/2025/09/17/ukraine-captures-kenyan-serving-in.html

How Surveillance Firms Use ‘Democracy’ As a Cover for Serving ICE and Trump https://www.404media.co/how-surveillance-firms-use-democracy-as-a-cover-for-serving-ice-and-trump/

Another Trump-Appointed U.S. Attorney Found to be Serving Unlawfully, Federal Judge Rules (Yunior Rivas/Democracy Docket)
https://www.democracydocket.com/news-alerts/another-trump-appointed-u-s-attorney-found-to-be-serving-unlawfully-federal-judge-rules/
http://www.memeorandum.com/251028/p156#a251028p156

Virginia Tech coaching search: Bruce Arians, Super Bowl winner and former Hokies QB, serving as consultant

https://www.cbssports.com/college-foo…

I got Epsom Salt. That’s just uncanny. https://beige.party/@RickiTarr/115124087333264067

New #Pixelfed Post:
#Fediverse

The most up-to-date light curve of interstellar comet #ATLAS - https://x.com/AsteroidEnergy/status/1983707109045989551 - with sparse data points all the way to perihelion, https://bsky.app/profile/kwalsh4a.bsky.social/post/3m4bwiczncs2z from PUNCH serving the latest. So as seen from Earth the brightness is about 9.5 mag. - and should still be 10.something when the comet can be observed from the ground again, from about 10 November onwards.

So, we had we had the Chez Dallman Pecan Chicken Bok Choy again. I wanted to take my wife's picture, but she protested and said she never looks good under the glaring kitchen light. I pushed the settings button in the camera app and said wryly, "Ah, there it is: turning on 'Make My Wife Beautiful' mode!" And I pushed at the screen.
My wife cracked up. (I did too.) It worked. She likes the photo, or at least she doesn't hate it. I can't post it here, though…

VeriLLM: A Lightweight Framework for Publicly Verifiable Decentralized Inference
Ke Wang, Felix Qu, Libin Xia, Zishuo Zhao, Chris Tong, Lynn Ai, Eric Yang
https://arxiv.org/abs/2509.24257

VeriLLM: A Lightweight Framework for Publicly Verifiable Decentralized Inference
Decentralized inference is an appealing paradigm for serving large language models (LLMs), offering strong security, high efficiency, and lower operating costs. Yet the permissionless setting admits no a priori trust in participating nodes, making output verifiability a prerequisite for secure deployment. We present VeriLLM, a publicly verifiable protocol for decentralized LLM inference that (i) achieves security under a one-honest-verifier assumption, (ii) attains near-negligible verification …

Evaluating SAP Joule for Code Generation
Joshua Heisler, Johannes Reisinger, Andreas Fischer
https://arxiv.org/abs/2509.24828 https://arxiv.org/pdf/2509.24…

Evaluating SAP Joule for Code Generation
SAP has released its own proprietary generative model SAP Joule, intended for various generative tasks, including serving as a code assistant for software engineers. While Joule is yet not focused on SAP-specific ABAP code generation, it can be used for other common languages, including Javascript. This paper compares SAP Joules Javascript coding capabilities against a total of 29 other models using the HumanEval-X Javascript benchmark. SAP Joule achieves a strict accuracy of 80.49% as the fift…

A Predictive and Synergistic Two-Layer Scheduling Framework for LLM Serving
Yue Zhang, Yuansheng Chen, Xuan Mo, Alex Xi, Jialun Li, WeiGang Wu
https://arxiv.org/abs/2509.23384 h…

A Predictive and Synergistic Two-Layer Scheduling Framework for LLM Serving
LLM inference serving typically scales out with a two-tier architecture: a cluster router distributes requests to multiple inference engines, each of which then in turn performs its own internal scheduling. However, this commonly used paradigm suffers from critical, systemic inefficiency caused by the information gaps across two layers. At the cluster-layer, the router mainly relies on lagging, coarse-grained metrics, such as average latency and queue length to make decisions, resulting in "dec…

Report finds dangerous mercury levels, highlights mislabeling in shark meat sold in EU https://news.mongabay.com/2025/10/report-finds-dangerous-mercury-levels-highlights-mislabeling-i…

Report finds dangerous mercury levels, highlights mislabeling in shark meat sold in EU
Shark meat sold across Europe may be serving up more than seafood. Dangerously high levels of methylmercury, toxic to humans, were present in nearly a third of shark meat samples taken from European restaurants, fishmongers and supermarkets, according to a new report by a group of marine conservation NGOs. The findings suggest that Europeans are […]

Chatbots Are Pushing Sanctioned Russian Propaganda
https://www.wired.com/story/chatbots-are-pushing-sanctioned-russian-propaganda/

Chatbots Are Pushing Sanctioned Russian Propaganda
ChatGPT, Gemini, DeepSeek, and Grok are serving users propaganda from Russian-backed media when asked about the invasion of Ukraine, new research finds.

That's sad: what was far and away my favorite restaurant during last year's visit to Portland (Ore.) closed. Brunch-serving vegan diners are a particular type of place I love that seems to be an endangered species these days, cf Beach'N in SF, also closed this year.

One of Portland’s oldest vegan-friendly restaurants has closed
“We can no long sustain operations and have served the final plate,” the restaurant announces.

Resource-efficient universal photonic processor based on time-multiplexed hybrid architectures
Jonas Lammers, Laura Ares, Federico Pegoraro, Philip Held, Benjamin Brecht, Jan Sperling, Christine Silberhorn
https://arxiv.org/abs/2509.22521

Resource-efficient universal photonic processor based on time-multiplexed hybrid architectures
For the ever-growing field of quantum information processing, large-scale, efficient multi-port interferometers serving as photonic processors are required. In this context, the suitability of quantum walks as the interferometric base for universal computation has been theoretically proven. In this work, we bridge the gap between theoretical proposals and state-of-the-art experimental capabilities by providing the recipe for the implementation of a universal photonic processor in discrete-time …

Fault Injection in On-Chip Interconnects: A Comparative Study of Wishbone, AXI-Lite, and AXI
Hongwei Zhao, Vianney Lapotre, Guy Gogniat
https://arxiv.org/abs/2509.24929 https://…

Fault Injection in On-Chip Interconnects: A Comparative Study of Wishbone, AXI-Lite, and AXI
Fault injection attacks exploit physical disturbances to compromise the functionality and security of integrated circuits. As System on Chip (SoC) architectures grow in complexity, the vulnerability of on chip communication fabrics has become increasingly prominent. Buses, serving as interconnects among various IP cores, represent potential vectors for fault-based exploitation. In this study, we perform simulation-driven fault injection across three mainstream bus protocols Wishbone, AXI Lite, …

After two decades in Congress,
Darrell Issa’s career is all about serving the ultra-wealthy like himself
– and serving Donald Trump. It’s time to send him packing and take our country back. 
On the City Council,
Marni von Wilpert
flipped San Diego’s reddest seat blue.
Now, she’s ready to do it again – in Congress.
https://www.

Marni von Wilpert for Congress
Marni von Wilpert is running for Congress to tackle the skyrocketing cost of living, protect reproductive freedom, defend LGBTQ rights, and make Washington work for working families — not just the wealthy few. With a proven record of service, results, and winning tough races, Marni is the battle-tested fighter we need to flip CA-48 and finally send Darrell Issa packing.

Rethinking Caching for LLM Serving Systems: Beyond Traditional Heuristics
Jungwoo Kim, Minsang Kim, Jaeheon Lee, Chanwoo Moon, Heejin Kim, Taeho Hwang, Woosuk Chung, Yeseong Kim, Sungjin Lee
https://arxiv.org/abs/2508.18736

Rethinking Caching for LLM Serving Systems: Beyond Traditional Heuristics
Serving Large Language Models (LLMs) at scale requires meeting strict Service Level Objectives (SLOs) under severe computational and memory constraints. Nevertheless, traditional caching strategies fall short: exact-matching and prefix caches neglect query semantics, while state-of-the-art semantic caches remain confined to traditional intuitions, offering little conceptual departure. Building on this, we present SISO, a semantic caching system that redefines efficiency for LLM serving. SISO in…

SparseServe: Unlocking Parallelism for Dynamic Sparse Attention in Long-Context LLM Serving
Qihui Zhou, Peiqi Yin, Pengfei Zuo, James Cheng
https://arxiv.org/abs/2509.24626 http…

SparseServe: Unlocking Parallelism for Dynamic Sparse Attention in Long-Context LLM Serving
Serving long-context LLMs is costly because attention computation grows linearly with context length. Dynamic sparse attention algorithms (DSAs) mitigate this by attending only to the key-value (KV) cache of critical tokens. However, with DSAs, the main performance bottleneck shifts from HBM bandwidth to HBM capacity: KV caches for unselected tokens must remain in HBM for low-latency decoding, constraining parallel batch size and stalling further throughput gains. Offloading these underutilized…

Drop Site uncovered new information about individuals, donor networks, and businesses helping Canary Mission, a pro-Israel organization serving the U.S.'s deportation and repression efforts.
https://www.dropsitenews.com/p/canary-miss

"For me, our job as artists is to serve the story, serve the director, and serve the fellow actors. And if you do that, by osmosis you’re serving yourself because you’ll get the best out of yourself."
—David Oyelowo
#acting #coaching

The application is currently not serving requests at this endpoint. It may not have been started or is still starting.

Zero-Waiting Load Balancing with Heterogeneous Servers in Heavy Traffic
Xin Liu, Lei Ying
https://arxiv.org/abs/2509.23918 https://arxiv.org/pdf/2509.23918…

Zero-Waiting Load Balancing with Heterogeneous Servers in Heavy Traffic
We study the steady-state delay performance of load balancing in large-scale systems with heterogeneous servers in the heavy-traffic regimes. The system consists of $N$ servers, each with a local buffer of size $b-1$, serving jobs in the first-in-first-out (FIFO) order. Jobs arrive according to a Poisson process with rate $λN$, where $λ= 1 - N^{-α}$ for any $α\in (0,1)$. Service times are assumed to be exponentially distributed with fully heterogeneous rates, where the service rate of each …

As Trump Defunds Infrastructure, Water Systems Serving Millions Face Flood Risk - WhoWhatWhy
https://whowhatwhy.org/science/environment/as-trump-defunds-infrastructure-water-systems-serving-millions-face-flood-risk/

Information Transmission in Quorum Sensing for Gut Microbiome
O. Tansel Baydas, Efe Yatgin, Ozgur B. Akan
https://arxiv.org/abs/2509.25057 https://arxiv.or…

Information Transmission in Quorum Sensing for Gut Microbiome
Microorganisms employ sophisticated mechanisms for intercellular communication and environmental sensing, with quorum sensing serving as a fundamental regulatory process. Dysregulation of quorum sensing has been implicated in various diseases. While most theoretical studies focus on mathematical modeling of quorum sensing dynamics, the communication-theoretic aspects remain less explored. In this study, we investigate the information processing capabilities of quorum sensing systems using a sto…

happens to the best of us
https://blog.cloudflare.com/deep-dive-into-cloudflares-sept-12-dashboard-and-api-outage/

A deep dive into Cloudflare’s September 12, 2025 dashboard and API outage
Cloudflare’s Dashboard and a set of related APIs were unavailable or partially available for an hour starting on Sep 12, 17:57 UTC. The outage did not affect the serving of cached files via the Cloudflare CDN or other security features at the Cloudflare Edge.

I've been thinking a bit about historians' use of theory. I've decided it's often similar to a comment a cafeteria worker made to me decades ago. They were serving chili dogs and I asked if I could just have a bowl of chili instead. "Oh no, dear," she said, "this is chili dog chili, it's not for eating." I've long suspected that a good chunk of my field thinks of theory along these lines. We put it on the menu and let our students consume it, even …

Measuring the environmental impact of delivering AI at Google Scale
Cooper Elsworth, Keguo Huang, David Patterson, Ian Schneider, Robert Sedivy, Savannah Goodman, Ben Townsend, Parthasarathy Ranganathan, Jeff Dean, Amin Vahdat, Ben Gomes, James Manyika
https://arxiv.org/abs/2508.15734

Measuring the environmental impact of delivering AI at Google Scale
The transformative power of AI is undeniable - but as user adoption accelerates, so does the need to understand and mitigate the environmental impact of AI serving. However, no studies have measured AI serving environmental metrics in a production environment. This paper addresses this gap by proposing and executing a comprehensive methodology for measuring the energy usage, carbon emissions, and water consumption of AI inference workloads in a large-scale, AI production environment. Our approa…

A profile of Noah Urban, who was a key member of the Scattered Spider group because of his social engineering skills and is serving a 10-year prison sentence (Margi Murphy/Bloomberg)

DRIFT-Net: A Spectral--Coupled Neural Operator for PDEs Learning
Jiayi Li, Flora D. Salim
https://arxiv.org/abs/2509.24868 https://arxiv.org/pdf/2509.24868…

DRIFT-Net: A Spectral--Coupled Neural Operator for PDEs Learning
Learning PDE dynamics with neural solvers can significantly improve wall-clock efficiency and accuracy compared with classical numerical solvers. In recent years, foundation models for PDEs have largely adopted multi-scale windowed self-attention, with the scOT backbone in \textsc{Poseidon} serving as a representative example. However, because of their locality, truly globally consistent spectral coupling can only be propagated gradually through deep stacking and window shifting. This weakens…

GSM8K-V: Can Vision Language Models Solve Grade School Math Word Problems in Visual Contexts
Fan Yuan, Yuchen Yan, Yifan Jiang, Haoran Zhao, Tao Feng, Jinyan Chen, Yanwei Lou, Wenqi Zhang, Yongliang Shen, Weiming Lu, Jun Xiao, Yueting Zhuang
https://arxiv.org/abs/2509.25160

GSM8K-V: Can Vision Language Models Solve Grade School Math Word Problems in Visual Contexts
Vision language models (VLMs) achieve unified modeling of images and text, enabling them to accomplish complex real-world tasks through perception, planning, and reasoning. Among these tasks, reasoning is particularly representative, with mathematical reasoning serving as a prominent example. It highlights the high-level capability of VLMs to comprehend mathematical information in images and to perform sophisticated reasoning. Recently, numerous visual mathematical reasoning benchmarks have bee…

Particle Acceleration and Transport in the Large-scale Current Sheet under an Erupting Magnetic Flux Rope
Hao Wu, Yang Guo, Rony Keppens, Chun Xia, Yang Su, Xiangliang Kong, Mingde Ding
https://arxiv.org/abs/2509.22265

Particle Acceleration and Transport in the Large-scale Current Sheet under an Erupting Magnetic Flux Rope
We investigate the acceleration and transport of electrons in the highly fine-structured current sheet that develops during magnetic flux rope (MFR) eruptions. Our work combines ultra-resolved MHD simulations of MFR eruption, with test-particle studies performed using the guiding center approximation. Our grid-adaptive, fully three-dimensional, high-resolution magnetohydrodynamic simulations model MFR eruptions that form complex current sheet topologies, serving as background electromagnetic fi…

I appreciate the premise of reducing red meat in one's diet... but if you expect me to drop fish or seafood, you can go to hell.
✅ The Self-Importance of Luxury Dining: Eleven Madison Park is serving meat again—a sign of American tastes, and of fine-dining hubris - The Atlantic
https://archive.ph/BxO7Z

Nova: Real-Time Agentic Vision-Language Model Serving with Adaptive Cross-Stage Parallelization
Yuhang Xu, Shengzhong Liu, Dong Zhang, Bingheng Yan, Fan Wu, Guihai Chen
https://arxiv.org/abs/2509.21301

Nova: Real-Time Agentic Vision-Language Model Serving with Adaptive Cross-Stage Parallelization
This paper presents Nova, a real-time scheduling framework for serving agentic vision-language models (VLMs) on a single GPU with balanced per-request latency and overall request process throughput. Our design begins by enabling effective pipelining across vision encode, LLM prefill, and LLM decode stages of VLMs, by exploiting their heterogeneous resource demands during execution and incorporating elastic GPU spatial partitioning among stages to maximally utilize the compute and memory resourc…

LLM Serving Optimization with Variable Prefill and Decode Lengths
Meixuan Wang, Yinyu Ye, Zijie Zhou
https://arxiv.org/abs/2508.06133 https://arxiv.org/pdf…

LLM Serving Optimization with Variable Prefill and Decode Lengths
We study the problem of serving LLM (Large Language Model) requests where each request has heterogeneous prefill and decode lengths. In LLM serving, the prefill length corresponds to the input prompt length, which determines the initial memory usage in the KV cache. The decode length refers to the number of output tokens generated sequentially, with each additional token increasing the KV cache memory usage by one unit. Given a set of n requests, our goal is to schedule and process them to mini…

From https://bsky.app/profile/did:plc:cysfyu7ook326epmpvun2qtb/post/3lzs7noi6oc23 —
> Prof. Lee says that the problem for Canada Post is the smartphone because people send texts, not letters. Fine, but don't gut it. I say, make Canada Post a pub…

A federal judge ruled Thursday that Trump’s former lawyer,
Alina Habba, has been unlawfully serving as the the top federal prosecutor in New Jersey since last month.
U.S. District Judge Matthew Brann held that Habba’s term as the interim U.S. attorney ended in July,
and the Trump administration’s “novel series of legal and personnel moves” to keep her in the role
-- without getting confirmation from the U.S. Senate
-- didn’t follow procedures required by federal l…

Judge says Alina Habba has been unlawfully serving as US attorney in New Jersey
A judge has ruled that President Donald Trump’s former lawyer, Alina Habba, has been unlawfully serving as the the top federal prosecutor in New Jersey. U.S. District Judge Matthew Brann held Thursday that Habba’s term as the interim U.S. attorney ended in July, and the Trump administration’s “novel series of legal and personnel moves” to keep her in the role without getting confirmation from the U.S. Senate didn’t follow procedures required by federal law. It wasn’t immediately c…

MPFormer: Adaptive Framework for Industrial Multi-Task Personalized Sequential Retriever
Yijia Sun, Shanshan Huang, Linxiao Che, Haitao Lu, Qiang Luo, Kun Gai, Guorui Zhou
https://arxiv.org/abs/2508.20400

MPFormer: Adaptive Framework for Industrial Multi-Task Personalized Sequential Retriever
Modern industrial recommendation systems encounter a core challenge of multi-stage optimization misalignment: a significant semantic gap exists between the multi-objective optimization paradigm widely used in the ranking phase and the single-objective modeling in the retrieve phase. Although the mainstream industry solution achieves multi-objective coverage through parallel multi-path single-objective retrieval, this approach leads to linear growth of training and serving resources with the num…

Very high-energy gamma-ray and neutrino emission from hadronic interaction in compact binary millisecond pulsars
Vittoria Vecchiotti, Manuel Linares
https://arxiv.org/abs/2508.20952

Very high-energy gamma-ray and neutrino emission from hadronic interaction in compact binary millisecond pulsars
Blackwidow and redback systems are millisecond pulsars in compact orbits with ultra-light and low-mass companions, respectively, collectively known as ``spider pulsars". In such systems, an intrabinary shock can form between the pulsar and the companion winds, serving as a site for particle acceleration and associated non-thermal emission. Assuming that protons can be extracted from the neutron star surface and accelerated at the intrabinary shock and/or within the pulsar wind, we model the ver…

FLAME: A Serving System Optimized for Large-Scale Generative Recommendation with Efficiency
Xianwen Guo, Bin Huang, Xiaomeng Wu, Guanlin Wu, Fangjian Li, Shijia Wang, Qiang Xiao, Chuanjiang Luo, Yong Li
https://arxiv.org/abs/2509.22681

FLAME: A Serving System Optimized for Large-Scale Generative Recommendation with Efficiency
Generative recommendation (GR) models possess greater scaling power compared to traditional deep learning recommendation models (DLRMs), yet they also impose a tremendous increase in computational burden. Measured in FLOPs, a typical GR model's workload sits in $10^9 \sim 10^{11}$ range, roughly four orders of magnitude higher than traditional DLRMs. Delivering accurate results in a few tens of milliseconds while processing billions of such requests per day puts extreme demands on the performan…

GreenLLM: SLO-Aware Dynamic Frequency Scaling for Energy-Efficient LLM Serving
Qunyou Liu, Darong Huang, Marina Zapater, David Atienza
https://arxiv.org/abs/2508.16449 https://

GreenLLM: SLO-Aware Dynamic Frequency Scaling for Energy-Efficient LLM Serving
Large Language Models (LLMs) are becoming the backbone of modern cloud services, yet their inference costs are dominated by GPU energy. Unlike traditional GPU workloads, LLM inference has two stages with different characteristics: the prefill phase, which is latency sensitive and scales quadratically with prompt length, and the decode phase, which progresses token by token with unpredictable length. Current GPU power governors (for example, NVIDIA's default) overlook this asymmetry and treat bo…

Intra-request branch orchestration for efficient LLM reasoning
Weifan Jiang, Rana Shahout, Yilun Du, Michael Mitzenmacher, Minlan Yu
https://arxiv.org/abs/2509.24957 https://

Intra-request branch orchestration for efficient LLM reasoning
Large Language Models (LLMs) increasingly rely on inference-time reasoning algorithms such as chain-of-thought and multi-branch reasoning to improve accuracy on complex tasks. These methods, however, substantially increase token usage and per-request latency. Prior work has largely focused on reducing token usage, often at the expense of accuracy, while overlooking other latency factors. We present DUCHESS, an LLM serving system that reduces cost and latency without sacrificing accuracy through…

RServe: Overlapping Encoding and Prefill for Efficient LMM Inference
Tianyu Guo, Tianming Xu, Xianjie Chen, Junru Chen, Nong Xiao, Xianwei Zhang
https://arxiv.org/abs/2509.24381

RServe: Overlapping Encoding and Prefill for Efficient LMM Inference
Large multimodal models (LMMs) typically employ an encoding module to transform multimodal data inputs into embeddings, which are then fed to language models for further processing. However, efficiently serving LMMs remains highly challenging due to the inherent complexity of their inference pipelines. Traditional serving engines co-locate the encoding module and the language model, leading to significant resource interference and tight data dependency. Recent studies have alleviated this issue…

"In 2024, the group focused on recruiting and retention, women serving on submarines and mothers reintegrating to military life after pregnancy"
Pentagon shutters women’s advisory group
https://taskandpurpose.com/news/defense-womens-advisory-group/

National Federation of Community Broadcasters gets a $1.25M MacArthur grant; NFCB represents ~200 stations mostly serving rural and underrepresented communities (Austin Fuller/Current)
https://current.org/2025/10/national-federation-of-c…

National Federation of Community Broadcasters lands $1.25M MacArthur grant
The grant “is both an affirmation of the essential role community radio plays in civic life and an investment in the infrastructure that sustains it,” said NFCB CEO Rima Dael.

Cycle is All You Need: More Is Different
Xin Li
https://arxiv.org/abs/2509.21340 https://arxiv.org/pdf/2509.21340…

Cycle is All You Need: More Is Different
We propose an information-topological framework in which cycle closure is the fundamental mechanism of memory and consciousness. Memory is not a static store but the ability to re-enter latent cycles in neural state space, with invariant cycles serving as carriers of meaning by filtering order-specific noise and preserving what persists across contexts. The dot-cycle dichotomy captures this: transient dots scaffold exploration, while nontrivial cycles encode low-entropy content invariants that …

Embodied AI: From LLMs to World Models
Tongtong Feng, Xin Wang, Yu-Gang Jiang, Wenwu Zhu
https://arxiv.org/abs/2509.20021 https://arxiv.org/pdf/2509.20021

Embodied AI: From LLMs to World Models
Embodied Artificial Intelligence (AI) is an intelligent system paradigm for achieving Artificial General Intelligence (AGI), serving as the cornerstone for various applications and driving the evolution from cyberspace to physical systems. Recent breakthroughs in Large Language Models (LLMs) and World Models (WMs) have drawn significant attention for embodied AI. On the one hand, LLMs empower embodied AI via semantic reasoning and task decomposition, bringing high-level natural language instruc…

DHS moves to bar aid groups from serving undocumented immigrants (Brianna Sacks/Washington Post)
https://www.washingtonpost.com/weather/2025/08/27/dhs-fema-undocumented-immigrants-aid-groups-grants/?pwapi_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJyZWFzb24iOiJnaWZ0IiwibmJmIjoxNzU2MjY3MjAwLCJpc3MiOiJzdWJzY3JpcHRpb25zIiwiZXhwIjoxNzU3NjQ5NTk5LCJpYXQiOjE3NTYyNjcyMDAsImp0aSI6ImM2ZDBlZDIyLTE5NmUtNGQzZi04ZmNiLWFkZDFjZjRlYWFkZCIsInVybCI6Imh0dHBzOi8vd3d3Lndhc2hpbmd0b25wb3N0LmNvbS93ZWF0aGVyLzIwMjUvMDgvMjcvZGhzLWZlbWEtdW5kb2N1bWVudGVkLWltbWlncmFudHMtYWlkLWdyb3Vwcy1ncmFudHMvIn0.jLBr6sPapG7mcyHKjeTPUwshLfB7UHhyklQxwk-f1xo&itid=gfta
http://www.memeorandum.com/250827/p96#a250827p96

Dario Amodei addresses "inaccurate claims" about Anthropic's policy stances after David Sacks said the "real issue" is "Anthropic's agenda to backdoor Woke AI" (Ashley Capoot/CNBC)
https://www.cnbc.com/2025/10/21/anthropic-ceo-trump-s…

Anthropic CEO disputes Trump AI czar David Sacks' claims that company is 'woke'
The company has faced repeated criticism from David Sacks, the venture capitalist serving as President Donald Trump's AI and crypto czar, in recent days.

Grounded AI for Code Review: Resource-Efficient Large-Model Serving in Enterprise Pipelines
Sayan Mandal, Hua Jiang
https://arxiv.org/abs/2510.10290 https://

Grounded AI for Code Review: Resource-Efficient Large-Model Serving in Enterprise Pipelines
Automated code review adoption lags in compliance-heavy settings, where static analyzers produce high-volume, low-rationale outputs, and naive LLM use risks hallucination and incurring cost overhead. We present a production system for grounded, PR-native review that pairs static-analysis findings with AST-guided context extraction and a single-GPU, on-demand serving stack (quantized open-weight model, multi-tier caching) to deliver concise explanations and remediation guidance. Evaluated on saf…

The official Xubuntu website was compromised over the weekend (18/19 October 2025) briefly serving up Windows malware to users trying to download the distro.
https://www.omgubuntu.co.uk/2025/10/xubuntu-website-malware-hack

Xubuntu Website Compromised, Served Malware Downloads - OMG! Ubuntu
The official Xubuntu website briefly served Windows malware to users trying to download the distro between 18-19 October. ISOs were not affected.

Optimal quantum simulation of linear non-unitary dynamics
Guang Hao Low, Rolando D. Somma
https://arxiv.org/abs/2508.19238 https://arxiv.org/pdf/2508.19238…

Optimal quantum simulation of linear non-unitary dynamics
We present a quantum algorithm for simulating the time evolution generated by any bounded, time-dependent operator $-A$ with non-positive logarithmic norm, thereby serving as a natural generalization of the Hamiltonian simulation problem. Our method generalizes the recent Linear-Combination-of-Hamiltonian-Simulation (LCHS) framework. In instances where $A$ is time-independent, we provide a block-encoding of the evolution operator $e^{-At}$ with $\mathcal{O}\big(t\log\frac{1}ε)$ queries to the …

ExpertWeave: Efficiently Serving Expert-Specialized Fine-Tuned Adapters at Scale
Ge Shi, Hanieh Sadri, Qian Wang, Yu Zhang, Ying Xiong, Yong Zhang, Zhenan Fan
https://arxiv.org/abs/2508.17624

ExpertWeave: Efficiently Serving Expert-Specialized Fine-Tuned Adapters at Scale
Expert-Specialized Fine-Tuning (ESFT) adapts Mixture-of-Experts (MoE) large language models to enhance their task-specific performance by selectively tuning the top-activated experts for the task. Serving these fine-tuned models at scale is challenging: deploying merged models in isolation is prohibitively resource-hungry, while existing multi-adapter serving systems with LoRA-style additive updates are incompatible with ESFT's expert-oriented paradigm. We present ExpertWeave, a system that ser…

Systematic Characterization of LLM Quantization: A Performance, Energy, and Quality Perspective
Tianyao Shi, Yi Ding
https://arxiv.org/abs/2508.16712 https://

Systematic Characterization of LLM Quantization: A Performance, Energy, and Quality Perspective
Large language models (LLMs) have demonstrated remarkable capabilities across diverse domains, but their heavy resource demands make quantization-reducing precision to lower-bit formats-critical for efficient serving. While many quantization methods exist, a systematic understanding of their performance, energy, and quality tradeoffs in realistic serving conditions remains a gap. In this work, we first develop a fully automated online characterization framework qMeter, and then conduct an in-de…

Structure-aware Hypergraph Transformer for Diagnosis Prediction in Electronic Health Records
Haiyan Wang, Ye Yuan
https://arxiv.org/abs/2508.20500 https://…

Structure-aware Hypergraph Transformer for Diagnosis Prediction in Electronic Health Records
Electronic Health Records (EHR) systematically organize patient health data through standardized medical codes, serving as a comprehensive and invaluable source for predictive modeling. Graph neural networks (GNNs) have demonstrated effectiveness in modeling interactions between medical codes within EHR. However, existing GNN-based methods are inadequate due to: a) their reliance on pairwise relations fails to capture the inherent higher-order dependencies in clinical data, and b) the localized…

Parallax: Efficient LLM Inference Service over Decentralized Environment
Chris Tong, Youhe Jiang, Gufeng Chen, Tianyi Zhao, Sibian Lu, Wenjie Qu, Eric Yang, Lynn Ai, Binhang Yuan
https://arxiv.org/abs/2509.26182

Parallax: Efficient LLM Inference Service over Decentralized Environment
Deploying a large language model (LLM) inference service remains costly because centralized serving depends on specialized GPU clusters and high-bandwidth interconnects in datacenters. An appealing alternative is to leverage collaborative decentralized GPU pools. However, heterogeneity in GPU and limited interconnected network bandwidth, along with potentially dynamic availability, make efficient scheduling the central challenge in this scenario. In this paper, we present Parallax, a decentrali…

Alibaba Cloud details a GPU pooling system that it claims reduced the number of Nvidia H20 required by 82% when serving dozens of LLMs of up to 72B parameters (Vincent Chow/South China Morning Post)
https://www.scmp.com/business/article/3329

Alibaba Cloud claims to slash Nvidia GPU use by 82% with new pooling system
The new Aegaeon system can serve dozens of large language models using a fraction of the GPUs previously required, potentially reshaping AI workloads.

Justice Department says U.S. won't defend grants for Hispanic-serving colleges, calling them unconstitutional - CBS News
https://www.cbsnews.com/news/justice-department-hispanic-colleges-grants-unconstitutional/

Synthesizing Artifact Dataset for Pixel-level Detection
Dennis Menn, Feng Liang, Diana Marculescu
https://arxiv.org/abs/2509.19589 https://arxiv.org/pdf/25…

Synthesizing Artifact Dataset for Pixel-level Detection
Artifact detectors have been shown to enhance the performance of image-generative models by serving as reward models during fine-tuning. These detectors enable the generative model to improve overall output fidelity and aesthetics. However, training the artifact detector requires expensive pixel-level human annotations that specify the artifact regions. The lack of annotated data limits the performance of the artifact detector. A naive pseudo-labeling approach-training a weak detector and using…

Embodied AI: From LLMs to World Models
Tongtong Feng, Xin Wang, Yu-Gang Jiang, Wenwu Zhu
https://arxiv.org/abs/2509.20021 https://arxiv.org/pdf/2509.20021

Embodied AI: From LLMs to World Models
Embodied Artificial Intelligence (AI) is an intelligent system paradigm for achieving Artificial General Intelligence (AGI), serving as the cornerstone for various applications and driving the evolution from cyberspace to physical systems. Recent breakthroughs in Large Language Models (LLMs) and World Models (WMs) have drawn significant attention for embodied AI. On the one hand, LLMs empower embodied AI via semantic reasoning and task decomposition, bringing high-level natural language instruc…

HyperFlexis: Joint Design of Algorithms and Systems for Multi-SLO Serving and Fast Scaling
Zahra Yousefijamarani, Xinglu Wang, Qian Wang, Morgan Lindsay Heisler, Taha Shabani, Niloofar Gholipour, Parham Yassini, Hong Chang, Kan Chen, Qiantao Zhang, Xiaolong Bai, Jiannan Wang, Ying Xiong, Yong Zhang, Zhenan Fan
https://arxiv.org/abs/2508.15…

HyperFlexis: Joint Design of Algorithms and Systems for Multi-SLO Serving and Fast Scaling
Modern large language model (LLM) serving systems face challenges from highly variable requests with diverse lengths, priorities, and stage-specific service-level objectives (SLOs). Meeting these requires real-time scheduling, rapid and cost-effective scaling, and support for both collocated and disaggregated Prefill/Decode (P/D) architectures. We present \textbf{HyperFlexis}, a unified LLM serving system that integrates algorithmic and system-level innovations to jointly optimize scheduling …

City Matters, a free monthly newspaper serving the City of London since 2016, enters voluntary liquidation, citing rising print costs and declining ad revenue (Alice Brooker/Press Gazette)
https://pressgazette.co.uk/publishers/reg…

Fleet Street's local newspaper goes out of business
City Matters launched in 2016, aiming to "fill a local news gap" for City of London which, at the time, had "no dedicated title".

A senior CIA officer who oversaw Russia analysis
has been stripped of her security clearance,
part of a sweeping removal of
37 serving and former officials accused of "betray[ing] their oath to the Constitution," the Economist reported on Aug. 21.
The officer, who served as the CIA’s top Russia and Eurasia analyst during the 2016 election
and helped produce the report detailing Moscow’s interference on behalf of Donald Trump,
was among the most s…

Trump administration escalates purge of senior US intelligence officials, Economist reports
The officer, who served as the CIA’s top Russia and Eurasia analyst during the 2016 election and helped produce the report detailing Moscow’s interference on behalf of Donald Trump, was among the most senior career officials targeted.

Expert-as-a-Service: Towards Efficient, Scalable, and Robust Large-scale MoE Serving
Ziming Liu, Boyu Tian, Guoteng Wang, Zhen Jiang, Peng Sun, Zhenhua Han, Tian Tang, Xiaohe Hu, Yanmin Jia, Yan Zhang, He Liu, Mingjun Zhang, Yiqi Zhang, Qiaoling Chen, Shenggan Cheng, Mingyu Gao, Yang You, Siyuan Feng
https://arxiv.org/abs/2509.17863…

Expert-as-a-Service: Towards Efficient, Scalable, and Robust Large-scale MoE Serving
Mixture-of-Experts (MoE) models challenge serving infrastructures with dynamic, sparse expert utilization, causing instability on conventional systems designed for dense architectures. We propose EaaS, a novel serving system to enable efficient, scalable, and robust MoE deployment. Our system disaggregates MoE modules into independent, stateless services. This design enables fine-grained resource scaling and provides inherent fault tolerance by decoupling compute units. The architecture is powe…

Justice Dept. declines to defend grants for Hispanic-serving colleges, calling them unconstitutional (Associated Press)
https://apnews.com/article/hispanic-colleges-trump-a795fc966590681f41410c2b3e268ac0
http://www.memeorandum.com/250822/p117#a250822p117

Equinox: Holistic Fair Scheduling in Serving Large Language Models
Zhixiang Wei, James Yen, Jingyi Chen, Ziyang Zhang, Zhibai Huang, Chen Chen, Xingzi Yu, Yicheng Gu, Chenggang Wu, Yun Wang, Mingyuan Xia, Jie Wu, Hao Wang, Zhengwei Qi
https://arxiv.org/abs/2508.16646

Equinox: Holistic Fair Scheduling in Serving Large Language Models
We address the limitations of current LLM serving with a dual-counter framework separating user and operator perspectives. The User Fairness Counter measures quality of service via weighted tokens and latency; the Resource Fairness Counter measures operational efficiency through throughput and GPU utilization. Since these metrics are only available post-execution, creating a scheduling paradox, we introduce a deterministic Mixture of Prediction Experts (MoPE) framework to predict user-perceived…

TinyML Towards Industry 4.0: Resource-Efficient Process Monitoring of a Milling Machine
Tim Langer, Matthias Widra, Volkhard Beyer
https://arxiv.org/abs/2508.16553 https://

TinyML Towards Industry 4.0: Resource-Efficient Process Monitoring of a Milling Machine
In the context of industry 4.0, long-serving industrial machines can be retrofitted with process monitoring capabilities for future use in a smart factory. One possible approach is the deployment of wireless monitoring systems, which can benefit substantially from the TinyML paradigm. This work presents a complete TinyML flow from dataset generation, to machine learning model development, up to implementation and evaluation of a full preprocessing and classification pipeline on a microcontrolle…

Strata: Hierarchical Context Caching for Long Context Language Model Serving
Zhiqiang Xie, Ziyi Xu, Mark Zhao, Yuwei An, Vikram Sharma Mailthody, Scott Mahlke, Michael Garland, Christos Kozyrakis
https://arxiv.org/abs/2508.18572

Strata: Hierarchical Context Caching for Long Context Language Model Serving
Large Language Models (LLMs) with expanding context windows face significant performance hurdles. While caching key-value (KV) states is critical for avoiding redundant computation, the storage footprint of long-context caches quickly exceeds GPU memory capacity, forcing production systems to adopt hierarchical caching across memory hierarchies. However, transferring large cached contexts back to the GPU introduces severe performance bottlenecks: fragmented I/O from paged layouts prevents full …

TokenLake: A Unified Segment-level Prefix Cache Pool for Fine-grained Elastic Long-Context LLM Serving
Bingyang Wu, Zili Zhang, Yinmin Zhong, Guanzhe Huang, Yibo Zhu, Xuanzhe Liu, Xin Jin
https://arxiv.org/abs/2508.17219

TokenLake: A Unified Segment-level Prefix Cache Pool for Fine-grained Elastic Long-Context LLM Serving
Prefix caching is crucial to accelerate multi-turn interactions and requests with shared prefixes. At the cluster level, existing prefix caching systems are tightly coupled with request scheduling to optimize cache efficiency and computation performance together, leading to load imbalance, data redundancy, and memory fragmentation of caching systems across instances. To address these issues, memory pooling is promising to shield the scheduler from the underlying cache management so that it can …

Predictable LLM Serving on GPU Clusters
Erfan Darzi, Shreeanant Bharadwaj, Sree Bhargavi Balija
https://arxiv.org/abs/2508.20274 https://arxiv.org/pdf/2508…

Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs
Xing Chen, Rong Shi, Lu Zhao, Lingbin Wang, Xiao Jin, Yueqiang Chen, Hongfeng Sun
https://arxiv.org/abs/2509.17542

Disaggregated Prefill and Decoding Inference System for Large Language Model Serving on Multi-Vendor GPUs
LLM-based applications have been widely used in various industries, but with the increasing of models size, an efficient large language model (LLM) inference system is an urgent problem to be solved for service providers. Since the inference system is divided into two stage with different characteristics: Prefill and Decode, the two stage will interfere with each other during the inference process. Toward this end, a P-D disaggregated inference framework is proposed by some researchers. Current…

ShadowServe: Interference-Free KV Cache Fetching for Distributed Prefix Caching
Xingyu Xiang, Raj Joshi, Yuhan Liu, Jiayi Yao, Chenxingyu Zhao, Junchen Jiang, Yang Zhou, Eddie Kohler, Minlan Yu
https://arxiv.org/abs/2509.16857

ShadowServe: Interference-Free KV Cache Fetching for Distributed Prefix Caching
Distributed prefix caching accelerates long-context LLM serving by reusing KV cache entries for common context prefixes. However, KV cache fetches can become a bottleneck when network bandwidth is limited. Compression mitigates the bandwidth issue, but can degrade overall performance when decompression interferes with model computation. We present ShadowServe, the first SmartNIC-accelerated, interference-free prefix caching system for LLM serving. ShadowServe separates a control plane on the …

Taming the Chaos: Coordinated Autoscaling for Heterogeneous and Disaggregated LLM Inference
Rongzhi Li, Ruogu Du, Zefang Chu, Sida Zhao, Chunlei Han, Zuocheng Shi, Yiwen Shao, Huanle Han, Long Huang, Zherui Liu, Shufan Liu
https://arxiv.org/abs/2508.19559

Taming the Chaos: Coordinated Autoscaling for Heterogeneous and Disaggregated LLM Inference
Serving Large Language Models (LLMs) is a GPU-intensive task where traditional autoscalers fall short, particularly for modern Prefill-Decode (P/D) disaggregated architectures. This architectural shift, while powerful, introduces significant operational challenges, including inefficient use of heterogeneous hardware, network bottlenecks, and critical imbalances between prefill and decode stages. We introduce HeteroScale, a coordinated autoscaling framework that addresses the core challenges of …

FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters
Yanying Lin, Shijie Peng, Chengzhi Lu, Chengzhong Xu, Kejiang Ye
https://arxiv.org/abs/2510.11938

FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters
Serving Large Language Models (LLMs) in production faces significant challenges from highly variable request patterns and severe resource fragmentation in serverless clusters. Current systems rely on static pipeline configurations that struggle to adapt to dynamic workload conditions, leading to substantial inefficiencies. We present FlexPipe, a novel system that dynamically reconfigures pipeline architectures during runtime to address these fundamental limitations. FlexPipe decomposes models i…

VoltanaLLM: Feedback-Driven Frequency Control and State-Space Routing for Energy-Efficient LLM Serving
Jiahuan Yu (University of Illinois Urbana-Champaign), Aryan Taneja (University of Illinois Urbana-Champaign), Junfeng Lin (Tsinghua University), Minjia Zhang (University of Illinois Urbana-Champaign)
https://arxiv.org/abs/2509.04827

VoltanaLLM: Feedback-Driven Frequency Control and State-Space Routing for Energy-Efficient LLM Serving
Modern Large Language Model (LLM) serving systems increasingly support interactive applications, like real-time chat assistants, code generation tools, and agentic workflows. However, the soaring energy cost of LLM inference presents a growing challenge for sustainable and cost-effective deployment. This paper introduces VoltanaLLM, a system for SLO-aware, energy-efficient LLM serving, built from a control theory perspective. VoltanaLLM co-designs frequency scaling and request routing in emergi…

Hetis: Serving LLMs in Heterogeneous GPU Clusters with Fine-grained and Dynamic Parallelism
Zizhao Mo, Jianxiong Liao, Huanle Xu, Zhi Zhou, Chengzhong Xu
https://arxiv.org/abs/2509.08309

Hetis: Serving LLMs in Heterogeneous GPU Clusters with Fine-grained and Dynamic Parallelism
The significant resource demands in LLM serving prompts production clusters to fully utilize heterogeneous hardware by partitioning LLM models across a mix of high-end and low-end GPUs. However, existing parallelization approaches often struggle to scale efficiently in heterogeneous environments due to their coarse-grained and static parallelization strategies. In this paper, we introduce Hetis, a new LLM system tailored for heterogeneous GPU clusters. Hetis addresses two critical challenges:…

Block: Balancing Load in LLM Serving with Context, Knowledge and Predictive Scheduling
Wei Da, Evangelia Kalyvianaki
https://arxiv.org/abs/2508.03611 https://

Block: Balancing Load in LLM Serving with Context, Knowledge and Predictive Scheduling
This paper presents Block, a distributed scheduling framework designed to optimize load balancing and auto-provisioning across instances in large language model serving frameworks by leveraging contextual information from incoming requests. Unlike popular model serving systems that rely on monolithic and heuristic task schedulers, Block operates as a fully distributed, stateless, and predictive scheduling system to achieve low overhead, reliability, and scalability. It leverages the determinist…

Kairos: Low-latency Multi-Agent Serving with Shared LLMs and Excessive Loads in the Public Cloud
Jinyuan Chen, Jiuchen Shi, Quan Chen, Minyi Guo
https://arxiv.org/abs/2508.06948

Kairos: Low-latency Multi-Agent Serving with Shared LLMs and Excessive Loads in the Public Cloud
Multi-agent applications utilize the advanced capabilities of large language models (LLMs) for intricate task completion through agent collaboration in a workflow. Under this situation, requests from different agents usually access the same shared LLM to perform different kinds of tasks, forcing the shared LLM to suffer excessive loads. However, existing works have low serving performance for these multi-agent applications, mainly due to the ignorance of inter-agent latency and resource differe…