Tootfinder

(1/2) Preston Thorpe Is a Software Engineer at a San Francisco Startup. He’s Also Serving His 11th Year in Prison. (TechCrunch, 24 July 2025)
https://techcrunch.com/2025/07/24/preston-thorpe-is-a-software…

Preston Thorpe is a software engineer at a San Francisco startup — he’s also serving his 11th year in prison | TechCrunch
A senior software engineer for Turso, Thorpe is part of an experimental program in the Maine state prison system that allows incarcerated people to work remote jobs from custody.

PPipe: Efficient Video Analytics Serving on Heterogeneous GPU Clusters via Pool-Based Pipeline Parallelism
Z. Jonny Kong, Qiang Xu, Y. Charlie Hu
https://arxiv.org/abs/2507.18748

PPipe: Efficient Video Analytics Serving on Heterogeneous GPU Clusters via Pool-Based Pipeline Parallelism
With the rapid innovation of GPUs, heterogeneous GPU clusters in both public clouds and on-premise data centers have become increasingly commonplace. In this paper, we demonstrate how pipeline parallelism, a technique wellstudied for throughput-oriented deep learning model training, can be used effectively for serving latency-bound model inference, e.g., in video analytics systems, on heterogeneous GPU clusters. Our work exploits the synergy between diversity in model layers and diversity in GP…

Synthesia says it has over 65K customers and serves more than 70% of the Fortune 100, with its AI avatars mainly used for training and internal communications (Allie Garfinkle/Fortune)
https://fortune.com/2025/06/26/ai-digital-avatars-corporate-vi…

AI avatars are here in full force—and they’re serving some of the world’s biggest companies
Digital avatars make training and internal comms videos much easier and cheaper to produce.

When the package suggests some fancy way of serving, but you get away with putting it on top of a slice of bread.

N.C. Has Allowed a Likely Carcinogen Into Three Rivers Serving 900,000 People - Inside Climate News
https://insideclimatenews.org/news/29062025/north-carolina-plastics-company-dumping-likely-carcinogen-into-rivers/

On the rehabilitative power of being able to work from prison.
https://techcrunch.com/2025/07/24/preston-thorpe-is-a-software-engineer-at-a-san-francisco-startup-hes-also-serving-his-11th-year-in-prison/

Preston Thorpe is a software engineer at a San Francisco startup — he’s also serving his 11th year in prison | TechCrunch
A senior software engineer for Turso, Thorpe is part of an experimental program in the Maine state prison system that allows incarcerated people to work remote jobs from custody.

Internal docs: Scale AI's efforts to train Google's Gemini were flooded with "spammy behavior" from unqualified independent contractors submitting shoddy work (Sam Blum/Inc)
https://www.inc.com/sam-blum/exclusive-sca

Exclusive: Scale AI's Spam, Security Woes Plagued the Company While Serving Google
How the startup that just scored a $14 billion investment from Meta struggled to contain 'spammy behavior' from unqualified contributors as it trained Gemini.

Adaptive Request Scheduling for CodeLLM Serving with SLA Guarantees
Shi Chang, Boyuan Chen, Kishanthan Thangarajah, Hanan Lutfiyya, Ahmed E. Hassan
https://arxiv.org/abs/2506.19677

Adaptive Request Scheduling for CodeLLM Serving with SLA Guarantees
Code Large Language Models (CodeLLMs) are increasingly integrated into modern software development workflows, yet efficiently serving them in resource-constrained, self-hosted environments remains a significant challenge. Existing LLM serving systems employs Continuous Batching for throughput improvement. However, they rely on static batch size configurations that cannot adapt to fluctuating request rates or heterogeneous workloads, leading to frequent SLA (Service Level Agreement) violations a…

In the hands of someone like Trump,
deals are ways to evade, postpone or subvert the efficient work of markets.
Trump does not like markets,
precisely because they are impersonal and objective.
Their results – for corporations, entrepreneurs, investors and shareholders – are subject to clear measures of success and failure.
Because deals are personal, adversarial and incomplete,
they are perfect grist for Trump’s relentless publicity machine,
and a…

This https://arxiv.org/abs/2504.14775 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling
Pipeline parallelism has emerged as a predominant approach for deploying large language models (LLMs) across distributed nodes, owing to its lower communication overhead compared to tensor parallelism. While demonstrating high throughput in request serving, pipeline parallelism often suffers from performance limitations caused by pipeline bubbles, which are primarily resulted from imbalanced computation delays across batches. Existing methods like Sarathi-Serve attempt to address this through h…

Probably few noticed, but it looks like mirrors.cpan.org might have stopped serving web requests earlier this month. It was being phased out awhile ago AFAIK, but I'm sure there are plenty of dead links out there now.
Last wayback copy of the page is here: https://web.archive.org/web/2025071102

Handoff Design in User-Centric Cell-Free Massive MIMO Networks Using DRL
Hussein A. Ammar, Raviraj Adve, Shahram Shahbazpanahi, Gary Boudreau, Israfil Bahceci
https://arxiv.org/abs/2507.20966

Handoff Design in User-Centric Cell-Free Massive MIMO Networks Using DRL
In the user-centric cell-free massive MIMO (UC-mMIMO) network scheme, user mobility necessitates updating the set of serving access points to maintain the user-centric clustering. Such updates are typically performed through handoff (HO) operations; however, frequent HOs lead to overheads associated with the allocation and release of resources. This paper presents a deep reinforcement learning (DRL)-based solution to predict and manage these connections for mobile users. Our solution employs th…

Sandwich: Separating Prefill-Decode Compilation for Efficient CPU LLM Serving
Juntao Zhao, Jiuru Li, Chuan Wu
https://arxiv.org/abs/2507.18454 https://arxi…

Sandwich: Separating Prefill-Decode Compilation for Efficient CPU LLM Serving
Utilizing CPUs to serve large language models (LLMs) is a resource-friendly alternative to GPU serving. Existing CPU-based solutions ignore workload differences between the prefill and the decode phases of LLM inference, applying a static per-NUMA (Non-Uniform Memory Access) node model partition and utilizing vendor libraries for operator-level execution, which is suboptimal. We propose Sandwich, a hardware-centric CPU-based LLM serving engine that uses different execution plans for the prefill…

Toward Dual-Functional LAWN: Control-Aware System Design for Aerodynamics-Aided UAV Formations
Jun Wu, Weijie Yuan, Qingqing Cheng, Haijia Jin
https://arxiv.org/abs/2507.19910 h…

Toward Dual-Functional LAWN: Control-Aware System Design for Aerodynamics-Aided UAV Formations
Integrated sensing and communication (ISAC) has emerged as a pivotal technology for advancing low-altitude wireless networks (LAWNs), serving as a critical enabler for next-generation communication systems. This paper investigates the system design for energy-saving unmanned aerial vehicle (UAV) formations in dual-functional LAWNs, where a ground base station (GBS) simultaneously wirelessly controls multiple UAV formations and performs sensing tasks. To enhance flight endurance, we exploit the …

An Explainable Equity-Aware P2P Energy Trading Framework for Socio-Economically Diverse Microgrid
Abhijan Theja, Mayukha Pal
https://arxiv.org/abs/2507.18738 https://

An Explainable Equity-Aware P2P Energy Trading Framework for Socio-Economically Diverse Microgrid
Fair and dynamic energy allocation in community microgrids remains a critical challenge, particularly when serving socio-economically diverse participants. Static optimization and cost-sharing methods often fail to adapt to evolving inequities, leading to participant dissatisfaction and unsustainable cooperation. This paper proposes a novel framework that integrates multi-objective mixed-integer linear programming (MILP), cooperative game theory, and a dynamic equity-adjustment mechanism driven…

A New Approach to the Construction of Subdivision Algorithms
Alexander Dietz
https://arxiv.org/abs/2507.20897 https://arxiv.org/pdf/2507.20897

A New Approach to the Construction of Subdivision Algorithms
In this thesis, a new approach for constructing subdivision algorithms for generalized quadratic and cubic B-spline subdivision for subdivision surfaces and volumes is presented. First, a catalog of quality criteria for these subdivision algorithms is developed, serving as a guideline for the construction process. The construction begins by generating the desired subdominant eigenvectors as the vertices of regular convex 3-polytopes for volumes using circle packings. Subsequently, these polyt…

Leveraging GANs for citation intent classification and its impact on citation network analysis
Davi A. Bezerra, Filipi N. Silva, Diego R. Amancio
https://arxiv.org/abs/2505.21162 …

Leveraging GANs for citation intent classification and its impact on citation network analysis
Citations play a fundamental role in the scientific ecosystem, serving as a foundation for tracking the flow of knowledge, acknowledging prior work, and assessing scholarly influence. In scientometrics, they are also central to the construction of quantitative indicators. Not all citations, however, serve the same function: some provide background, others introduce methods, or compare results. Therefore, understanding citation intent allows for a more nuanced interpretation of scientific impact…

Sparrow, which uses AI to consolidate fragmented leave management processes, raised a $35M Series B led by Silver Lake Waterman, for $64M in total funding (Michael Nuñez/VentureBeat)
https://venturebeat.com/ai/sparrow-raises-35m-ser…

Sparrow raises $35M Series B to automate the employee leave management nightmare
Sparrow raises $35M Series B to scale AI-powered employee leave management platform that has grown 14x since 2021, serving 1,000+ companies and saving $200M in payroll costs.

Ad-Shield: 976M people globally use ad blockers that block on-site analytics, whitelisted ads, registration walls, and more, up from 590M in 2019 (Charlotte Tobitt/Press Gazette)
https://pressgazette.co.uk/marketing/adblockers-stop…

Adblockers stop publishers serving ads to (or even seeing) 1bn web users
Dark traffic' up by 49% in three years, according to new study.

I will sit here.
And there.
And any.
And every where.
Destabilizing work site be d’m’d- I will devote myself to serving the American people.
By continuing my work on sorting out and prepping for extreme weather across the continental USA
Every single one of my colleagues is just as dedicated.
Why am I doing research ‘in exile’?
If it’s a test of my commitment to my mission: bring it.
https://bsky.app/profile/drkatemarvel.bsky.social/post/3lpbwem2yvk2o

How #Vietnam bets on serving the #Trump organization. Despicable, including land evictions—not particularly a thing the Vietnamese people fancy.

Serving coffee on aircraft causes turbulence.

"For me, our job as artists is to serve the story, serve the director, and serve the fellow actors. And if you do that, by osmosis you’re serving yourself because you’ll get the best out of yourself."
-David Oyelowo
#acting #coaching

Unlock the Potential of Fine-grained LLM Serving via Dynamic Module Scaling
Jingfeng Wu, Yiyuan He, Minxian Xu, Xitong Gao, Kejiang Ye, Chengzhong Xu
https://arxiv.org/abs/2507.18006

Unlock the Potential of Fine-grained LLM Serving via Dynamic Module Scaling
The rise of large language models (LLMs) has created new opportunities across various fields but has also introduced significant challenges in resource management. Current LLM serving systems face a fundamental tension: balancing serving demands with limited resources while adapting to unpredictable traffic patterns. Static deployments lead to suboptimal resource utilization and performance degradation under dynamic workloads. Furthermore, the high cost of adjusting instances hinders dynamic sc…

Happy SysAdmin Day everyone! Here is a head scratcher. Website kept serving the default new site page. Even when I made a new index.html file. So I made a test.html and got Forbidden. Error log says "client denied by server configuration: /var/www/newwebsite/test.html" Not directory permissions and not selinux. So I made /var/www/newweb2 and moved the files over. Voila, working. Both test.htnl and new index.html. I'm leaving it alone because it's working and it's Friday…

weee… I can finally stop serving WebP images! 🥳
Just in time when I have to start worrying about JPEG XL. 🤪
https://caniuse.com/avif

Color me confused - I do not know what is the proper condiment to use when serving fried air.
I tried luminiferous aether, but that didn't really do much. Maybe I'll get a bottle of phlogiston.

DRPAAS: https://developers.googleblog.com/en/google-url-shortener-links-will-no-longer-be-available/
Digital Rug Pull As A Service, via

Google URL Shortener links will no longer be available- Google Developers Blog
Understand how you will be impacted by our decision to turn off the serving portion of Google URL Shortener.

BotHash: Efficient and Training-Free Bot Detection Through Approximate Nearest Neighbor
Edoardo Di Paolo, Fabio De Gaspari, Angelo Spognardi
https://arxiv.org/abs/2506.20503

BotHash: Efficient and Training-Free Bot Detection Through Approximate Nearest Neighbor
Online Social Networks (OSNs) are a cornerstone in modern society, serving as platforms for diverse content consumption by millions of users each day. However, the challenge of ensuring the accuracy of information shared on these platforms remains significant, especially with the widespread dissemination of disinformation. Social bots -- automated accounts designed to mimic human behavior, frequently spreading misinformation -- represent one of the critical problems of OSNs. The advent of Large…

"O News Program, When Art Thou On?" Using the announcement of a single serving site to teach about the history of public broadcasting in Germany, well done @…!
https://johl.io/blog/o-news-program-when-art-thou-on/

Exoplanet exploration in the context of the PLATO mission: From detection to population studies
Amadeo Castro-Gonz\'alez
https://arxiv.org/abs/2506.19898

Exoplanet exploration in the context of the PLATO mission: From detection to population studies
This thesis constitutes an observational effort to advance our understanding of planet formation and evolution across different regimes. To do so, we exploited Kepler/K2 and TESS data in combination with ground-based observations, serving as a preparation for the PLATO mission. This endeavour led to the discovery and statistical validation of four planets, the confirmation and characterisation of two other planets, the detection of 14 new planet candidates, and the revision of the properties of…

Ezra Klein: "Antisemitism is serving as a beard for an assault on the ideals and institutions that have made America into a place where Jews can flourish. If Israel becomes a right-wing ethnostate, and if opposition to that state is antisemitic, then Jews will become mascots for a politics that would have made the Jewish diaspora unthinkable."

Opinion | Why American Jews No Longer Understand One Another
The consensus that held them together for generations is breaking down.

The Evolving Homefront: Millennials Stay Put as Older Buyers Dominate the Market
Penn Mutual Asset Management - July 24, 2025
“While housing demand may soften due to economic pressures, strong rental demand is expected to continue, particularly among younger adults navigating high costs and limited financial flexibility.”

The Evolving Homefront: Millennials Stay Put as Older Buyers Dominate the Market | Penn Mutual Asset Management, LLC
Penn Mutual Asset Management is committed to serving the institutional marketplace by offering fixed income investment solutions and client-focused services

A Global Commuting Origin-Destination Flow Dataset for Urban Sustainable Development
Can Rong, Jingtao Ding, Meng Li, Yong Li
https://arxiv.org/abs/2505.17111

Point Cloud Environment-Based Channel Knowledge Map Construction
Yancheng Wang, Wei Guo, Guanying Chen, Ye Zhang, Shuguang Cui
https://arxiv.org/abs/2506.21112

Point Cloud Environment-Based Channel Knowledge Map Construction
Channel knowledge map (CKM) provides certain levels of channel state information (CSI) for an area of interest, serving as a critical enabler for environment-aware communications by reducing the overhead of frequent CSI acquisition. However, existing CKM construction schemes adopt over-simplified environment information, which significantly compromises their accuracy. To address this issue, this work proposes a joint model- and data-driven approach to construct CKM by leveraging point cloud env…

Replaced article(s) found for cs.LG. https://arxiv.org/list/cs.LG/new
[2/9]:
- Medha: Efficiently Serving Multi-Million Context Length LLM Inference Requests Without Approximat...
Agrawal, Qiu, Chen, Goiri, Zhang, Shahid, Ramjee, Tumanov, Choukse

Looks like the Goo.gl link shortener dies next month. A reminder of why link shorteners are a bad idea.
(Also note that ".ly" is owned by the country of Libya. Maybe not the most stable place to park your content behind as well).
https://developers.googleblog.co…

Google URL Shortener links will no longer be available- Google Developers Blog
Understand how you will be impacted by our decision to turn off the serving portion of Google URL Shortener.

BucketServe: Bucket-Based Dynamic Batching for Smart and Efficient LLM Inference Serving
Wanyi Zheng, Minxian Xu, Shengye Song, Kejiang Ye
https://arxiv.org/abs/2507.17120 https…

BucketServe: Bucket-Based Dynamic Batching for Smart and Efficient LLM Inference Serving
Large language models (LLMs) have become increasingly popular in various areas, traditional business gradually shifting from rule-based systems to LLM-based solutions. However, the inference of LLMs is resource-intensive or latency-sensitive, posing significant challenges for serving systems. Existing LLM serving systems often use static or continuous batching strategies, which can lead to inefficient GPU memory utilization and increased latency, especially under heterogeneous workloads. These …

Athletics celebrate groundbreaking of $1.75 billion stadium project in Las Vegas https://www.foxsports.com/articles/mlb/athletics-celebrate-groundbreaking-of-175-billion-stadium-project-in-las-vegas

NFLPA president: Interim leader vote due soon https://www.espn.com/nfl/story/_/id/45798049/nflpa-president-interim-executive-director-vote-future

Ranked choice voting (RCV)
is an increasingly popular electoral institution that has been posited by reformers and media outlets to produce transformative effects on electoral outcomes and representation.
This article finds no empirical support for the proposition that RCV changed fiscal outcomes or the ideological composition of city councils
— both on absolute terms and relative to mass opinion.
Furthermore, the roll-call based ideal points of legislators serving …

This is obviously bad from #whatsapp, but also, the way the journalist describes what the chatbot does, as if it had intentions, is pretty bad too.
"It was the beginning of a bizarre exchange of the kind more and more people are having with AI systems, in which chatbots try to negotiate their way out of trouble, deflect attention from their mistakes and contradict themselve…

I am super-pleased to be serving on the PC of the excellent and much-needed "Undone Science in CS" 2026: "to provide an opportunity to pause and reflect on the epistemological and ethical aspects of computer science". For much more information, please see:
https://www.undonecs.org/2026/…

Pickens on CeeDee Lamb, Dak Prescott, Brian Schottenheimer and 'fresh air' with Cowboys https://www.dallascowboys.com/news/pickens-on-ceedee-lamb-dak-prescott-brian-schottenheimer-and-fresh-air-with-cowboys…

From Prompts to Constructs: A Dual-Validity Framework for LLM Research in Psychology
Zhicheng Lin
https://arxiv.org/abs/2506.16697 https://

From Prompts to Constructs: A Dual-Validity Framework for LLM Research in Psychology
Large language models (LLMs) are rapidly being adopted across psychology, serving as research tools, experimental subjects, human simulators, and computational models of cognition. However, the application of human measurement tools to these systems can produce contradictory results, raising concerns that many findings are measurement phantoms--statistical artifacts rather than genuine psychological phenomena. In this Perspective, we argue that building a robust science of AI psychology require…

Good Morning #Canada
One of the most influential Canadians that most of Canada has never heard of was Elijah Harper (March 3, 1949 - May 17, 2013). He served with distinction as a politician in Manitoba, serving as minister of Northern Affairs. Harper is best known for the role he played in scuttling the Meech Lake Accord, for which he was named the Canadian Press newsmaker of the year in 1990. The Meech Lake Accord was an attempt by then Prime Minister Mulroney to gain acceptance of the constitution by Quebec. The proposed amendments, negotiated in secret by the 11 Premiers, would have eroded federal powers by granting the provinces and territories more rights over key areas.
Without Harper's stubbornness, Canada’s federal powers would have been weakened, and we very well could have ended up splintered like the USA.
#CanadaIsAwesome #CanadianHeroes
https://www.thecanadianencyclopedia.ca/en/article/elijah-harper

On the Power of Spatial Locality on Online Routing Problems
Swapnil Guragain, Gokarna Sharma
https://arxiv.org/abs/2506.17517 https://

On the Power of Spatial Locality on Online Routing Problems
We consider the online versions of two fundamental routing problems, traveling salesman (TSP) and dial-a-ride (DARP), which have a variety of relevant applications in logistics and robotics. The online versions of these problems concern with efficiently serving a sequence of requests presented in a real-time on-line fashion located at points of a metric space by servers (salesmen/vehicles/robots). In this paper, motivated from real-world applications, such as Uber/Lyft rides, where some limited…

Cloud Native System for LLM Inference Serving
Minxian Xu, Junhan Liao, Jingfeng Wu, Yiyuan He, Kejiang Ye, Chengzhong Xu
https://arxiv.org/abs/2507.18007 https://

Cloud Native System for LLM Inference Serving
Large Language Models (LLMs) are revolutionizing numerous industries, but their substantial computational demands create challenges for efficient deployment, particularly in cloud environments. Traditional approaches to inference serving often struggle with resource inefficiencies, leading to high operational costs, latency issues, and limited scalability. This article explores how Cloud Native technologies, such as containerization, microservices, and dynamic scheduling, can fundamentally impr…

Relativistic Calculations of Energy Levels, Field Shift Factors, and Polarizabilities of Mercury and Copernicium
Hongxu Liu, Jize Han, Yanmei Yu, Yanfeng Ge, Yong Liu, Zhiguo Huang
https://arxiv.org/abs/2507.18490 https://arxiv.org/pdf/2507.18490 https://arxiv.org/html/2507.18490
arXiv:2507.18490v1 Announce Type: new
Abstract: Mercury (Hg) and superheavy element copernicium (Cn) are investigated using equation-of-motion relativistic coupled-cluster (EOM-RCC) and configuration interaction plus many-body perturbation theory (CI MBPT) methods. Key atomic properties including ionization potentials (IP), excitation energies (EEs), isotope field shift factors (F), and static electric dipole polarizabilities ({\alpha}) are calculated for ground and low-lying excited states. To evaluate the theoretical accuracy, calculations for both Hg and Cn are performed, with experimental data of Hg serving as benchmarks. Furthermore, basis set dependence has been systematically evaluated in the EOM-RCC calculations, with corresponding uncertainty estimates having been provided. The calculated atomic properties could provide valuable insights into the electronic structure and chemical behavior of superheavy elements.
toXiv_bot_toot

Real-Time Communication-Aware Ride-Sharing Route Planning for Urban Air Mobility: A Multi-Source Hybrid Attention Reinforcement Learning Approach
Yuejiao Xie, Maonan Wang, Di Zhou, Man-On Pun, Zhu Han
https://arxiv.org/abs/2507.14249

Real-Time Communication-Aware Ride-Sharing Route Planning for Urban Air Mobility: A Multi-Source Hybrid Attention Reinforcement Learning Approach
Urban Air Mobility (UAM) systems are rapidly emerging as promising solutions to alleviate urban congestion, with path planning becoming a key focus area. Unlike ground transportation, UAM trajectory planning has to prioritize communication quality for accurate location tracking in constantly changing environments to ensure safety. Meanwhile, a UAM system, serving as an air taxi, requires adaptive planning to respond to real-time passenger requests, especially in ride-sharing scenarios where pas…

Robustness of analogue Hawking radiation in cavities with moving boundaries
Alberto Garc\'ia Mart\'in-Caro, Javier Olmedo, Jose M. S\'anchez Vel\'azquez
https://arxiv.org/abs/2507.13894

Robustness of analogue Hawking radiation in cavities with moving boundaries
In this work we explore the limitations and robustness of thermal radiation in dynamical Casimir systems serving as analogs for Hawking radiation. Through detailed numerical analysis, we characterize particle production spectra in cavities with moving boundaries under various configurations, including expanding, collapsing, and rigidly accelerating scenarios. We find that thermal signatures emerge in specific expanding cavity configurations but are highly dependent on frequency bands and accele…

"Asked about Zuckerberg’s claim that the AI was “the most intelligent”, Gray said: “That has definitely been thrown into doubt in this instance.”"
But possibly (probably?) it is the most intelligent #AI - after all, "most" and "best" are relative terms. It doesn't mean it is good, or reliable, or dependable, or reproducible. Just not as bad as others.
And f…

"Asked about Zuckerberg’s claim that the AI was “the most intelligent”, Gray said: “That has definitely been thrown into doubt in this instance.”"
But possibly (probably?) it is the most intelligent #AI - after all, "most" and "best" are relative terms. It doesn't mean it is good, or reliable, or dependable, or reproducible. Just not as bad as others.
And f…

MIRAGE: KV Cache Optimization through Parameter Remapping for Multi-tenant LLM Serving
Ruihao Li, Shagnik Pal, Vineeth Narayan Pullu, Prasoon Sinha, Jeeho Ryoo, Lizy K. John, Neeraja J. Yadwadkar
https://arxiv.org/abs/2507.11507

MIRAGE: KV Cache Optimization through Parameter Remapping for Multi-tenant LLM Serving
KV cache accelerates LLM inference by avoiding redundant computation, at the expense of memory. To support larger KV caches, prior work extends GPU memory with CPU memory via CPU-offloading. This involves swapping KV cache between GPU and CPU memory. However, because the cache updates dynamically, such swapping incurs high CPU memory traffic. We make a key observation that model parameters remain constant during runtime, unlike the dynamically updated KV cache. Building on this, we introduce MI…

RankMixer: Scaling Up Ranking Models in Industrial Recommenders
Jie Zhu, Zhifang Fan, Xiaoxie Zhu, Yuchen Jiang, Hangyu Wang, Xintian Han, Haoran Ding, Xinmin Wang, Wenlin Zhao, Zhen Gong, Huizhi Yang, Zheng Chai, Zhe Chen, Yuchao Zheng, Qiwei Chen, Feng Zhang, Xun Zhou, Peng Xu, Xiao Yang, Di Wu, Zuotao Liu
https://arxiv.org/ab…

RankMixer: Scaling Up Ranking Models in Industrial Recommenders
Recent progress on large language models (LLMs) has spurred interest in scaling up recommendation systems, yet two practical obstacles remain. First, training and serving cost on industrial Recommenders must respect strict latency bounds and high QPS demands. Second, most human-designed feature-crossing modules in ranking models were inherited from the CPU era and fail to exploit modern GPUs, resulting in low Model Flops Utilization (MFU) and poor scalability. We introduce RankMixer, a hardware…

PolyServe: Efficient Multi-SLO Serving at Scale
Kan Zhu, Haiyang Shi, Le Xu, Jiaxin Shan, Arvind Krishnamurthy, Baris Kasikci, Liguang Xie
https://arxiv.org/abs/2507.17769 https…

PolyServe: Efficient Multi-SLO Serving at Scale
Advances in Large Language Models (LLMs) have led to a surge of LLM-powered applications. These applications have diverse token-generation latency requirements. As a result, simply classifying workloads as latency-sensitive (LS) or best-effort (BE) overlooks the nuances within the latency-sensitive category and results in suboptimal user experiences and scheduling opportunities. However, efficiently serving requests with multiple SLO requirements poses significant challenges. First, all request…

The Rest is Silence: Leveraging Unseen Species Models for Computational Musicology
Fabian C. Moss, Jan Haji\v{c} jr., Adrian Nachtwey, Laurent Pugin
https://arxiv.org/abs/2507.14638

The Rest is Silence: Leveraging Unseen Species Models for Computational Musicology
For many decades, musicologists have engaged in creating large databases serving different purposes for musicological research and scholarship. With the rise of fields like music information retrieval and digital musicology, there is now a constant and growing influx of musicologically relevant datasets and corpora. In historical or observational settings, however, these datasets are necessarily incomplete, and the true extent of a collection of interest remains unknown -- silent. Here, we appl…

But this makes me happy.
(My Iocaine mostly generates garbage based on Karl Marx' "Capital", I might add some more stuff in there)

This https://arxiv.org/abs/2506.03940 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCR_…

Depermissioning Web3: a Permissionless Accountable RPC Protocol for Blockchain Networks
In blockchain networks, so-called "full nodes" serve data to and relay transactions from clients through an RPC interface. This serving layer enables integration of "Web3" data, stored on blockchains, with "Web2" mobile or web applications that cannot directly participate as peers in a blockchain network. In practice, the serving layer is dominated by a small number of centralized services ("node providers") that offer permissioned access to RPC endpoints. Clients register with these providers …

When serving shaved ice
One simply must remember:
Syrup, not shave cream.
#dailyhaikuprompt - shaved ice
#haiku
#poem

Rain gardens are a solution to heavy rainfalls and flooding in gardens. Native plants save time, money, and water while benefiting ecosystems. #climatechange #climatesolutions #climate

Homeowner shares satisfying video of newly installed yard feature serving critical purpose: 'Great work'
One Reddit user showed off their incredible rain garden and how it prevents lawn flooding during heavy rainfall.

Enhanced Absorption in Thin-Film Silicon Solar Cells Using a Broadband Plasmonic Nanostructure
Partha Mondal, Omar Alkhazragi, Hakan Bagci
https://arxiv.org/abs/2506.16118

Enhanced Absorption in Thin-Film Silicon Solar Cells Using a Broadband Plasmonic Nanostructure
The design and fabrication of a metal-dielectric-metal absorber that achieves strong absorption from the ultraviolet (UV) to the near-infrared (near-IR) spectrum are presented. The proposed nanostructure consists of a periodic titanium (Ti) array as the top layer, a thin silicon dioxide (SiO2) spacer, and a continuous aluminum (Al) layer serving as the back reflector. Comprehensive optimization of structural parameters results in an average absorptance of 96% in the 280-1000 nm wavelength range…

"For me, our job as artists is to serve the story, serve the director, and serve the fellow actors. And if you do that, by osmosis you’re serving yourself because you’ll get the best out of yourself."
-David Oyelowo
#acting #coaching

Context-Aware CodeLLM Eviction for AI-assisted Coding
Kishanthan Thangarajah, Boyuan Chen, Shi Chang, Ahmed E. Hassan
https://arxiv.org/abs/2506.18796 http…

Context-Aware CodeLLM Eviction for AI-assisted Coding
AI-assisted coding tools powered by Code Large Language Models (CodeLLMs) are increasingly integrated into modern software development workflows. To address concerns around privacy, latency, and model customization, many enterprises opt to self-host these models. However, the diversity and growing number of CodeLLMs, coupled with limited accelerator memory, introduce practical challenges in model management and serving efficiency. This paper presents CACE, a novel context-aware model eviction s…

Pimba: A Processing-in-Memory Acceleration for Post-Transformer Large Language Model Serving
Wonung Kim, Yubin Lee, Yoonsung Kim, Jinwoo Hwang, Seongryong Oh, Jiyong Jung, Aziz Huseynov, Woong Gyu Park, Chang Hyun Park, Divya Mahajan, Jongse Park
https://arxiv.org/abs/2507.10178

Pimba: A Processing-in-Memory Acceleration for Post-Transformer Large Language Model Serving
Transformers are the driving force behind today's Large Language Models (LLMs), serving as the foundation for their performance and versatility. Yet, their compute and memory costs grow with sequence length, posing scalability challenges for long-context inferencing. In response, the algorithm community is exploring alternative architectures, such as state space models (SSMs), linear attention, and recurrent neural networks (RNNs), which we refer to as post-transformers. This shift presents a k…

Large Language Models as Psychological Simulators: A Methodological Guide
Zhicheng Lin
https://arxiv.org/abs/2506.16702 https://arxiv…

Large Language Models as Psychological Simulators: A Methodological Guide
Large language models (LLMs) offer emerging opportunities for psychological and behavioral research, but methodological guidance is lacking. This article provides a framework for using LLMs as psychological simulators across two primary applications: simulating roles and personas to explore diverse contexts, and serving as computational models to investigate cognitive processes. For simulation, we present methods for developing psychologically grounded personas that move beyond demographic cate…

A Joint Planning Model for Fixed and Mobile Electric Vehicle Charging Stations Considering Flexible Capacity Strategy
Zhe Yu, Xue Hu, Qin Wang
https://arxiv.org/abs/2507.17587 h…

A Joint Planning Model for Fixed and Mobile Electric Vehicle Charging Stations Considering Flexible Capacity Strategy
The widespread adoption of electric vehicles (EVs) has significantly increased demand on both transportation and power systems, posing challenges to their stable operation. To support the growing need for EV charging, both fixed charging stations (FCSs) and mobile charging stations (MCSs) have been introduced, serving as key interfaces between the power grid and traffic network. Recognizing the importance of collaborative planning across these sectors, this paper presents a two-stage joint plan…

Toward a Lightweight and Robust Design for Caching with Predictions
Peng Chen, Hailiang Zhao, Jiaji Zhang, Xueyan Tang, Yixuan Wang, Shuiguang Deng
https://arxiv.org/abs/2507.16242

Toward a Lightweight and Robust Design for Caching with Predictions
The online caching problem aims to minimize cache misses when serving a sequence of requests under a limited cache size. While naive learning-augmented caching algorithms achieve ideal $1$-consistency, they lack robustness guarantees. Existing robustification methods either sacrifice $1$-consistency or introduce significant computational overhead. In this paper, we introduce \textsc{Guard}, a lightweight robustification framework that enhances the robustness of a broad class of learning-augment…

BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
Jianmin Hu, Minxian Xu, Kejiang Ye, Chengzhong Xu
https://arxiv.org/abs/2507.17133 https://

BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
In recent years, the Mixture-of-Experts (MoE) architecture has been widely applied to large language models (LLMs), providing a promising solution that activates only a subset of the model's parameters during computation, thereby reducing overall memory requirements and allowing for faster inference compared to dense models. Despite these advantages, existing systems still face issues of low efficiency due to static model placement and lack of dynamic workloads adaptation. This leads to subopti…

Potrero Yard working group has an open seat for a representative of a senior-serving organization, if that’s you or someone you know.
The group advises SFMTA on the bus yard modernization housing development on the Mission/Potrero Hill border
https://www.sfmta.com/projects/potrero-yar

Potrero Yard Modernization Project
Rebuilding Potrero Yard to better serve you!

Henry Ruggs III apologizes to family of woman he killed in drunken car crash https://www.nytimes.com/athletic/6434749/2025/06/18/henry-ruggs-apology-dui-car-crash/

The Foundation Cracks: A Comprehensive Study on Bugs and Testing Practices in LLM Libraries
Weipeng Jiang, Xiaoyu Zhang, Xiaofei Xie, Jiongchi Yu, Yuhan Zhi, Shiqing Ma, Chao Shen
https://arxiv.org/abs/2506.12320

The Foundation Cracks: A Comprehensive Study on Bugs and Testing Practices in LLM Libraries
Large Language Model (LLM) libraries have emerged as the foundational infrastructure powering today's AI revolution, serving as the backbone for LLM deployment, inference optimization, fine-tuning, and production serving across diverse applications. Despite their critical role in the LLM ecosystem, these libraries face frequent quality issues and bugs that threaten the reliability of AI systems built upon them. To address this knowledge gap, we present the first comprehensive empirical investig…

Report: Google's move from Samsung to TSMC for the Tensor G5 chip in the upcoming Pixel 10 series was a "shock" and a wake-up call for Samsung's foundry efforts (Ben Schoon/9to5Google)
https://9to5google.com/2025/06/18/google-tsm…

Report: Google’s switch to TSMC for Pixel 10 chip was a ‘shock’ and wake-up call for Samsung
According to a report, Google's shift to TSMC for the Pixel 10's Tensor G5 chip was a "shock" to Samsung and is serving as a wake-up call.

Internal memo: Adam Mosseri says Meta has named long time executive Connor Hayes as head of Threads, starting in mid-September; Hayes joined Meta in 2011 (Sara Fischer/Axios)
https://www.axios.com/2025/07/17/meta-threads-instagram-connor-hayes

Exclusive: Meta names Connor Hayes head of Threads
Hayes takes on the role after serving as Meta's vice president of product for generative AI.

The Technical Debt Gamble: A Case Study on Technical Debt in a Large-Scale Industrial Microservice Architecture
Klara Borowa, Andrzej Ratkowski, Roberto Verdecchia
https://arxiv.org/abs/2506.16214

The Technical Debt Gamble: A Case Study on Technical Debt in a Large-Scale Industrial Microservice Architecture
Microservice architectures provide an intuitive promise of high maintainability and evolvability due to loose coupling. However, these quality attributes are notably vulnerable to technical debt (TD). Few studies address TD in microservice systems, particularly on a large scale. This research explores how TD manifests in a large-scale microservice-based industrial system. The research is based on a mixed-method case study of a project including over 100 microservices and serving over 15k locati…

PolyServe: Efficient Multi-SLO Serving at Scale
Kan Zhu, Haiyang Shi, Le Xu, Jiaxin Shan, Arvind Krishnamurthy, Baris Kasikci, Liguang Xie
https://arxiv.org/abs/2507.17769 https…

Cloud Native System for LLM Inference Serving
Minxian Xu, Junhan Liao, Jingfeng Wu, Yiyuan He, Kejiang Ye, Chengzhong Xu
https://arxiv.org/abs/2507.18007 https://

Socio-Technical Smell Dynamics in Code Samples: A Multivocal Review on Emergence, Evolution, and Co-Occurrence
Arthur Bueno, Bruno Cafeo, Maria Cagnin, Awdren Font\~ao
https://arxiv.org/abs/2507.13481

Socio-Technical Smell Dynamics in Code Samples: A Multivocal Review on Emergence, Evolution, and Co-Occurrence
Code samples play a pivotal role in open-source ecosystems (OSSECO), serving as lightweight artifacts that support knowledge transfer, onboarding, and framework adoption. Despite their instructional relevance, these samples are often governed informally, with minimal review and unclear ownership, which increases their exposure to socio-technical degradation. In this context, the co-occurrence and longitudinal interplay of code smells (e.g., large classes, poor modularity) and community smells (…

Unlock the Potential of Fine-grained LLM Serving via Dynamic Module Scaling
Jingfeng Wu, Yiyuan He, Minxian Xu, Xitong Gao, Kejiang Ye, Chengzhong Xu
https://arxiv.org/abs/2507.18006

The US Army Reserve launches Detachment 201, an innovation corps with Palantir CTO Shyam Sankar and Meta CTO Andrew Bosworth serving as lieutenant colonels (Heather Somerville/Wall Street Journal)
https://www.wsj.com/tech/army-reserve-tech-ex…

DDiT: Dynamic Resource Allocation for Diffusion Transformer Model Serving
Heyang Huang, Cunchen Hu, Jiaqi Zhu, Ziyuan Gao, Liangliang Xu, Yizhou Shan, Yungang Bao, Sun Ninghui, Tianwei Zhang, Sa Wang
https://arxiv.org/abs/2506.13497

DDiT: Dynamic Resource Allocation for Diffusion Transformer Model Serving
The Text-to-Video (T2V) model aims to generate dynamic and expressive videos from textual prompts. The generation pipeline typically involves multiple modules, such as language encoder, Diffusion Transformer (DiT), and Variational Autoencoders (VAE). Existing serving systems often rely on monolithic model deployment, while overlooking the distinct characteristics of each module, leading to inefficient GPU utilization. In addition, DiT exhibits varying performance gains across different resoluti…

Canary, which offers hotel guest management tools, raised an $80M Series D led by Brighton Park Capital, valuing the company at about $600M (Dominic-Madori Davis/TechCrunch)
https://techcrunch.com/2025/06/12/hotel-mana…

Hotel management platform Canary nabs $80M Series D from BPC, YC, Insight Partners | TechCrunch
Canary offers a product suite that encompasses every part of the guest journey, from booking to departure. The company offers services that let hotel guests check in on mobile devices or text the front desk for help. And it's leaning into AI in serving its customers’ guests through voice, web, and text with an LLM, sometimes offering instant responses.

FCPO: Federated Continual Policy Optimization for Real-Time High-Throughput Edge Video Analytics
Lucas Liebe, Thanh-Tung Nguyen, Dongman Lee
https://arxiv.org/abs/2507.18047 htt…

FCPO: Federated Continual Policy Optimization for Real-Time High-Throughput Edge Video Analytics
The growing complexity of Edge Video Analytics (EVA) facilitates new kind of intelligent applications, but creates challenges in real-time inference serving systems. State-of-the-art (SOTA) scheduling systems optimize global workload distributions for heterogeneous devices but often suffer from extended scheduling cycles, leading to sub-optimal processing in rapidly changing Edge environments. Local Reinforcement Learning (RL) enables quick adjustments between cycles but faces scalability, know…

Efficient and Workload-Aware LLM Serving via Runtime Layer Swapping and KV Cache Resizing
Zhaoyuan Su, Tingfeng Lan, Zirui Wang, Juncheng Yang, Yue Cheng
https://arxiv.org/abs/2506.02006

Efficient and Workload-Aware LLM Serving via Runtime Layer Swapping and KV Cache Resizing
Efficiently serving large language models (LLMs) under dynamic and bursty workloads remains a key challenge for real-world deployment. Existing serving frameworks and static model compression techniques fail to adapt to workload fluctuations, leading to either service-level objective (SLO) violations under full-precision serving or persistent accuracy degradation with static quantization. We present MorphServe, a dynamic, workload-aware LLM serving framework based on morphological adaptation. M…

MoLink: Distributed and Efficient Serving Framework for Large Models
Lewei Jin, Yongqi Chen, Kui Zhang, Yifan Zhuo, Yi Gao, Bowei Yang, Zhengong Cai, Wei Dong
https://arxiv.org/abs/2507.05043

MoLink: Distributed and Efficient Serving Framework for Large Models
Large language models represent a groundbreaking shift in generative AI. Yet, these advances come with a significant challenge: the high cost of model serving. To mitigate these costs, consumer-grade GPUs emerge as a more affordable alternative. This presents an opportunity for more cost-efficient LLM serving by leveraging these GPUs. However, it is non-trivial to achieve high-efficiency LLM serving on consumer-grade GPUs, mainly due to two challenges: 1) these GPUs are often deployed in limi…

An interview with Autodesk CEO Andrew Anagnost on settling the proxy fight with Starboard Value, serving the "design and make" workforce, US reshoring, and more (Andrew Edgecliffe-Johnson/Semafor)
https://www.semafor.com/article/07/02/2025

Autodesk’s Andrew Anagnost on surviving an activist attack
The design software CEO says resolving an investor standoff starts with admitting which critiques are fair.

Serving Large Language Models on Huawei CloudMatrix384
Pengfei Zuo, Huimin Lin, Junbo Deng, Nan Zou, Xingkun Yang, Yingyu Diao, Weifeng Gao, Ke Xu, Zhangyu Chen, Shirui Lu, Zhao Qiu, Peiyang Li, Xianyu Chang, Zhengzhong Yu, Fangzheng Miao, Jia Zheng, Ying Li, Yuan Feng, Bei Wang, Zaijian Zong, Mosong Zhou, Wenli Zhou, Houjiang Chen, Xingyu Liao, Yipeng Li, Wenxiao Zhang, Ping Zhu, Yinggang Wang, Chuanjie Xiao, Depeng Liang, Dong Cao, Juncheng Liu, Yongqiang Yang, Xiaolong Bai, Yi Li, H…

Serving Large Language Models on Huawei CloudMatrix384
The rapid evolution of large language models (LLMs), driven by growing parameter scales, adoption of mixture-of-experts (MoE) architectures, and expanding context lengths, imposes unprecedented demands on AI infrastructure. Traditional AI clusters face limitations in compute intensity, memory bandwidth, inter-chip communication, and latency, compounded by variable workloads and strict service-level objectives. Addressing these issues requires fundamentally redesigned hardware-software integrati…

Pornhub's parent company plans to stop serving adult content in France as soon as June 4, in protest of government measures forcing it to verify visitors' ages (Politico)
https://www.politico.eu/article/pornhub-suspend-porn-france/

Pornhub to suspend porn in France
Porn industry giant protests Paris’ measures to verify the age of users.

This https://arxiv.org/abs/2505.09999 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production
With the widespread adoption of Large Language Models (LLMs), serving LLM inference requests has become an increasingly important task, attracting active research advancements. Practical workloads play an essential role in this process: they are critical for motivating and benchmarking serving techniques and systems. However, the existing understanding of real-world LLM serving workloads is limited due to the lack of a comprehensive workload characterization. Prior analyses remain insufficient …

Nexus: Taming Throughput-Latency Tradeoff in LLM Serving via Efficient GPU Sharing
Xiaoxiang Shi, Colin Cai, Junjia Du, Zhanda Zhu, Xingda Wei, Zhihao Jia
https://arxiv.org/abs/2507.06608

Nexus: Taming Throughput-Latency Tradeoff in LLM Serving via Efficient GPU Sharing
Current prefill-decode (PD) disaggregation is typically deployed at the level of entire serving engines, assigning separate GPUs to handle prefill and decode phases. While effective at reducing latency, this approach demands more hardware. To improve GPU utilization, Chunked Prefill mixes prefill and decode requests within the same batch, but introduces phase interference between prefill and decode. While existing PD disaggregation solutions separate the phases across GPUs, we ask: can the sa…

This https://arxiv.org/abs/2501.14417 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

DeepServe: Serverless Large Language Model Serving at Scale
In this paper, we propose DEEPSERVE, a scalable and serverless AI platform designed to efficiently serve large language models (LLMs) at scale in cloud environments. DEEPSERVE addresses key challenges such as resource allocation, serving efficiency, and cold start latencies through four main design components. First, DEEPSERVE uses a simple serverless abstraction called the request-job-task model, which helps manage diverse AI workloads across posttraining and model-serving tasks. Second, DEEPS…

Cascadia: A Cascade Serving System for Large Language Models
Youhe Jiang, Fangcheng Fu, Wanru Zhao, Stephan Rabanser, Nicholas D. Lane, Binhang Yuan
https://arxiv.org/abs/2506.04203

Cascadia: A Cascade Serving System for Large Language Models
Recent advances in large language models (LLMs) have intensified the need to deliver both rapid responses and high-quality answers. More powerful models yield better results but incur higher inference latency, whereas smaller models are faster yet less capable. Recent work proposes balancing this latency-quality trade-off using model cascades, which route simpler queries to smaller models and more complex ones to larger models. However, enabling efficient cascade serving remains challenging. Cu…

This https://arxiv.org/abs/2502.00722 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs
Recent advancements in Large Language Models (LLMs) have led to increasingly diverse requests, accompanied with varying resource (compute and memory) demands to serve them. However, this in turn degrades the cost-efficiency of LLM serving as common practices primarily rely on homogeneous GPU resources. In response to this problem, this work conducts a thorough study about serving LLMs over heterogeneous GPU resources on cloud platforms. The rationale is that different GPU types exhibit distinct…

Characterizing Communication Patterns in Distributed Large Language Model Inference
Lang Xu, Kaushik Kandadi Suresh, Quentin Anthony, Nawras Alnaasan, Dhabaleswar K. Panda
https://arxiv.org/abs/2507.14392

Characterizing Communication Patterns in Distributed Large Language Model Inference
Large Language Models (LLMs) built on transformer architectures have transformed natural language processing, achieving remarkable performance across diverse applications. While distributed inference frameworks enable practical deployment of these models, inter-GPU communication creates significant performance constraints that limit service quality in real-world systems. This paper investigates communication dynamics in distributed LLM serving-analyzing how various parallelization approaches co…

SLED: A Speculative LLM Decoding Framework for Efficient Edge Serving
Xiangchen Li, Dimitrios Spatharakis, Saeid Ghafouri, Jiakun Fan, Dimitrios Nikolopoulos
https://arxiv.org/abs/2506.09397

SLED: A Speculative LLM Decoding Framework for Efficient Edge Serving
Regardless the advancements in device capabilities, efficient inferencing advanced large language models (LLMs) at the edge remains challenging due to limited device memory and power constraints. Existing strategies, such as aggressive quantization, pruning, or remote inference, trade accuracy for efficiency or lead to substantial cost burdens. This position paper introduces a new approach that leverages speculative decoding, previously viewed primarily as a decoding acceleration technique for …

EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devices
Zheyu Shen, Yexiao He, Ziyao Wang, Yuning Zhang, Guoheng Sun, Wanghao Ye, Ang Li
https://arxiv.org/abs/2507.01438

EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devices
Large Language Models (LLMs) have gained significant attention due to their versatility across a wide array of applications. Fine-tuning LLMs with parameter-efficient adapters, such as Low-Rank Adaptation (LoRA), enables these models to efficiently adapt to downstream tasks without extensive retraining. Deploying fine-tuned LLMs on multi-tenant edge devices offers substantial benefits, such as reduced latency, enhanced privacy, and personalized responses. However, serving LLMs efficiently on re…

KVCache Cache in the Wild: Characterizing and Optimizing KVCache Cache at a Large Cloud Provider
Jiahao Wang, Jinbo Han, Xingda Wei, Sijie Shen, Dingyan Zhang, Chenguang Fang, Rong Chen, Wenyuan Yu, Haibo Chen
https://arxiv.org/abs/2506.02634

KVCache Cache in the Wild: Characterizing and Optimizing KVCache Cache at a Large Cloud Provider
Serving large language models (LLMs) is important for cloud providers, and caching intermediate results (KV\$) after processing each request substantially improves serving throughput and latency. However, there is limited understanding of how LLM serving benefits from KV\$ caching, where system design decisions like cache eviction policies are highly workload-dependent. In this paper, we present the first systematic characterization of the KV\$ workload patterns from one of the leading LLM serv…