Tootfinder

"All our energy is Carbon neutral", boasts the Compute-Industrial Complex.
So we feel fine playing with all the AI toys running on data centres guzzling water destined for fields and homes , warming thousands of cumulative kilometres of rivers to ecosystem-collapsing levels.
#AI #datacenters

"All our energy is Carbon neutral", boasts the Compute-Industrial Complex.
So we feel fine playing with all the AI toys running on data centres guzzling water destined for fields and homes , warming thousands of cumulative kilometres of rivers to ecosystem-collapsing levels.
#AI #datacenters

Towards Disaggregation-Native Data Streaming between Devices
Nils Asmussen, Michael Roitzsch
https://arxiv.org/abs/2406.09421 https://

Towards Disaggregation-Native Data Streaming between Devices
Disaggregation is an ongoing trend to increase flexibility in datacenters. With interconnect technologies like CXL, pools of CPUs, accelerators, and memory can be connected via a datacenter fabric. Applications can then pick from those pools the resources necessary for their specific workload. However, this vision becomes less clear when we consider data movement. Workloads often require data to be streamed through chains of multiple devices, but typically, these data streams physically do not …

There's some irony / assholery in the fact that fetching https://openai.com/gptbot-ranges.txt from datacenter IP space gives a 403 error most of the time. Gosh, how dare anyone have the gall to try to automatically fetch someone's resources for their own use 🤯

Investigating Memory Failure Prediction Across CPU Architectures
Qiao Yu, Wengui Zhang, Min Zhou, Jialiang Yu, Zhenli Sheng, Jasmin Bogatinovski, Jorge Cardoso, Odej Kao
https://arxiv.org/abs/2406.05354

Investigating Memory Failure Prediction Across CPU Architectures
Large-scale datacenters often experience memory failures, where Uncorrectable Errors (UEs) highlight critical malfunction in Dual Inline Memory Modules (DIMMs). Existing approaches primarily utilize Correctable Errors (CEs) to predict UEs, yet they typically neglect how these errors vary between different CPU architectures, especially in terms of Error Correction Code (ECC) applicability. In this paper, we investigate the correlation between CEs and UEs across different CPU architectures, inclu…

TraceMesh: Scalable and Streaming Sampling for Distributed Traces
Zhuangbin Chen, Zhihan Jiang, Yuxin Su, Michael R. Lyu, Zibin Zheng
https://arxiv.org/abs/2406.06975

TraceMesh: Scalable and Streaming Sampling for Distributed Traces
Distributed tracing serves as a fundamental element in the monitoring of cloud-based and datacenter systems. It provides visibility into the full lifecycle of a request or operation across multiple services, which is essential for understanding system dependencies and performance bottlenecks. To mitigate computational and storage overheads, most tracing frameworks adopt a uniform sampling strategy, which inevitably captures overlapping and redundant information. More advanced methods employ lea…

ElasticRec: A Microservice-based Model Serving Architecture Enabling Elastic Resource Scaling for Recommendation Models
Yujeong Choi, Jiin Kim, Minsoo Rhu
https://arxiv.org/abs/2406.06955

ElasticRec: A Microservice-based Model Serving Architecture Enabling Elastic Resource Scaling for Recommendation Models
With the increasing popularity of recommendation systems (RecSys), the demand for compute resources in datacenters has surged. However, the model-wise resource allocation employed in current RecSys model serving architectures falls short in effectively utilizing resources, leading to sub-optimal total cost of ownership. We propose ElasticRec, a model serving architecture for RecSys providing resource elasticity and high memory efficiency. ElasticRec is based on a microservice-based software arc…

This https://arxiv.org/abs/2310.02784 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

MAD Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systems
Training and deploying large-scale machine learning models is time-consuming, requires significant distributed computing infrastructures, and incurs high operational costs. Our analysis, grounded in real-world large model training on datacenter-scale infrastructures, reveals that 14~32% of all GPU hours are spent on communication with no overlapping computation. To minimize this outstanding communication latency and other inherent at-scale inefficiencies, we introduce an agile performance model…