Tootfinder

A #visual guide on #troubleshooting #Kubernetes deployments

proxy_pass with nginx fails with Safari, because Upgrade http header is passed on
https://winkelwagen.de/2025/06/24/troubleshooting-proxy_pass-with-safari/

Troubleshooting proxy_pass with Safari
I setup a proxy pass to have the app on one server and the hostname terminate by DNS on another machine. Setting up a reverse proxy is pretty easy: location /directory/ { proxy_pass proxy_set_heade…

Well that was an interesting one to debug... My blocky DNS service was down after a cluster restart
A given #metallb speaker won’t advertise the service if:
- the service has externalTrafficPolicy=local and there are no running endpoints on the speaker’s node
To use externalTrafficPolicy=local, the tolerations on metallb pods must match the tolerations on the destination pods…

PerfTracker: Online Performance Troubleshooting for Large-scale Model Training in Production
Yu Guan, Zhiyu Yin, Haoyu Chen, Sheng Cheng, Chaojie Yang, Tianyin Xu, Yang Zhang, Hanyu Zhao, Yong Li, Dennis Cai, Ennan Zhai
https://arxiv.org/abs/2506.08528

PerfTracker: Online Performance Troubleshooting for Large-scale Model Training in Production
Troubleshooting performance problems of large model training (LMT) is immensely challenging, due to unprecedented scales of modern GPU clusters, the complexity of software-hardware interactions, and the data intensity of the training process. Existing troubleshooting approaches designed for traditional distributed systems or datacenter networks fall short and can hardly apply to real-world training systems. In this paper, we present PerfTracker, the first online troubleshooting system utilizing…

Linux is when your computer doesn't boot, you think it's a kernel bug, you spend half the day troubleshooting, you identify an actual hardware defect, you go out and buy new hardware, and then you find out it actually was a kernel bug all along.
#linux #kernel

Bought a couple breadboard power supplies from Canaduino awhile back. Noticed one on a friends bench and after examining it decided I needed one too. Better than the ones with linear chips. If this fails it just stops working does not send whatever source voltage was to your project. Took awhile to get. They used a shipper called Sendle to ship it from Canada to the U.S.
Anyway I looked around and ordered one of these to to send to the niece.

Electronic Education DIY Kit “The broken code lock” (Troubleshooting Project)
A DIY soldering kit with “integrated” design flaws. Understand the working principle, find the mistakes on the PCB and in the schematics, and make it work.

Replaced article(s) found for cs.LG. https://arxiv.org/list/cs.LG/new/
[7/7]:
PerfTracker: Online Performance Troubleshooting for Large-scale Model Training in Production

Troubleshooting infinite recursion is my least favorite #nixos problem.

Thanks @… for your article on Docker cgroup on Raspberry Pi, helped me out of a pickle!
https://akashrajpurohit.com/…

Resolving Missing Memory Stats in Docker Stats on Raspberry Pi
Running Docker containers on your Raspberry Pi can be a great way to explore different software and services. However, encountering issues with retrieving memory usage information using docker stats can be frustrating. In this blog post will dive into the common reasons behind missing memory stats and guide you through the troubleshooting steps to resolve them.

Replaced article(s) found for cs.OS. https://arxiv.org/list/cs.OS/new/
[1/1]:
PerfTracker: Online Performance Troubleshooting for Large-scale Model Training in Production

AidAI: Automated Incident Diagnosis for AI Workloads in the Cloud
Yitao Yang, Yangtao Deng, Yifan Xiong, Baochun Li, Hong Xu, Peng Cheng
https://arxiv.org/abs/2506.01481

AidAI: Automated Incident Diagnosis for AI Workloads in the Cloud
AI workloads experience frequent incidents due to intensive hardware utilization and extended training times. The current incident management workflow is provider-centric, where customers report incidents and place the entire troubleshooting responsibility on the infrastructure provider. However, the inherent knowledge gap between customer and provider significantly impacts incident resolution efficiency. In AI infrastructure, incidents may take several days on average to mitigate, resulting in…

Network Digital Twin for 6G and Beyond: An End-to-End View Across Multi-Domain Network Ecosystems
Dinh-Hieu Tran, Nazar Waheed, Yuris Mulya Saputra, Xingqin Lin, Cong T. Nguyen, Tedros Salih Abdu, Van Nhan Vo, Van-Quan Pham, Madyan Alsenwi, Abuzar Babikir Mohammad Adam, Symeon Chatzinotas, Eva Lagaunas, Hung Tran, Tu Ho Dac, Nguyen Van Huynh
https://…

Network Digital Twin for 6G and Beyond: An End-to-End View Across Multi-Domain Network Ecosystems
With the rapid development of technology, the number of smart mobile users is increasing, accompanied by growing demands from applications such as virtual/augmented reality (VR/XR), remote surgery, autonomous vehicles, and real-time holographic communications, all of which require high transmission rates and ultra-low latency in 6G and beyond networks (6G+). This poses enormous challenges in efficiently deploying large-scale networks, including network design, planning, troubleshooting, optimiz…

Replaced article(s) found for cs.OS. https://arxiv.org/list/cs.OS/new/
[1/1]:
PerfTracker: Online Performance Troubleshooting for Large-scale Model Training in Production