Tootfinder

Computing Floating-Point Errors by Injecting Perturbations
Youshuai Tan, Zhanwei Zhang, Jinfu Chen, Zishuo Ding, Jifeng Xuan, Weiyi Shang
https://arxiv.org/abs/2507.08467

Computing Floating-Point Errors by Injecting Perturbations
Floating-point programs form the foundation of modern science and engineering, providing the essential computational framework for a wide range of applications, such as safety-critical systems, aerospace engineering, and financial analysis. Floating-point errors can lead to severe consequences. Although floating-point errors widely exist, only a subset of inputs may trigger significant errors in floating-point programs. Therefore, it is crucial to determine whether a given input could produce s…

🏃🏻‍♀️ Why Is Japan Still Investing In Custom Floating Point Accelerators?
https://www.nextplatform.com/2025/09/04/why-is-japan-still-investing-in-custom-floating-point-accelerators/

Why Is Japan Still Investing In Custom Floating Point Accelerators?
It has taken nearly two decades and an immense amount of work by millions of people for high performance computing to go mainstream with GenAI. And now,

Towards Verified Compilation of Floating-point Optimization in Scientific Computing Programs
Mohit Tekriwal, John Sarracino
https://arxiv.org/abs/2509.09019 https://

Towards Verified Compilation of Floating-point Optimization in Scientific Computing Programs
Scientific computing programs often undergo aggressive compiler optimization to achieve high performance and efficient resource utilization. While performance is critical, we also need to ensure that these optimizations are correct. In this paper, we focus on a specific class of optimizations, floating-point optimizations, notably due to fast math, at the LLVM IR level. We present a preliminary work, which leverages the Verified LLVM framework in the Rocq theorem prover, to prove the correctnes…

A 28nm Multiply-Accumulate ASIC Architecture for On-Chip Data Compression in MHz Frame Rate X-ray and Electron Pixel Detectors
Rami Rasheedi, Nicholas Contini, Mohamed Adel Gharib, Sebastian Strempfer, Senthil Gnanasekaran, Salma Abdelzaher, Tejas Guruswamy, Kazutomo Yoshii, Mike Hammer, Henry Shi, Yu-Sheng Chen, Lorenzo Rota, Dionisio Doering, Angelo Dragone, Tao Zhou, Antonino Miceli

A 28nm Multiply-Accumulate ASIC Architecture for On-Chip Data Compression in MHz Frame Rate X-ray and Electron Pixel Detectors
Modern X-ray detector systems urgently require compact, efficient, and fast data compression schemes to handle the transmission of big data from pixel arrays, enabling frame rates in the MHz regime. In this work, a data compression ASIC that implements a streaming fixed-length lossy compression scheme is introduced and analyzed, proving the feasibility and benefits of on-chip compression. The compression scheme utilizes a vector matrix product logic, which performs a number of floating-point mu…

A Large-Scale Study of Floating-Point Usage in Statically Typed Languages
Andrea Gilot, Tobias Wrigstad, Eva Darulova
https://arxiv.org/abs/2509.04936 https://

A Large-Scale Study of Floating-Point Usage in Statically Typed Languages
Reasoning about floating-point arithmetic is notoriously hard. While static and dynamic analysis techniques or program repair have made significant progress, more work is still needed to make them relevant to real-world code. On the critical path to that goal is understanding what real-world floating-point code looks like. To close that knowledge gap, this paper presents the first large-scale empirical study of floating-point arithmetic usage in statically typed languages across public GitHub r…

Extended Abstract: Partial-encapsulate and Its Support for Floating-point Operations in ACL2
Matt Kaufmann, J Strother Moore
https://arxiv.org/abs/2508.00015 https://

Extended Abstract: Partial-encapsulate and Its Support for Floating-point Operations in ACL2
We illustrate the power of partial-encapsulate, showing how it is used in the implementation of floating-point operations in ACL2.

Adding complex numbers to expression template algorithmic differentiation tools
Max Sagebaum, Nicolas R. Gauger
https://arxiv.org/abs/2508.05371 https://ar…

Adding complex numbers to expression template algorithmic differentiation tools
Operator overloading algorithmic differentiation (AD) tools are usually only developed for floating-point values. Algorithmic optimization for, e.g., linear systems solvers or matrix-matrix multiplications are often introduced via external functions or manual function specializations. Complex numbers can be viewed as aggregates of two floating-point values on which specialized operations are applied. Typically, these operations can be handled by the regular floating-point operations from the AD…

from my link log —
Elementary functions NOT following the IEEE 754 floating-point standard.
http://www.hlsl.co.uk/blog/2020/1/29/ieee754-is-not-followed
saved 2025-02-11

Elementary Functions and NOT Following the IEEE 754 Floating-Point Standard — TRYING TO FIND THE OBVIOUS
The IEEE-754 Standard for floating-point numbers was introduced initially in 1985 to solve the problem of diverse floating point implementations prevent code being portable and increase stability across platforms. It has been widely adopted and revised over the last 30 years and if you have dealt

Request-Only Optimization for Recommendation Systems
Liang Guo, Wei Li, Lucy Liao, Huihui Cheng, Rui Zhang, Yu Shi, Yueming Wang, Yanzun Huang, Keke Zhai, Pengchao Wang, Timothy Shi, Xuan Cao, Shengzhi Wang, Renqin Cai, Zhaojie Gong, Omkar Vichare, Rui Jian, Leon Gao, Shiyan Deng, Xingyu Liu, Xiong Zhang, Fu Li, Wenlei Xie, Bin Wen, Rui Li, Xing Liu, Jiaqi Zhai

Request-Only Optimization for Recommendation Systems
Deep Learning Recommendation Models (DLRMs) represent one of the largest machine learning applications on the planet. Industry-scale DLRMs are trained with petabytes of recommendation data to serve billions of users every day. To utilize the rich user signals in the long user history, DLRMs have been scaled up to unprecedented complexity, up to trillions of floating-point operations (TFLOPs) per example. This scale, coupled with the huge amount of training data, necessitates new storage and tra…

Floating-Point Data Transformation for Lossless Compression
Samirasadat Jamalidinan, Kazem Cheshmi
https://arxiv.org/abs/2506.18062 https://

Floating-Point Data Transformation for Lossless Compression
Floating-point data is widely used across various domains. Depending on the required precision, each floating-point value can occupy several bytes. Lossless storage of this information is crucial due to its critical accuracy, as seen in applications such as medical imaging and language model weights. In these cases, data size is often significant, making lossless compression essential. Previous approaches either treat this data as raw byte streams for compression or fail to leverage all pattern…

Numerical Errors in Quantitative System Analysis With Decision Diagrams
Sebastiaan Brand, Arend-Jan Quist, Richard M. K. van Dijk, Alfons Laarman
https://arxiv.org/abs/2508.02673

Numerical Errors in Quantitative System Analysis With Decision Diagrams
Decision diagrams (DDs) are a powerful data structure that is used to tackle the state-space explosion problem, not only for discrete systems, but for probabilistic and quantum systems as well. While many of the DDs used in the probabilistic and quantum domains make use of floating-point numbers, this is not without challenges. Floating-point computations are subject to small rounding errors, which can affect both the correctness of the result and the effectiveness of the DD's compression. In t…

Theoretical Analysis of Relative Errors in Gradient Computations for Adversarial Attacks with CE Loss
Yunrui Yu, Hang Su, Cheng-zhong Xu, Zhizhong Su, Jun Zhu
https://arxiv.org/abs/2507.22428

Theoretical Analysis of Relative Errors in Gradient Computations for Adversarial Attacks with CE Loss
Gradient-based adversarial attacks using the Cross-Entropy (CE) loss often suffer from overestimation due to relative errors in gradient computation induced by floating-point arithmetic. This paper provides a rigorous theoretical analysis of these errors, conducting the first comprehensive study of floating-point computation errors in gradient-based attacks across four distinct scenarios: (i) unsuccessful untargeted attacks, (ii) successful untargeted attacks, (iii) unsuccessful targeted attack…

What Every Computer Scientist Should Know About Floating-Point Arithmetic #floats

RAPTOR: Practical Numerical Profiling of Scientific Applications
Faveo Hoerold, Ivan R. Ivanov, Akash Dhruv, William S. Moses, Anshu Dubey, Mohamed Wahib, Jens Domke
https://arxiv.org/abs/2507.04647

RAPTOR: Practical Numerical Profiling of Scientific Applications
The proliferation of low-precision units in modern high-performance architectures increasingly burdens domain scientists. Historically, the choice in HPC was easy: can we get away with 32 bit floating-point operations and lower bandwidth requirements, or is FP64 necessary? Driven by Artificial Intelligence, vendors introduced novel low-precision units for vector and tensor operations, and FP64 capabilities stagnate or are reduced. This is forcing scientists to re-evaluate their codes, but a tri…

Full Integer Arithmetic Online Training for Spiking Neural Networks
Ismael Gomez, Guangzhi Tang
https://arxiv.org/abs/2509.06636 https://arxiv.org/pdf/2509…

Full Integer Arithmetic Online Training for Spiking Neural Networks
Spiking Neural Networks (SNNs) are promising for neuromorphic computing due to their biological plausibility and energy efficiency. However, training methods like Backpropagation Through Time (BPTT) and Real Time Recurrent Learning (RTRL) remain computationally intensive. This work introduces an integer-only, online training algorithm using a mixed-precision approach to improve efficiency and reduce memory usage by over 60%. The method replaces floating-point operations with integer arithmetic …

Jack Unit: An Area- and Energy-Efficient Multiply-Accumulate (MAC) Unit Supporting Diverse Data Formats
Seock-Hwan Noh, Sungju Kim, Seohyun Kim, Daehoon Kim, Jaeha Kung, Yeseong Kim
https://arxiv.org/abs/2507.04772

Jack Unit: An Area- and Energy-Efficient Multiply-Accumulate (MAC) Unit Supporting Diverse Data Formats
In this work, we introduce an area- and energy-efficient multiply-accumulate (MAC) unit, named Jack unit, that is a jack-of-all-trades, supporting various data formats such as integer (INT), floating point (FP), and microscaling data format (MX). It provides bit-level flexibility and enhances hardware efficiency by i) replacing the carry-save multiplier (CSM) in the FP multiplier with a precision-scalable CSM, ii) performing the adjustment of significands based on the exponent differences withi…

I read Bostrom's book SUPERINTELLIGENCE back when I worked at Google —
Blaise A— insisted that it go on the "AI" SF book club reading list
(as an aside: I started the book club; we were mostly reading, uh, cautionary stories)
SUPERINTELLIGENCE is garbage, & I said so: throw around big numbers until your moral calculus has a floating-point overflow & you can be convinced of obviously crazy things
(phrasing wasn't so clean at the time)
.…

Analysis of Floating-Point Matrix Multiplication Computed via Integer Arithmetic
Ahmad Abdelfattah, Jack Dongarra, Massimiliano Fasi, Mantas Mikaitis, Fran\c{c}oise Tisseur
https://arxiv.org/abs/2506.11277

Analysis of Floating-Point Matrix Multiplication Computed via Integer Arithmetic
Ootomo, Ozaki, and Yokota [Int. J. High Perform. Comput. Appl., 38 (2024), p. 297-313] have proposed a strategy to recast a floating-point matrix multiplication in terms of integer matrix products. The factors A and B are split into integer slices, the product of these slices is computed exactly, and AB is approximated by accumulating these integer products in floating-point arithmetic. This technique is particularly well suited to mixed-precision matrix multiply-accumulate units with integer s…

Accurate Reduced Floating-Point Precision Implicit Monte Carlo
Simon Butson, Mathew Cleveland, Alex Long, Todd Palmer
https://arxiv.org/abs/2506.11962 http…

Accurate Reduced Floating-Point Precision Implicit Monte Carlo
This work describes methodologies to successfully implement the Implicit Monte Carlo (IMC) scheme for thermal radiative transfer in reduced-precision floating-point arithmetic. The methods used can be broadly categorized into scaling approaches and floating-point arithmetic manipulations. Scaling approaches entail re-scaling values to ensure computations stay within a representable range. Floating-point arithmetic manipulations involve changes to order of operations and alternative summation al…

Invariant Generation for Floating-Point Programs via Constraint Solving
Xuran Cai, Liqian Chen, Hongfei Fu
https://arxiv.org/abs/2507.15017 https://…

Invariant Generation for Floating-Point Programs via Constraint Solving
In numeric-intensive computations, it is well known that the execution of floating-point programs is imprecise as floating point arithmetics (e.g., addition, subtraction, multiplication, division, etc.) incurs rounding errors. Albeit the rounding error is small for every single floating-point operation, the aggregation of such error in multiple operations may be dramatic and cause catastrophic program failures. Therefore, to ensure the correctness of floating-point programs, the effect of float…

LLM-Based Program Generation for Triggering Numerical Inconsistencies Across Compilers
Yutong Wang, Cindy Rubio-Gonz\'alez
https://arxiv.org/abs/2509.00256 https://

LLM-Based Program Generation for Triggering Numerical Inconsistencies Across Compilers
Floating-point inconsistencies across compilers can undermine the reliability of numerical software. We present LLM4FP, the first framework that uses Large Language Models (LLMs) to generate floating-point programs specifically designed to trigger such inconsistencies. LLM4FP combines Grammar-Based Generation and Feedback-Based Mutation to produce diverse and valid programs. We evaluate LLM4FP across multiple compilers and optimization levels, measuring inconsistency rate, time cost, and progra…

Back to Bits: Extending Shannon's communication performance framework to computing
Max Hawkins, Richard Vuduc
https://arxiv.org/abs/2508.05621 https://…

Back to Bits: Extending Shannon's communication performance framework to computing
This work proposes a novel computing performance unit grounded in information theory. Modern computing systems are increasingly diverse, supporting low-precision formats, hardware specialization, and emerging paradigms such as analog, quantum, and reversible logic. Traditional metrics like floating-point operations (flops) no longer accurately capture this complexity. We frame computing as the transformation of information through a channel and define performance in terms of the mutual informat…

Improving Deep Learning Framework Testing with Model-Level Metamorphic Testing
Yanzhou Mu, Juan Zhai, Chunrong Fang, Xiang Chen, Zhixiang Cao, Peiran Yang, Kexin Zhao, An Guo, Zhenyu Chen
https://arxiv.org/abs/2507.04354

Improving Deep Learning Framework Testing with Model-Level Metamorphic Testing
Deep learning (DL) frameworks are essential to DL-based software systems, and framework bugs may lead to substantial disasters, thus requiring effective testing. Researchers adopt DL models or single interfaces as test inputs and analyze their execution results to detect bugs. However, floating-point errors, inherent randomness, and the complexity of test inputs make it challenging to analyze execution results effectively, leading to existing methods suffering from a lack of suitable test oracl…

Accuracy of Mathematical Functions in Julia
Mantas Mikaitis, Tejaswa Rizyal
https://arxiv.org/abs/2509.05666 https://arxiv.org/pdf/2509.05666

Accuracy of Mathematical Functions in Julia
Basic computer arithmetic operations, such as $+$, $\times$, or $÷$ are correctly rounded, whilst mathematical functions such as $e^x$, $\ln(x)$, or $\sin(x)$ in general are not, meaning that separate implementations may provide different results when presented with an exact same input, and that their accuracy may differ. We present a methodology and a software tool that is suited for exhaustive and non-exhaustive testing of mathematical functions of Julia in various floating-point formats. Th…

Lattice Random Walk Discretisations of Stochastic Differential Equations
Samuel Duffield, Maxwell Aifer, Denis Melanson, Zach Belateche, Patrick J. Coles
https://arxiv.org/abs/2508.20883

Lattice Random Walk Discretisations of Stochastic Differential Equations
We introduce a lattice random walk discretisation scheme for stochastic differential equations (SDEs) that samples binary or ternary increments at each step, suppressing complex drift and diffusion computations to simple 1 or 2 bit random values. This approach is a significant departure from traditional floating point discretisations and offers several advantages; including compatibility with stochastic computing architectures that avoid floating-point arithmetic in place of directly manipulati…

from my link log —
Posit floating point numbers: thin triangles and other tricks.
http://marc-b-reynolds.github.io/math/2019/02/06/Posit1.html
saved 2025-06-18

STF: Shallow-Level Temporal Feedback to Enhance Spiking Transformers
Zeqi Zheng, Zizheng Zhu, Yingchao Yu, Yanchen Huang, Changze Lv, Junfeng Tang, Zhaofei Yu, Yaochu Jin
https://arxiv.org/abs/2508.00387

STF: Shallow-Level Temporal Feedback to Enhance Spiking Transformers
Transformer-based Spiking Neural Networks (SNNs) suffer from a great performance gap compared to floating-point Artificial Neural Networks (ANNs) due to the binary nature of spike trains. Recent efforts have introduced deep-level feedback loops to transmit high-level semantic information to narrow this gap. However, these designs often span multiple deep layers, resulting in costly feature transformations, higher parameter overhead, increased energy consumption, and longer inference latency. To…

On Automating Proofs of Multiplier Adder Trees using the RTL Books
Mayank Manjrekar (Arm Inc.)
https://arxiv.org/abs/2507.19010 https://arxiv.org/pdf/2507.…

On Automating Proofs of Multiplier Adder Trees using the RTL Books
We present an experimental, verified clause processor ctv-cp that fits into the framework used at Arm for formal verification of arithmetic hardware designs. This largely automates the ACL2 proof development effort for integer multiplier modules that exist in designs ranging from floating-point division to matrix multiplication.

Fast and Scalable Mixed Precision Euclidean Distance Calculations Using GPU Tensor Cores
Brian Curless, Michael Gowanlock
https://arxiv.org/abs/2508.21230 https://

Fast and Scalable Mixed Precision Euclidean Distance Calculations Using GPU Tensor Cores
Modern GPUs are equipped with tensor cores (TCs) that are commonly used for matrix multiplication in artificial intelligence workloads. However, because they have high computational throughput, they can lead to significant performance gains in other algorithms if they can be successfully exploited. We examine using TCs to compute Euclidean distance calculations, which are used in many data analytics applications. Prior work has only investigated using 64 bit floating point (FP64) data for compu…

GDNSQ: Gradual Differentiable Noise Scale Quantization for Low-bit Neural Networks
Sergey Salishev, Ian Akhremchik
https://arxiv.org/abs/2508.14004 https://

GDNSQ: Gradual Differentiable Noise Scale Quantization for Low-bit Neural Networks
Quantized neural networks can be viewed as a chain of noisy channels, where rounding in each layer reduces capacity as bit-width shrinks; the floating-point (FP) checkpoint sets the maximum input rate. We track capacity dynamics as the average bit-width decreases and identify resulting quantization bottlenecks by casting fine-tuning as a smooth, constrained optimization problem. Our approach employs a fully differentiable Straight-Through Estimator (STE) with learnable bit-width, noise scale an…

DGEMM without FP64 Arithmetic -- using FP64 Emulation and FP8 Tensor Cores with Ozaki Scheme
Daichi Mukunoki
https://arxiv.org/abs/2508.00441 https://arxiv…

DGEMM without FP64 Arithmetic -- using FP64 Emulation and FP8 Tensor Cores with Ozaki Scheme
Since AI computations require low-precision matrix multiplications, processors with enhanced performance for these operations are increasing along with the growing demand for AI computations. However, it is difficult to use these operations directly for scientific computations. The Ozaki scheme, an accurate matrix multiplication method proposed by Ozaki et al. in 2012, enables FP64 matrix multiplication (DGEMM) using low-precision floating-point operations such as FP16. The method was subsequen…

SETI@home: Data Acquisition and Front-End Processing
Eric J. Korpela (Space Sciences Laboratory, University of California, Berkeley), David P. Anderson (Space Sciences Laboratory, University of California, Berkeley), Jeff Cobb (Space Sciences Laboratory, University of California, Berkeley), Matt Lebofsky (Space Sciences Laboratory, University of California, Berkeley), Wei Liu (Space Sciences Laboratory, University of California, Berkeley), Dan Werthimer (Space Sciences Laboratory, Univ…

SETI@home: Data Acquisition and Front-End Processing
SETI@home is a radio Search for Extraterrestrial Intelligence (SETI) project, looking for technosignatures in data recorded at multiple observatories from 1998 to 2020. Most radio SETI projects analyze data using dedicated processing hardware. SETI@home uses a different approach: time-domain data is distributed over the Internet to $\gt 10^{5}$ volunteered home computers, which analyze it. The large amount of computing power this affords ($\sim 10^{15}$ floating-point operations per second (FPO…

Efficient vectorized evaluation of Gaussian AO integrals on modern central processing units
Andrey Asadchev, Edward F. Valeev
https://arxiv.org/abs/2506.12501

Efficient vectorized evaluation of Gaussian AO integrals on modern central processing units
We report an implementation of the McMurchie-Davidson evaluation scheme for 1- and 2-particle Gaussian AO integrals designed for efficient execution on modern central processing units (CPUs) with Single Instruction Multiple Data (SIMD) instruction sets. Like in our recent MD implementation for graphical processing units (GPUs) [J. Chem. Phys. 160, 244109 (2024)], variable-sized batches of shellsets of integrals are evaluated at a time. By optimizing for the floating point instruction throughput…

AR-LIF: Adaptive reset leaky-integrate and fire neuron for spiking neural networks
Zeyu Huang, Wei Meng, Quan Liu, Kun Chen, Li Ma
https://arxiv.org/abs/2507.20746 https://

AR-LIF: Adaptive reset leaky-integrate and fire neuron for spiking neural networks
Spiking neural networks possess the advantage of low energy consumption due to their event-driven nature. Compared with binary spike outputs, their inherent floating-point dynamics are more worthy of attention. The threshold level and re- set mode of neurons play a crucial role in determining the number and timing of spikes. The existing hard reset method causes information loss, while the improved soft reset method adopts a uniform treatment for neurons. In response to this, this paper designs…

Pole vaulter straight up and vertical after making it over the cross bar
#photo #photography #sports #track

Permutation-Avoiding FFT-Based Convolution
Nicolas Venkovic, Hartwig Anzt
https://arxiv.org/abs/2506.12718 https://arxiv.org/pdf/2506…

Permutation-Avoiding FFT-Based Convolution
Fast Fourier Transform (FFT) libraries are widely used for evaluating discrete convolutions. Most FFT implementations follow some variant of the Cooley-Tukey framework, in which the transform is decomposed into butterfly operations and index-reversal permutations. While butterfly operations dominate the floating-point operation count, the memory access patterns induced by index-reversal permutations significantly degrade the FFT's arithmetic intensity. In practice, discrete convolutions are oft…