Tootfinder

WanJuan-CC: A Safe and High-Quality Open-sourced English Webtext Dataset
Jiantao Qiu, Haijun Lv, Zhenjiang Jin, Rui Wang, Wenchang Ning, Jia Yu, ChaoBin Zhang, Pei Chu, Yuan Qu, Runyu Peng, Zhiyuan Zeng, Huanze Tang, Ruiliang Xu, Wei Li, Hang Yan, Conghui He
https://arxiv.org/abs/2402.19282<…

WanJuan-CC: A Safe and High-Quality Open-sourced English Webtext Dataset
This paper presents WanJuan-CC, a safe and high-quality open-sourced English webtext dataset derived from Common Crawl data. The study addresses the challenges of constructing large-scale pre-training datasets for language models, which require vast amounts of high-quality data. A comprehensive process was designed to handle Common Crawl data, including extraction, heuristic rule filtering, fuzzy deduplication, content safety filtering, and data quality filtering. From approximately 68 billion …

An open letter on the position of scientists and researchers on the
recently proposed changes to the EU’s proposed Child Sexual Abuse Regulation. As of the 1st May 2024, the letter has been signed by 254 scientists and researchers from 33 countries. (among them @… )

Open Letter CSA v2.pdf
Nextcloud - a safe home for all your data

Or find something better to do with your life than working at Unilever. https://toot.community/@openculture/112158156601363997

Open Culture (Official) (@openculture@toot.community)
Attached: 1 image Learn to Become a Supply Chain Data Analyst with Unilever’s New Certificate Program https://www.openculture.com/2024/03/learn-to-become-a-supply-chain-data-analyst-with-unilevers-new-certificate-program.html

🔊 #NowPlaying on KEXP's #DriveTime
Radiohead:
🎵 Weird Fishes/Arpeggi
#Radiohead
https://open.spotify.com/track/4wajJ1o7jWIg62YqpkHC7S
https://ghostdata.bandcamp.com/track/radiohead-weird-fishes-arpeggi-ghost-data-remix

AI companies to universities: Personalized tutors will make you obsolete
Also AI companies: Thanks for recording your lectures so we can sell them on the open market to train personalized tutors
https://annettevee.substack.com/p/when-student-data-is-the-new-oil

Advanced analysis of single-molecule spectroscopic data
Joshua L. Botha, Bertus van Heerden, Tjaart P. J. Kr\"uger
https://arxiv.org/abs/2404.18945 https://arxiv.org/pdf/2404.18945
arXiv:2404.18945v1 Announce Type: new
Abstract: We present Full SMS, a multipurpose graphical user interface (GUI)-based software package for analysing single-molecule spectroscopy (SMS) data. SMS typically delivers multiparameter data -- such as fluorescence brightness, lifetime, and spectra -- of molecular- or nanometre-scale particles such as single dye molecules, quantum dots, or fluorescently labelled biological macromolecules. Full SMS allows an unbiased statistical analysis of fluorescence brightness through level resolution and clustering, analysis of fluorescence lifetimes through decay fitting, as well as the calculation of second-order correlation functions and the display of fluorescence spectra and raster-scan images. Additional features include extensive data filtering options, a custom HDF5-based file format, and flexible data export options. The software is open source and written in Python but GUI-based so it may be used without any programming knowledge. A multi-process architecture was employed for computational efficiency. The software is also designed to be easily extendable to include additional import data types and analysis capabilities.

Differentiated Security Architecture for Secure and Efficient Infotainment Data Communication in IoV Networks
Jiani Fan, Lwin Khin Shar, Jiale Guo, Wenzhuo Yang, Dusit Niyato, Kwok-Yan Lam
https://arxiv.org/abs/2403.20136

Differentiated Security Architecture for Secure and Efficient Infotainment Data Communication in IoV Networks
This paper aims to provide differentiated security protection for infotainment data communication in Internet-of-Vehicle (IoV) networks. The IoV is a network of vehicles that uses various sensors, software, built-in hardware, and communication technologies to enable information exchange between pedestrians, cars, and urban infrastructure. Negligence on the security of infotainment data communication in IoV networks can unintentionally open an easy access point for social engineering attacks. Th…

#LinuxFoundation Launches #OpenSource Valkey Community
https://www.

The definition of what constitutes an "open system" has really been sanded down by "AI". Just dropping random weights and a bit of network structure counts as open: No idea how and on what something was trained, which data was filtered out, which earlier training runs were discarded.
It's not really open at all. Maybe "free" as in "free beer" but from learning potential, ability to understand the system and maybe extend it, it's not any …

LLM compression will be the final nail in the coffin of the open web
There is no benefit to create content just for it to be copied, compressed, and regurgitated without attribution or payment
Arc Browser is pretty cool, but it assumes websites will continue to publish high quality data without anyone ever visiting them directly

This https://arxiv.org/abs/2403.17165 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csHC_…

Building an Open-Source Community to Enhance Autonomic Nervous System Signal Analysis: DBDP-Autonomic
Smartphones and wearable sensors offer an unprecedented ability to collect peripheral psychophysiological signals across diverse timescales, settings, populations, and modalities. However, open-source software development has yet to keep pace with rapid advancements in hardware technology and availability, creating an analytical barrier that limits the scientific usefulness of acquired data. We propose a community-driven, open-source peripheral psychophysiological signal pre-processing and anal…

PeLLE: Encoder-based language models for Brazilian Portuguese based on open data
Guilherme Lamartine de Mello, Marcelo Finger, and Felipe Serras, Miguel de Mello Carpi, Marcos Menon Jose, Pedro Henrique Domingues, Paulo Cavalim
https://arxiv.org/abs/2402.19204

PeLLE: Encoder-based language models for Brazilian Portuguese based on open data
In this paper we present PeLLE, a family of large language models based on the RoBERTa architecture, for Brazilian Portuguese, trained on curated, open data from the Carolina corpus. Aiming at reproducible results, we describe details of the pretraining of the models. We also evaluate PeLLE models against a set of existing multilingual and PT-BR refined pretrained Transformer-based LLM encoders, contrasting performance of large versus smaller-but-curated pretrained models in several downstream …

Deep Learning for Educational Data Science
Juan D. Pinto, Luc Paquette
https://arxiv.org/abs/2404.19675 https://arxiv.org/pdf/2404.19675
arXiv:2404.19675v1 Announce Type: new
Abstract: With the ever-growing presence of deep artificial neural networks in every facet of modern life, a growing body of researchers in educational data science -- a field consisting of various interrelated research communities -- have turned their attention to leveraging these powerful algorithms within the domain of education. Use cases range from advanced knowledge tracing models that can leverage open-ended student essays or snippets of code to automatic affect and behavior detectors that can identify when a student is frustrated or aimlessly trying to solve problems unproductively -- and much more. This chapter provides a brief introduction to deep learning, describes some of its advantages and limitations, presents a survey of its many uses in education, and discusses how it may further come to shape the field of educational data science.

Global well-posedness for supercritical SQG with perturbations of radially symmetric data
Aynur Bulut, Hongjie Dong
https://arxiv.org/abs/2402.19439 https:…

Global well-posedness for supercritical SQG with perturbations of radially symmetric data
We study the global well-posedness of the supercritical dissipative surface quasi-geostrophic (SQG) equation, a key model in geophysical fluid dynamics. While local well-posedness is known, achieving global well-posedness for large initial data remains open. Motivated by enhanced decay in radial solutions, we aim to establish global well-posedness for small perturbations of potentially large radial data. Our main result shows that for small perturbations of radial data, the SQG equation admits …

"The findings demonstrate that repository shutdown does happen and can result in permanent data loss… Data #reuse & #citation are increasingly promoted by journals, funders and other stakeholders. If these practices become more common, data loss might pose a threat to the permanence of the scholarly…

Do disappearing data repositories pose a threat to open science and the scholarly record?
Research data repositories play a vital role in ensuring research is reproducible, replicable and reusable. Yet, the infrastructure supporting them can be impermanent. Drawing on a new dataset Doro…

This https://arxiv.org/abs/2307.02140 has been replaced.
link: https://scholar.google.com/scholar?q=a

Towards Open Federated Learning Platforms: Survey and Vision from Technical and Legal Perspectives
Traditional Federated Learning (FL) follows a server-dominated cooperation paradigm which narrows the application scenarios of FL and decreases the enthusiasm of data holders to participate. To fully unleash the potential of FL, we advocate rethinking the design of current FL frameworks and extending it to a more generalized concept: Open Federated Learning Platforms, positioned as a crowdsourcing collaborative machine learning infrastructure for all Internet users. We propose two reciprocal co…

Handling Open Research Data within the Max Planck Society -- Looking Closer at the Year 2020
Martin Boosen, Michael Franke, Yves Vincent Grossmann, Sy Dat Ho, Larissa Leiminger, Jan Matthiesen
https://arxiv.org/abs/2402.18182

Handling Open Research Data within the Max Planck Society -- Looking Closer at the Year 2020
This paper analyses the practice of publishing research data within the Max Planck Society in the year 2020. The central finding of the study is that up to 40\% of the empirical text publications had research data available. The aggregation of the available data is predominantly analysed. There are differences between the sections of the Max Planck Society but they are not as great as one might expect. In the case of the journals, it is also apparent that a data policy can increase the availabi…

Imagine if game studios would open source models of real world buildings, sites, etc. which they have researched for their games (Ubisoft).
Even if they are a decade old data, I wonder if such models could help indie dev studios.
#gaming #gamedesign

Bevy 0.13 is out! 🕺
For those who don't know, Bevy is a data-driven game engine built in Rust.
Check out the new features 👇
https://bevyengine.org/news/bevy-0-13/

Bevy 0.13
Bevy is a refreshingly simple data-driven game engine built in Rust. It is free and open-source forever!

Processing HSV Colored Medical Images and Adapting Color Thresholds for Computational Image Analysis: a Practical Introduction to an open-source tool
Lie Cai, Andre Pfob
https://arxiv.org/abs/2404.17878 https://arxiv.org/pdf/2404.17878
arXiv:2404.17878v1 Announce Type: new
Abstract: Background: Using artificial intelligence (AI) techniques for computational medical image analysis has shown promising results. However, colored images are often not readily available for AI analysis because of different coloring thresholds used across centers and physicians as well as the removal of clinical annotations. We aimed to develop an open-source tool that can adapt different color thresholds of HSV-colored medical images and remove annotations with a simple click.
Materials and Methods: We built a function using MATLAB and used multi-center international shear wave elastography data (NCT 02638935) to test the function. We provide step-by-step instructions with accompanying code lines.
Results: We demonstrate that the newly developed pre-processing function successfully removed letters and adapted different color thresholds of HSV-colored medical images.
Conclusion: We developed an open-source tool for removing letters and adapting different color thresholds in HSV-colored medical images. We hope this contributes to advancing medical image processing for developing robust computational imaging algorithms using diverse multi-center big data. The open-source Matlab tool is available at https://github.com/cailiemed/image-threshold-adapting.

PEFSL: A deployment Pipeline for Embedded Few-Shot Learning on a FPGA SoC
Lucas Grativol Ribeiro (IMT Atlantique - MEE, Lab\_STICC\_BRAIn, Lab-STICC\_2AI, LHC), Lubin Gauthier (Lab\_STICC\_BRAIn, IMT Atlantique - MEE), Mathieu Leonardon (IMT Atlantique - MEE, Lab\_STICC\_BRAIn), J\'er\'emy Morlier (IMT Atlantique - MEE, Lab\_STICC\_BRAIn), Antoine Lavrard-Meyer (IMT Atlantique), Guillaume Muller (Mines Saint-\'Etienne MSE, FAYOL-ENSMSE, FAYOL-ENSMSE), Virginie Fresse (LHC, …

PEFSL: A deployment Pipeline for Embedded Few-Shot Learning on a FPGA SoC
This paper tackles the challenges of implementing few-shot learning on embedded systems, specifically FPGA SoCs, a vital approach for adapting to diverse classification tasks, especially when the costs of data acquisition or labeling prove to be prohibitively high. Our contributions encompass the development of an end-to-end open-source pipeline for a few-shot learning platform for object classification on a FPGA SoCs. The pipeline is built on top of the Tensil open-source framework, facilitati…

This https://arxiv.org/abs/2311.04590 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIR_…

Rethinking Cross-Domain Sequential Recommendation under Open-World Assumptions
Cross-Domain Sequential Recommendation (CDSR) methods aim to tackle the data sparsity and cold-start problems present in Single-Domain Sequential Recommendation (SDSR). Existing CDSR works design their elaborate structures relying on overlapping users to propagate the cross-domain information. However, current CDSR methods make closed-world assumptions, assuming fully overlapping users across multiple domains and that the data distribution remains unchanged from the training environment to the …

"Passion for Provenance (data): On collection data, museum systems and open collaboration via wikis" @… https://

This https://arxiv.org/abs/2309.12924 has been replaced.
initial toot: https://mastoxiv.page/@arXi…

Automated grading workflows for providing personalized feedback to open-ended data science assignments
Open-ended assignments - such as lab reports and semester-long projects - provide data science and statistics students with opportunities for developing communication, critical thinking, and creativity skills. However, providing grades and formative feedback to open-ended assignments can be very time consuming and difficult to do consistently across students. In this paper, we discuss the steps of a typical grading workflow and highlight which steps can be automated in an approach that we call …

Heute vor 39 Jahren: Am 2. Mai 1985 zündeten die #USA im Rahmen von Operation Grenadier die 9. Atombombe "Towanda". Grenadier war eine Serie von #Kernwaffentests bei der 1984/85 insgesamt 16 Bomben im Testgebiet in

European Peatlands & Policies Open Data Mapathon 2024
Seeking teams (2 to 4 people) from all European countries to register for European Peatlands & Policies Open Data Mapathon 2024 #Mapathon2024
Date: 6th April Venue: Online/University of Galway

First Prize €1,200.

Bluesky says: Ready to federate. Their discussion of the differences between their approach and Fedi’s is interesting.
https://bsky.social/about/blog/02-22-2024-open-social-web

Bluesky: An Open Social Web - Bluesky
We’re excited to announce that the Bluesky network is federating and opening up in a way that allows you to host your own data.

February 22, 2024: "Today, we’re excited to announce that the Bluesky network is federating and opening up in a way that allows you to host your own data."
https://bsky.social/about/blog/02-22-2024-open-social-web

Bluesky: An Open Social Web - Bluesky
We’re excited to announce that the Bluesky network is federating and opening up in a way that allows you to host your own data.

To democratize research with sensitive data, we should make synthetic data more accessible
Erik-Jan van Kesteren
https://arxiv.org/abs/2404.17271 https://<…

To democratize research with sensitive data, we should make synthetic data more accessible
For over 30 years, synthetic data has been heralded as a promising solution to make sensitive datasets accessible. However, despite much research effort and several high-profile use-cases, the widespread adoption of synthetic data as a tool for open, accessible, reproducible research with sensitive data is still a distant dream. In this opinion, Erik-Jan van Kesteren, head of the ODISSEI Social Data Science team, argues that in order to progress towards widespread adoption of synthetic data as …

Rechtzeitig für die diesjährige #LoveDataWeek ♥️ erscheint der Bericht zum Beitrag von @lozross@post.lurk.org & @… von @… im Rahmen des internationalen Workshops

OpenMedLM: Prompt engineering can out-perform fine-tuning in medical question-answering with open-source large language models
Jenish Maharjan, Anurag Garikipati, Navan Preet Singh, Leo Cyrus, Mayank Sharma, Madalina Ciobanu, Gina Barnes, Rahul Thapa, Qingqing Mao, Ritankar Das
https://arxiv.org/abs/2402.19371

OpenMedLM: Prompt engineering can out-perform fine-tuning in medical question-answering with open-source large language models
LLMs have become increasingly capable at accomplishing a range of specialized-tasks and can be utilized to expand equitable access to medical knowledge. Most medical LLMs have involved extensive fine-tuning, leveraging specialized medical data and significant, thus costly, amounts of computational power. Many of the top performing LLMs are proprietary and their access is limited to very few research groups. However, open-source (OS) models represent a key area of growth for medical LLMs due to …

Investigating the dissemination of STEM content on social media with computational tools
Oluwamayokun Oshinowo, Priscila Delgado, Meredith Fay, C. Alessandra Luna, Anjana Dissanayaka, Rebecca Jeltuhin, David R. Myers
https://arxiv.org/abs/2404.18944

Investigating the dissemination of STEM content on social media with computational tools
Social media platforms can quickly disseminate STEM content to diverse audiences, but their operation can be mysterious. We used open-source machine learning methods such as clustering, regression, and sentiment analysis to analyze over 1000 videos and metrics thereof from 6 social media STEM creators. Our data provide insights into how audiences generate interest signals(likes, bookmarks, comments, shares), on the correlation of various signals with views, and suggest that content from newer c…

Bluesky federation has launched.
https://bsky.social/about/blog/02-22-2024-open-social-web

Bluesky: An Open Social Web - Bluesky
We’re excited to announce that the Bluesky network is federating and opening up in a way that allows you to host your own data.

This https://arxiv.org/abs/2308.02279 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_…

Open superclusters I: The most populated primordial groups of open clusters in the third quadrant of the Galactic disc
An open supercluster (OSC) is defined as a cluster of at least six open clusters (OCs) born from the same giant molecular cloud (GMC). We survey the recent catalogs of OCs based on Gaia data and relevant literature to find 17 OSCs of the third galactic quadrant, along with 190 likely members of them. OSCs are frequent enough to be considered an extra class of objects in the hierarchy of star formation. Some of these supersystems are new and most of them contain more members than previously thou…

3D-Speaker-Toolkit: An Open Source Toolkit for Multi-modal Speaker Verification and Diarization
Yafeng Chen, Siqi Zheng, Hui Wang, Luyao Cheng, Tinglong Zhu, Changhe Song, Rongjie Huang, Ziyang Ma, Qian Chen, Shiliang Zhang, Xihao Li
https://arxiv.org/abs/2403.19971

3D-Speaker-Toolkit: An Open Source Toolkit for Multi-modal Speaker Verification and Diarization
This paper introduces 3D-Speaker-Toolkit, an open source toolkit for multi-modal speaker verification and diarization. It is designed for the needs of academic researchers and industrial practitioners. The 3D-Speaker-Toolkit adeptly leverages the combined strengths of acoustic, semantic, and visual data, seamlessly fusing these modalities to offer robust speaker recognition capabilities. The acoustic module extracts speaker embeddings from acoustic features, employing both fully-supervised and …

"#Apple points to #WebApps as the open alternative to the App Store, and actions to remove them have created deep concern in the web community.
#iOS demoting Web Apps to shortcuts threaten data loss and undermi…

Open Letter to Tim Cook: Sabotaging Web Apps Is Indefensible

Redis, the popular in-memory data store, switches from the open source 3-clause BSD license to a controversial dual-license model (Frederic Lardinois/TechCrunch)
https://techcrunch.com/2024/03/21/redis-switches-li…

Redis switches licenses, acquires Speedb to go beyond its core in-memory database | TechCrunch
Redis, the popular in-memory data store, is switching away from the open source three-clause BSD license. Instead, in a move that is clearly aimed to

Given the rather sloppy headlines about what “Wordpress” is planning to do with allowing AI systems to collect user data, I will point out:
- All the references to Wordpress refer to Wordpress.com, the hosting company whose parent company is Automattic. They do not refer to Wordpress the open source software project.
- Wordpress.com has published a blog post outlining their policies so that you don’t need to rely on speculative and vaguely alarming news articles

More Control Over the Content You Share
There are currently very few options for individual users to control how their content is used for AI training, and we want to change that. That’s why we’re launching a new tool that lets you opt o…

Jetzt Marcel Ackermanns Vortrag "Datenintegration aus offenen Quellen in der dblp computer science bibliography" beim #kimws24. Ich kriege einen kleinen Flashback zu meiner Aktivität in der Working Group on Open Bibliographic Data der OKFN, in deren Rahmen wir 2011 Marcels Gastbeitrag zur Freigabe der DBLP-Daten veröffentlicht haben:

Germany has open sourced a lot of data related to public #EV charging infrastructure.
Charging data is anonymised to protect operator interests, but it may still be useful for others to work with.
15k different ad-hoc pricing structures also registered - who wants to do an analysis on that?

This https://arxiv.org/abs/2404.10255 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

Privacy-Preserving Training-as-a-Service for On-Device Intelligence: Concept, Architectural Scheme, and Open Problems
On-device intelligence (ODI) enables artificial intelligence (AI) applications to run on end devices, providing real-time and customized AI services without relying on remote servers. However, training models for on-device deployment face significant challenges due to the decentralized and privacy-sensitive nature of users' data, along with end-side constraints related to network connectivity, computation efficiency, etc. Existing training paradigms, such as cloud-based training, federated lear…

Transactions on Graph Data & Knowledge (TGDK) is indexed now in the
Directory of Open Access Journals (DOAJ)
#knowledgegraphs

Transactions on Graph Data and Knowledge – DOAJ
A peer-reviewed, open access journal in graph algorithms, graph databases, graph representation learning, knowledge graphs, knowledge representation & linked data.

In another move[1] to stay up-to-date with latest version of Zig (v0.12.0), I've also updated all code (and .zig.zon depencency info) in the still-just-a-baby zig.thi.ng repo:
https://github.com/thi-ng/zig-thing
[1] Related (from yesterday):

GitHub - thi-ng/zig-thing: Various, still unstructured experiments / data structures / open-learning with Zig
Various, still unstructured experiments / data structures / open-learning with Zig - thi-ng/zig-thing

Wikidata & AI, together again – Wikimedia Tech News https://tech-news.wikimedia.de/en/2024/02/19/wikidata-ai-together-again/

Wikidata & AI, together again - Wikimedia Tech News
The recently launched LangChain is an open-source framework for constructing applications with LLM backends; users can ask ChatGPT a question and have it reference data from the Wikidata knowledge graph before responding.

Colosseum: The Open RAN Digital Twin
Michele Polese, Leonardo Bonati, Salvatore D'Oro, Pedram Johari, Davide Villa, Sakthivel Velumani, Rajeev Gangula, Maria Tsampazi, Clifton Paul Robinson, Gabriele Gemmi, Andrea Lacava, Stefano Maxenti, Hai Cheng, Tommaso Melodia
https://arxiv.org/abs/2404.17317

Colosseum: The Open RAN Digital Twin
Recent years have witnessed the Open Radio Access Network (RAN) paradigm transforming the fundamental ways cellular systems are deployed, managed, and optimized. This shift is led by concepts such as openness, softwarization, programmability, interoperability, and intelligence of the network, all of which had never been applied to the cellular ecosystem before. The realization of the Open RAN vision into practical architectures, intelligent data-driven control loops, and efficient software impl…

This https://arxiv.org/abs/2403.15721 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

Radical-Cylon: A Heterogeneous Data Pipeline for Scientific Computing
Managing and preparing complex data for deep learning, a prevalent approach in large-scale data science can be challenging. Data transfer for model training also presents difficulties, impacting scientific fields like genomics, climate modeling, and astronomy. A large-scale solution like Google Pathways with a distributed execution environment for deep learning models exists but is proprietary. Integrating existing open-source, scalable runtime tools and data frameworks on high-performance comp…

Wow! My newsletter has 1800 subscribers now 🤩 💃
I am so happy that so many people are interested 🫶
To celebrate, I am preparing a very special issue this week:
✨ 3 simple rues for creating Open Science Policies ✨
(a collaboration with Sander Bosch)
Interested? You can have it in your inbox on Friday or view it directly on the newsletter page:

Heidi's newsletter
Are you frustrated by the way research works? Do you want to do better research and drive scientific progress? Then this newsletter is for you! Learn how to make your research reproducible. Receive useful tips about Open Science. Be the first to know about new learning material I offer (mostly for free). Receive emails with useful titbits about open and reproducible data science by signing up for free!

Join Hellmar Becker at this year's Berlin Buzzwords to learn how to track data lineage in a real-time, open source analytics pipeline. #bbuzz
https://2024.berlinbuzzwords.de/sessio

"Bei keiner der für 2024 angekündigten deutschen Open-Data-Day-Veranstaltungen findet sich das Thema Gender Data Gap.
„Die Auswirkungen geschlechtsspezifischer Datenlücken werden völlig unterschätzt“, so die Präsidentin des djb, Ursula Matthiessen-Kreuder. „Es besteht ein dringender Bedarf an nach Geschlechtern aufgeschlüsselten Daten.“"

Spent 3 days and 4 different operating systems trying to get an NVIDIA desktop GPU to act like a data center GPU in Linux. Turns out it was one kernel parameter. Woof.
#nvidia #linux #ai

EndToEndML: An Open-Source End-to-End Pipeline for Machine Learning Applications
Nisha Pillai, Athish Ram Das, Moses Ayoola, Ganga Gireesan, Bindu Nanduri, Mahalingam Ramkumar
https://arxiv.org/abs/2403.18203

EndToEndML: An Open-Source End-to-End Pipeline for Machine Learning Applications
Artificial intelligence (AI) techniques are widely applied in the life sciences. However, applying innovative AI techniques to understand and deconvolute biological complexity is hindered by the learning curve for life science scientists to understand and use computing languages. An open-source, user-friendly interface for AI models, that does not require programming skills to analyze complex biological data will be extremely valuable to the bioinformatics community. With easy access to differe…

How open are hybrid journals included in transformative agreements?
Najko Jahn
https://arxiv.org/abs/2402.18255 https://arxiv.org/pdf…

How open are hybrid journals included in transformative agreements?
The ongoing controversy surrounding transformative agreements, which aim to transition journal publishing to full open access, highlight the need for large-scale studies assessing the uptake of open access in hybrid journals. This includes evaluating the extent to which transformative agreements enabled open access. By combining publicly available data from various sources, including cOAlition S Journal Checker, Crossref, and OpenAlex, this study presents a novel approach that analyses over 700…

This is a nice statement of why #Bluesky exists and how it differs from other social networks like #X, #Threads and

StarCoder 2 and The Stack v2: The Next Generation
Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, Tianyang Liu, Max Tian, Denis Kocetkov, Arthur Zucker, Younes Belkada, Zijian Wang, Qian Liu, Dmitry Abulkhanov, Indraneil Paul, Zhuang Li, Wen-Ding Li, Megan Risdal, Jia Li, Jian Zhu, Terry Yue Zhuo, Evgenii Zheltonozhskii, Nii Osae Osae Dade, Wenhao Yu, Lucas Krau{\ss}, Naman Jain, Yixuan S…

StarCoder 2 and The Stack v2: The Next Generation
The BigCode project, an open-scientific collaboration focused on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder2. In partnership with Software Heritage (SWH), we build The Stack v2 on top of the digital commons of their source code archive. Alongside the SWH repositories spanning 619 programming languages, we carefully select other high-quality data sources, such as GitHub pull requests, Kaggle notebooks, and code documentation. This results in a…

Heute vor 39 Jahren: Am 2. April 1985 zündeten die #USA im Rahmen von Operation Grenadier die 7. Atombombe "Hermosa". Grenadier war eine Serie von #Kernwaffentests bei der 1984/85 insgesamt 16 Bomben im Testgebiet in

February 22, 2024: "Today, we’re excited to announce that the Bluesky network is federating and opening up in a way that allows you to host your own data."
https://bsky.social/about/blog/02-22-2024-open-social-web

Bluesky: An Open Social Web - Bluesky
We’re excited to announce that the Bluesky network is federating and opening up in a way that allows you to host your own data.

ein Blogbeitrag von @… zu #LOD und #GLAM|s. Neben einer technischen Hinführung wird das Potenzial auch allgemeiner herausgestellt: "Die Einrichtungen könne…

Robust Federated Contrastive Recommender System against Model Poisoning Attack
Wei Yuan, Chaoqun Yang, Liang Qu, Guanhua Ye, Quoc Viet Hung Nguyen, Hongzhi Yin
https://arxiv.org/abs/2403.20107

Robust Federated Contrastive Recommender System against Model Poisoning Attack
Federated Recommender Systems (FedRecs) have garnered increasing attention recently, thanks to their privacy-preserving benefits. However, the decentralized and open characteristics of current FedRecs present two dilemmas. First, the performance of FedRecs is compromised due to highly sparse on-device data for each client. Second, the system's robustness is undermined by the vulnerability to model poisoning attacks launched by malicious users. In this paper, we introduce a novel contrastive lea…

Bluesky federation has launched.
https://bsky.social/about/blog/02-22-2024-open-social-web

Bluesky: An Open Social Web - Bluesky
We’re excited to announce that the Bluesky network is federating and opening up in a way that allows you to host your own data.

Open Your Ears to Take a Look: A State-of-the-Art Report on the Integration of Sonification and Visualization
Kajetan Enge, Elias Elmquist, Valentina Caiola, Niklas R\"onnberg, Alexander Rind, Michael Iber, Sara Lenzi, Fangfei Lan, Robert H\"oldrich, Wolfgang Aigner
https://arxiv.org/abs/2402.16558

Open Your Ears to Take a Look: A State-of-the-Art Report on the Integration of Sonification and Visualization
The research communities studying visualization and sonification for data display and analysis share exceptionally similar goals, essentially making data of any kind interpretable to humans. One community does so by using visual representations of data, the other community does so by employing auditory (non-speech) representations of data. While the two communities have a lot in common, they developed mostly in parallel over the course of the last few decades. With this STAR, we discuss a colle…

Heute vor 73 Jahren: Am 1. April 1952 zündeten die #USA im Rahmen von Operation Tumbler–Snapper die Atombombe "Able". Tumbler–Snapper war eine Serie von #Kernwaffentests bei der 1952 insgesamt 8 Bomben im Testgebiet in

This https://arxiv.org/abs/2401.06788 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

The NPU-ASLP-LiAuto System Description for Visual Speech Recognition in CNVSRC 2023
This paper delineates the visual speech recognition (VSR) system introduced by the NPU-ASLP-LiAuto (Team 237) in the first Chinese Continuous Visual Speech Recognition Challenge (CNVSRC) 2023, engaging in the fixed and open tracks of Single-Speaker VSR Task, and the open track of Multi-Speaker VSR Task. In terms of data processing, we leverage the lip motion extractor from the baseline1 to produce multi-scale video data. Besides, various augmentation techniques are applied during training, enco…

Snowflake announces Arctic, an LLM optimized for enterprise tasks such as SQL generation, coding, and instruction following, with an Apache 2.0 license (Shubham Sharma/VentureBeat)
https://venturebeat.com/data-infrastru

"Access & use of #publicsector documents & political speeches are key to a thriving #democracy and civil society as they ensure transparency and accountability. Copyright & paywalls, however, can render access to these materials difficult or expensive.
Today we are proposing EU …

Access to and use of public sector documents and public speeches – new policy paper
Despite the importance of public sector documents as well as public speeches, they are not necessarily publicly available as they can be protected under copyright and hidden behind paywalls. Where this is the case, access to, and use of, these materials can be difficult or prohibitively expensive. This paper addresses many areas where existing European legislation, notably the Open Data Directive, falls short, and presents European-level policy options to remove copyright barriers and ensure th…

This https://arxiv.org/abs/2402.00786 has been replaced.
link: https://scholar.google.com/scholar?q=a

CroissantLLM: A Truly Bilingual French-English Language Model
We introduce CroissantLLM, a 1.3B language model pretrained on a set of 3T English and French tokens, to bring to the research and industrial community a high-performance, fully open-sourced bilingual model that runs swiftly on consumer-grade local hardware. To that end, we pioneer the approach of training an intrinsically bilingual model with a 1:1 English-to-French pretraining data ratio, a custom tokenizer, and bilingual finetuning datasets. We release the training dataset, notably containin…

Cycling on the Freeway: The Perilous State of Open Source Neuroscience Software
Britta U. Westner, Daniel R. McCloy, Eric Larson, Alexandre Gramfort, Daniel S. Katz, Arfon M. Smith, invited co-signees
https://arxiv.org/abs/2403.19394

Cycling on the Freeway: The Perilous State of Open Source Neuroscience Software
Most scientists need software to perform their research (Barker et al., 2020; Carver et al., 2022; Hettrick, 2014; Hettrick et al., 2014; Switters and Osimo, 2019), and neuroscientists are no exception. Whether we work with reaction times, electrophysiological signals, or magnetic resonance imaging data, we rely on software to acquire, analyze, and statistically evaluate the raw data we obtain - or to generate such data if we work with simulations. In recent years there has been a shift toward …

"Welche Vorteile Linked Open Data für Kulturerbe hat" @wikimediaDE: #LinkedData

Study on the Temporal Evolution of Literature Bradford Curves in the Context of Library Specialization
Haobai Xue, Xian Liu
https://arxiv.org/abs/2404.19267 https://arxiv.org/pdf/2404.19267
arXiv:2404.19267v1 Announce Type: new
Abstract: The Bradford's law of bibliographic scattering is a fundamental law in bibliometrics and can provide valuable guidance to academic libraries in literature search and procurement. However, the Bradford's curves can take various shapes at different time points and there is still a lack of causal explanation for it, so the prediction of its shape is still an open question. This paper attributes the deviation of Bradford curve from the theoretical J-shape to the integer constraints of the journal number and article number, and extends the Leimkuhler and Egghe's formula to cover the core region of very productive journals, where the theoretical journal number of which fall below one. The key parameters of the extended formula are identified and studied by using the Simon-Yule model. The reasons for the Groos Droop are explained and the critical point for the shape change are studied. Finally, the proposed formulae are validated with the empirical data found in the literature. It is found that the proposed method can be used to predict the evolution of Bradford's curves and thus guide the academic library for scientific literature procurement and utilization.

Radical-Cylon: A Heterogeneous Data Pipeline for Scientific Computing
Arup Kumar Sarker, Aymen Alsaadi, Niranda Perera, Mills Staylor, Gregor von Laszewski, Matteo Turilli, Ozgur Ozan Kilic, Mikhail Titov, Andre Merzky, Shantenu Jha, Geoffrey Fox
https://arxiv.org/abs/2403.15721

Radical-Cylon: A Heterogeneous Data Pipeline for Scientific Computing
Managing and preparing complex data for deep learning, a prevalent approach in large-scale data science can be challenging. Data transfer for model training also presents difficulties, impacting scientific fields like genomics, climate modeling, and astronomy. A large-scale solution like Google Pathways with a distributed execution environment for deep learning models exists but is proprietary. Integrating existing open-source, scalable runtime tools and data frameworks on high-performance comp…

Data-driven Energy Consumption Modelling for Electric Micromobility using an Open Dataset
Yue Ding, Sen Yan, Maqsood Hussain Shah, Hongyuan Fang, Ji Li, Mingming Liu
https://arxiv.org/abs/2403.17632

Data-driven Energy Consumption Modelling for Electric Micromobility using an Open Dataset
The escalating challenges of traffic congestion and environmental degradation underscore the critical importance of embracing E-Mobility solutions in urban spaces. In particular, micro E-Mobility tools such as E-scooters and E-bikes, play a pivotal role in this transition, offering sustainable alternatives for urban commuters. However, the energy consumption patterns for these tools are a critical aspect that impacts their effectiveness in real-world scenarios and is essential for trip planning…

Apparently even the European Data Protection Supervisor #EDPS , can't find the funds for it's mastodon servers. Which is bizar if you think about all the "talk" about not relying on #bigtech , a more independent EU, open standards, security, importance of reliable information etc.

EDPS decentralised social media pilot: the end of a successful story
Two years ago, the EDPS embarked on a pioneering journey to launch two social media platforms, EU Voice and EU Video. The pilot project has proved successful in delivering alternative, privacy-friendly and user-focused social media platforms. It is time to review the results.On 28 April 2022, ...

Schönes Praxisbeispiel für Linked Open Data: Man braucht LOD, damit Bobbi, der Berliner Chatbot-Bär, den Namen des aktuellen Regierenden Bürgermeisters kennt.
https://odis-berlin.de/aktuelles/2024-03-28-linked-open-data/

Vom Informationssilo zum Wissensnetzwerk
Am Beispiel der Organigramme arbeiten wir derzeit an einer praxisnahen Fallstudie, um zu lernen, wie mit wichtigen Basis-Informationen die Grundlage für Linked Open Data in Berlin gelegt werden kann.

Do Large Language Models Understand Conversational Implicature -- A case study with a chinese sitcom
Shisen Yue, Siyuan Song, Xinyuan Cheng, Hai Hu
https://arxiv.org/abs/2404.19509 https://arxiv.org/pdf/2404.19509
arXiv:2404.19509v1 Announce Type: new
Abstract: Understanding the non-literal meaning of an utterance is critical for large language models (LLMs) to become human-like social communicators. In this work, we introduce SwordsmanImp, the first Chinese multi-turn-dialogue-based dataset aimed at conversational implicature, sourced from dialogues in the Chinese sitcom $\textit{My Own Swordsman}$. It includes 200 carefully handcrafted questions, all annotated on which Gricean maxims have been violated. We test eight close-source and open-source LLMs under two tasks: a multiple-choice question task and an implicature explanation task. Our results show that GPT-4 attains human-level accuracy (94%) on multiple-choice questions. CausalLM demonstrates a 78.5% accuracy following GPT-4. Other models, including GPT-3.5 and several open-source models, demonstrate a lower accuracy ranging from 20% to 60% on multiple-choice questions. Human raters were asked to rate the explanation of the implicatures generated by LLMs on their reasonability, logic and fluency. While all models generate largely fluent and self-consistent text, their explanations score low on reasonability except for GPT-4, suggesting that most LLMs cannot produce satisfactory explanations of the implicatures in the conversation. Moreover, we find LLMs' performance does not vary significantly by Gricean maxims, suggesting that LLMs do not seem to process implicatures derived from different maxims differently. Our data and code are available at https://github.com/sjtu-compling/llm-pragmatics.

ein Blogbeitrag von @… zu #LOD und #GLAM|s. Neben einer technischen Hinführung wird das Potenzial auch allgemeiner herausgestellt: "Die Einrichtungen könne…

Studying LLM Performance on Closed- and Open-source Data
Toufique Ahmed, Christian Bird, Premkumar Devanbu, Saikat Chakraborty
https://arxiv.org/abs/2402.15100

Studying LLM Performance on Closed- and Open-source Data
Large Language models (LLMs) are finding wide use in software engineering practice. These models are extremely data-hungry, and are largely trained on open-source (OSS) code distributed with permissive licenses. In terms of actual use however, a great deal of software development still occurs in the for-profit/proprietary sphere, where the code under development is not, and never has been, in the public domain; thus, many developers, do their work, and use LLMs, in settings where the models may…

Nice overview about LLMs for data annotation including paper references of papers with open source code & data.
Zhen Tan et al, Large Language Models for Data Annotation: A Survey, https://arxiv.org/abs/2402.13446

Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation
Yuan Ge, Yilun Liu, Chi Hu, Weibin Meng, Shimin Tao, Xiaofeng Zhao, Hongxia Ma, Li Zhang, Hao Yang, Tong Xiao
https://arxiv.org/abs/2402.18191

Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation
With contributions from the open-source community, a vast amount of instruction tuning (IT) data has emerged. Given the significant resource allocation required by training and evaluating models, it is advantageous to have an efficient method for selecting high-quality IT data. However, existing methods for instruction data selection have limitations such as relying on fragile external APIs, being affected by biases in GPT models, or reducing the diversity of the selected instruction dataset. I…

An analysis of the effects of sharing research data, code, and preprints on citations
Giovanni Colavizza, Lauren Cadwallader, Marcel LaFlamme, Gr\'egory Dozot, St\'ephane Lecorney, Daniel Rappo, Iain Hrynaszkiewicz
https://arxiv.org/abs/2404.16171 https://arxiv.org/pdf/2404.16171
arXiv:2404.16171v1 Announce Type: new
Abstract: Calls to make scientific research more open have gained traction with a range of societal stakeholders. Open Science practices include but are not limited to the early sharing of results via preprints and openly sharing outputs such as data and code to make research more reproducible and extensible. Existing evidence shows that adopting Open Science practices has effects in several domains. In this study, we investigate whether adopting one or more Open Science practices leads to significantly higher citations for an associated publication, which is one form of academic impact. We use a novel dataset known as Open Science Indicators, produced by PLOS and DataSeer, which includes all PLOS publications from 2018 to 2023 as well as a comparison group sampled from the PMC Open Access Subset. In total, we analyze circa 122'000 publications. We calculate publication and author-level citation indicators and use a broad set of control variables to isolate the effect of Open Science Indicators on received citations. We show that Open Science practices are adopted to different degrees across scientific disciplines. We find that the early release of a publication as a preprint correlates with a significant positive citation advantage of about 20.2% on average. We also find that sharing data in an online repository correlates with a smaller yet still positive citation advantage of 4.3% on average. However, we do not find a significant citation advantage for sharing code. Further research is needed on additional or alternative measures of impact beyond citations. Our results are likely to be of interest to researchers, as well as publishers, research funders, and policymakers.

Situating Data Sets: Making Public Data Actionable for Housing Justice
Anh-Ton Tran, Grace Guo, Jordan Taylor, Katsuki Chan, Elora Raymond, Carl DiSalvo
https://arxiv.org/abs/2402.12505

Situating Data Sets: Making Public Data Actionable for Housing Justice
Activists, governmentsm and academics regularly advocate for more open data. But how is data made open, and for whom is it made useful and usable? In this paper, we investigate and describe the work of making eviction data open to tenant organizers. We do this through an ethnographic description of ongoing work with a local housing activist organization. This work combines observation, direct participation in data work, and creating media artifacts, specifically digital maps. Our interpretation…

Radical-Cylon: A Heterogeneous Data Pipeline for Scientific Computing
Arup Kumar Sarker, Aymen Alsaadi, Niranda Perera, Mills Staylor, Gregor von Laszewski, Matteo Turilli, Ozgur Ozan Kilic, Mikhail Titov, Andre Merzky, Shantenu Jha, Geoffrey Fox
https://arxiv.org/abs/2403.15721

Radical-Cylon: A Heterogeneous Data Pipeline for Scientific Computing
Managing and preparing complex data for deep learning, a prevalent approach in large-scale data science can be challenging. Data transfer for model training also presents difficulties, impacting scientific fields like genomics, climate modeling, and astronomy. A large-scale solution like Google Pathways with a distributed execution environment for deep learning models exists but is proprietary. Integrating existing open-source, scalable runtime tools and data frameworks on high-performance comp…

Elon Musk says xAI plans to open-source Grok this week; xAI released Grok in November 2023, offering access to "real-time" data via a $16/month X subscription (Manish Singh/TechCrunch)
https://techcrunch.com/2024/03/11/elon-musk-says…

Elon Musk to open-source AI chatbot Grok this week | TechCrunch
Elon Musk's AI startup xAI will open-source Grok, its chatbot rivaling ChatGPT, this week, the entrepreneur said, days after suing OpenAI and complaining Elon Musk's AI startup xAI will open-source Grok, its chatbot rivaling ChatGPT, this week, he said.

Apparently even the European Data Protection Supervisor #EDPS , can't find the funds for it's mastodon servers. Which is bizar if you think about all the "talk" about not relying on #bigtech , a more independent EU, open standards, security, importance of reliable information etc.

EDPS decentralised social media pilot: the end of a successful story
Two years ago, the EDPS embarked on a pioneering journey to launch two social media platforms, EU Voice and EU Video. The pilot project has proved successful in delivering alternative, privacy-friendly and user-focused social media platforms. It is time to review the results.On 28 April 2022, ...

Heute vor 32 Jahren: Am 30. April 1992 zündeten die #USA im Rahmen von Operation Julin die 3. Atombombe "Diamond Fortune". Julin war eine Serie von #Kernwaffentests bei der 1991/92 insgesamt 9 Bomben im Testgebiet in

Worldcoin announces Personal Custody, which saves biometric data captured by the Orb on users' personal devices, and plans to open source the Orb's software (RT Watson/The Block)
https://www.theblock.co/post/284123/worldcoin-to-end-storing-…

Heute vor 55 Jahren: Am 30. April 1969 zündeten die #USA im Rahmen von Operation Bowline die Atombomben "Blenton" & "Thistle". Bowline war eine Serie von #Kernwaffentests bei der 68/69 insgesamt 58 Bomben im Testgebiet in

This https://arxiv.org/abs/2310.18011 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDL_…

Data journeys in popular science: Producing climate change and COVID-19 data visualizations at Scientific American
Vast amounts of (open) data are increasingly used to make arguments about crisis topics such as climate change and global pandemics. Data visualizations are central to bringing these viewpoints to broader publics. However, visualizations often conceal the many contexts involved in their production, ranging from decisions made in research labs about collecting and sharing data to choices made in editorial rooms about which data stories to tell. In this paper, we examine how data visualizations a…

Heute vor 49 Jahren: Am 30. April 1975 testen die #USA die Atombombe "Obar". Die Operation Bedrock war eine Serie von 27 US-amerikanischen #Kernwaffentests, die 1974/75 auf der Nevada Test Site in Nevada unterirdisch durchgeführt wurde.

Cohere's nonprofit research lab releases its open-source multilingual LLM Aya, which it says can follow instructions in more than 100 languages (Alison Snyder/Axios)
https://www.axios.com/2024/02/13/open-source-ai-languages

New AI polyglot launched to help fill massive language gap in field
Most models that power today's generative AI tools are trained on data in English and Chinese, leaving a massive gap of thousands of languages.

Learning to Generate Instruction Tuning Datasets for Zero-Shot Task Adaptation
Nihal V. Nayak, Yiyang Nan, Avi Trost, Stephen H. Bach
https://arxiv.org/abs/2402.18334

Learning to Generate Instruction Tuning Datasets for Zero-Shot Task Adaptation
We introduce Bonito, an open-source model for conditional task generation: the task of converting unannotated text into task-specific training datasets for instruction tuning. Our goal is to enable zero-shot task adaptation of large language models on users' specialized, private data. We train Bonito on a new large-scale dataset with 1.65M examples created by remixing existing instruction tuning datasets into meta-templates. The meta-templates for a dataset produce training examples where the i…

Heute vor 37 Jahren: Am 30.04.1987 zündeten die #USA im Rahmen von Operation Musketeer die Atombombe "Hardin". Musketeer war eine Serie von #Kernwaffentests bei der 1986/87 insgesamt 16 Bomben im Testgebiet in

This https://arxiv.org/abs/2402.05211 has been replaced.
link: https://scholar.google.com/scholar?q=a

A Maturity Model for Urban Dataset Meta-data
In the current environment of data generation and publication, there is an ever-growing number of datasets available for download. This growth precipitates an existing challenge: sourcing and integrating relevant datasets for analysis is becoming more complex. Despite efforts by open data platforms, obstacles remain, predominantly rooted in inadequate metadata, unsuitable data presentation, complications in pinpointing desired data, and data integration. This paper delves into the intricacies o…

Cohere's nonprofit research lab releases its open-source multilingual LLM Aya, which it says can follow instructions in more than 100 languages (Alison Snyder/Axios)
https://www.axios.com/2024/02/13/open-source-ai-languages

New AI polyglot launched to help fill massive language gap in field
Most models that power today's generative AI tools are trained on data in English and Chinese, leaving a massive gap of thousands of languages.

This https://arxiv.org/abs/2310.18011 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDL_…

Data journeys in popular science: Producing climate change and COVID-19 data visualizations at Scientific American
Vast amounts of (open) data are increasingly used to make arguments about crisis topics such as climate change and global pandemics. Data visualizations are central to bringing these viewpoints to broader publics. However, visualizations often conceal the many contexts involved in their production, ranging from decisions made in research labs about collecting and sharing data to choices made in editorial rooms about which data stories to tell. In this paper, we examine how data visualizations a…

Balanced Data Sampling for Language Model Training with Clustering
Yunfan Shao, Linyang Li, Zhaoye Fei, Hang Yan, Dahua Lin, Xipeng Qiu
https://arxiv.org/abs/2402.14526

Balanced Data Sampling for Language Model Training with Clustering
Data plays a fundamental role in the training of Large Language Models (LLMs). While attention has been paid to the collection and composition of datasets, determining the data sampling strategy in training remains an open question. Most LLMs are trained with a simple strategy, random sampling. However, this sampling strategy ignores the unbalanced nature of training data distribution, which can be sub-optimal. In this paper, we propose ClusterClip Sampling to balance the text distribution of t…