Tootfinder

Opt-in global Mastodon full text search. Join the index!

@loleg@fosstodon.org
2024-11-12 13:24:23

One week to go with last places up for grabs! Join us at DINAcon 2024 next week in Bern to hear about:
💠 AI in Parliament: A Roadmap for the Parliamentary Library { Jacqueline Kucera }
🌸 How can large language models be used for sustainable procurement? { Marcel Gygli, Luca Rolshoven }
🌿 Source code sovereignty: local alternatives for an independent digital future { Tobias Brunner }
🧑‍⚖️ My workshop on open source contracts with Simon Schlauri
🌃

DINAcon logo and conference banner - future models for a digital, sovereign and sustainable Switzerland
@theawely@mamot.fr
2024-12-13 18:48:37

Excited about the new xLSTM model release. There are many well-though designs compared to transformers: recurrence (which should allows composability), gating (like Mamba & LSTM which is based on, which allows time complexity independent of the input size), state tracking (unlike Mamba & transformers). For now, these advantage aren’t apparent on benchmarks, but most training techniques are secrets, and the recent advances of LLMs evidenced that they matter a lot.