Neura News - AI News | Neura Market

Study Tests Whether Transformers Need All Three QKV Projections

A new paper systematically evaluates three variants of projection sharing in transformer attention: shared key-value, shared query-key, and a single projection. The authors found that sharing key and value projections performs on par with standard QKV attention while reducing KV cache by 50% with only 3.1% perplexity degradation. Combining this with grouped-query or multi-query attention can cut cache by up to 96.9%, enabling practical on-device inference.

Jun 53 minNeura News

AI Models

Bonsai Image 4B Brings Image Generation to Local Devices

PrismML has released Bonsai Image 4B, a family of compact image-generation models that run on local devices like laptops and phones. The models use binary and ternary weight representations to shrink the diffusion transformer to under 1 GB, enabling on-device inference on iPhone 17 Pro Max and other Apple Silicon hardware. Two variants offer different trade-offs between footprint and quality, retaining up to 95% of the accuracy of the full-precision FLUX.2 Klein 4B base model.

May 314 minNeura News

AI Models

Alibaba Qwen-Image-2.0 Doubles Compression, Cuts Steps to 4

Alibaba's Qwen-Image-2.0 model achieves twice the image compression and reduces generation steps from 40 to 4 through key technical advances. It features a more efficient VAE, updated transformer architecture, and a prompt expansion module. The model excels in benchmarks and handles complex outputs like posters with text.

May 144 minNeura News

Developer

Workshop to Train GPT Model from Scratch on Laptop

A GitHub project provides a hands-on workshop for building a full GPT training pipeline. Users write code for tokenization, transformer architecture, training loop, and text generation to train a 10 million parameter model on a laptop in under an hour. Designed for a single session, it uses character-level tokenization on Shakespeare data and supports Apple Silicon, NVIDIA GPUs, or CPU.

May 55 minNeura News

Automation

Jerry Tworek Launches Core Automation AI Lab

Former OpenAI researcher Jerry Tworek has started Core Automation, a new AI lab aimed at creating the most automated AI lab in the world by first automating its own research processes. The lab focuses on new learning algorithms that surpass pre-training and reinforcement learning methods, along with architectures that scale more effectively than transformers. Tworek, who left OpenAI after seven years in January 2026, joins other ex-OpenAI leaders in forming independent labs seeking fresh approaches to AI progress.

Apr 223 minNeura News