ML & AI news of the week

Photo by Priscilla Du Preez 🇨🇦 on Unsplash

A collection of the best ML & AI news every week (research, news, resources). Star this repository if you find it useful.

Here, you can find articles and tutorials about artificial intelligence Here some reviews on specific artificial intelligence topics

For each week you will find different sections:

• Research: the most important published research of the week.
• News: the most important news related to companies, institutions, and much more.
• Resources: released resources for artificial intelligence and machine learning.
• Perspectives: a collection of deep and informative articles about open questions in artificial intelligence.

and a meme for starting well the week.

Suggestions and corrections

Feel free to open an issue if you find some errors, if you have any suggestions, topics, or any other comments

Index

2023

• ML news: Week 18 - 24 December
• ML news: Week 11 - 17 December
• ML news: Week 4 - 10 December
• ML news: Week 27 November - 3 December
• ML news: Week 20-26 November
• ML news: Week 12-19 November
• ML news: Week 6-12 October
• ML news: Week 30 October - 5 November
• ML news: Week 23-29 October

2023

ML news: Week 18 - 24 December

Research

Link	description
Stabilizing Transformer Training by Preventing Attention Entropy Collapse.	Despite their incredible skills, transformers may be challenging to train because of their numerous instabilities. When the entropy of an Attention matrix collapses, it is one of the primary problems. With a straightforward reparametrization, our work offers a means to avoid it.
DiffusionLight: Light Probes for Free by Painting a Chrome Ball.	This effort overcomes the drawbacks of existing approaches that rely on HDR panorama datasets by introducing a unique method for predicting lighting in photos. The method uncovers a distinct link between chrome balls and diffusion noise by rendering chrome balls into conventional pictures using diffusion models.
DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving.	A new framework called DriveMLM takes advantage of massive language models to improve autonomous driving. This system performs better in simulations and interacts with current autonomous driving systems. It does this by fusing linguistic judgments with vehicle controls.
Graph Neural Networks with Diverse Spectral Filtering.	A novel technique known as DSF has been created by researchers to enhance spectral graph neural networks. The World Wide Web and other complicated networks can be handled more effectively by DSF by adding node-specific filter weights.
Evaluating and Mitigating Discrimination in Language Model Decisions.	A proactive approach to assessing language models' potential for discrimination is covered in this article. The process involves coming up with a broad range of possible prompts for different decision scenarios and variations in demographic data. The main tactic for reducing both positive and negative discrimination is cautious prompt engineering.
Faster Diffusion: Rethinking the Role of UNet Encoder in Diffusion Models.	A comparison of UNet encoders and decoders in diffusion models demonstrates the former's more stable behavior. Thanks to this realization, a novel encoder propagation strategy was developed, greatly accelerating jobs like text-to-image and text-to-video production.
VideoPoet: A large language model for zero-shot video generation.	VideoPoet, a large language model (LLM) that is capable of a wide variety of video generation tasks, including text-to-video, image-to-video, video stylization, video inpainting and outpainting, and video-to-audio.
Unmasking Deepfake Faces from Videos Using An Explainable Cost-Sensitive Deep Learning Approach.	a deep learning method for identifying deepfake faces in videos utilizing four pre-trained CNN models for high accuracy. official code.
Bi-directional Adapter for Multi-modal Tracking.	This effort solves the drawbacks of single-modal object tracking by introducing a multi-modal visual cue tracking paradigm that dynamically leverages the advantages of several modalities, such as RGB and infrared. official code.
Tokenize Anything via Prompting.	We present Tokenize Anything via Prompting, a unified and promptable model capable of simultaneously segmenting, recognizing, and captioning arbitrary regions, with flexible visual prompts (point, box, and sketch).
Gemini: A Family of Highly Capable Multimodal Models.	Gemini Paper: This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning.	Diffusion-based FontDiffuser is an automatic font production technique that works especially well with intricate characters and a wide range of style variants. It has a Style Contrastive Refinement module for style transfer and a Multi-scale Content Aggregation block for improved stroke preservation.
Splatter Image: Ultra-Fast Single-View 3D Reconstruction.	The Splatter Image is an ultra-fast method for single- and few-view 3D reconstruction. Training is done on 1 GPU, reconstruction at 38 FPS, and rendering at 588 FPS.
PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU.	The hypothesis that models include cool neurons that are utilized significantly less frequently and hot neurons that are used for nearly all inputs is investigated in this research. It is possible to conserve memory without significantly reducing throughput by preloading the hot neurons to the GPU. There's a code library companion available.
On Inference Stability for Diffusion Models.	A 'sequence-aware' loss function has been created by researchers to enhance Denoising Probabilistic Models (DPMs) and solve the problem of timestep correlation in picture production. Better FID and Inception Scores demonstrate that this new method provides a tighter estimation of loss and significantly improves picture quality on datasets such as CelebA and CIFAR10.
CLIP-DINOiser: Teaching CLIP a few DINO tricks.	For better semantic segmentation without annotations, the novel CLIP-DINOiser approach combines self-supervised features with the zero-shot capabilities of the CLIP model.
A Unified Framework for High-quality Text Synthesis in Arbitrary Images via Character-aware Diffusion Models.	A novel technique called UDiffText improves the legibility of text in AI-generated graphics. Through the use of a sophisticated text encoder and extensive dataset fine-tuning, UDiffText enhances text correctness and dramatically lowers spelling errors.

News

Link	description
Snapchat+ subscribers can now create and send AI-generated images.	Snapchat is releasing a few new AI powered features for Snapchat+ subscribers.
Google brings Gemini Pro to Vertex AI.	After coming to Bard and the Pixel 8 Pro last week, Gemini, Google’s recently announced flagship GenAI model family, is launching for Google Cloud customers using Vertex AI.
Competitive performance claims and industry-leading Inference performance on AMD Instinct MI300X.	With ROCm 6, AMD's flagship AI accelerator, the MI300X, can now perform for inference tasks on par with NVIDIA. The community will benefit greatly from this as it gives burgeoning AI firms access to new chips.
Agility is using large language models to communicate with its humanoid robots.	the company is showcasing some of that work in a short video shared through its social channels.
Intel unveils new data center chip with focus on AI growth.	On Thursday, the company debuted its 5th Gen Intel Xeon processors during its AI Everywhere event
ByteDance is secretly using OpenAI’s tech to build a competitor.	‘They really just don’t want to get caught.’ The frenzied race to win in generative AI means that even the biggest players are cutting corners.
Atlassian welcomes AI to the team.	AI capabilities are now generally available across Jira Software, Confluence, Jira Service Management, and more.
Harvey raises $80M Series B.	A new round of investment has been secured by Harvey AI, a legal service based on OpenAI technology, valuing the startup business at over $700 million. Using OpenAI, the business creates foundation models for applications related to law and legal practice.
whiterabbitneo/WhiteRabbitNeo-13B.	The Matrix-related startup whiterabbitneo has produced a 13B parameter language model that has been trained for both offensive and defensive cyber security. It is qualified to respond to inquiries and offer details on computer security.
Anthropic Updates Terms of Service.	We are introducing new, simplified Commercial Terms of Service with an expanded copyright indemnity, as well as an improved developer experience with our beta Messages API.
Pilotless FedEx, Reliable Robotics Plane Completes Flight.	Milestone for Autonomous Flight Could Lead to Regulatory Approval
Microsoft Copilot gets a music creation feature via Suno integration.	Microsoft Copilot, Microsoft’s AI-powered chatbot, can now compose songs thanks to an integration with GenAI music app Suno.
Stability AI announces paid membership for commercial use of its models.	The company said paid tiers will fund the future of its AI research.
More than 10,000 research papers were retracted in 2023 — a new record.	The number of articles being retracted rose sharply this year. Integrity experts say that this is only the tip of the iceberg.
Large language models direct automated chemistry laboratory.	Automation of chemistry research has focused on developing robots to execute jobs. Artificial intelligence technology has now been used not only to control robots but also to plan their tasks on the basis of simple human prompts. research article.
Introducing Text-to-CAD.	After changing its name, Zoo Dev (formerly Kitty Cad) unveiled a new text-to-cad feature. This robust platform creates 3D assets that can be printed or used as parts.
Waymo finds that its driverless cars ‘significantly outperformed’ humans.	New safety research from Waymo finds that its driverless cars “led to a significant reduction in the rates of police-reported and injury-causing crashes compared to human drivers.”
Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Model.	A novel training-free framework called Diff-Text enables the creation of photo-realistic images with text in any language. Using sketched images as priors, it improves the multilingual capabilities of the Stable Diffusion model.
Largest Dataset Powering AI Images Removed After Discovery of Child Sexual Abuse Material.	The model is a massive part of the AI-ecosystem, used by Stable Diffusion and other major generative AI products. The removal follows discoveries made by Stanford researchers, who found thousands of instances of suspected child sexual abuse material in the dataset.
Rite Aid banned from using facial recognition software after falsely identifying shoplifters.	FTC says the company's 'reckless use' of AI humiliated customers
AI startup Anthropic reportedly in talks to raise $750M on a $15B valuation.	Anthropic PBC, an artificial intelligence startup backed by Amazon.com Inc. and Google LLC, is reportedly in talks to raise $750 million in new funding at a valuation of $15 billion
Apple’s latest AI research could completely transform your iPhone.	two new papers introducing new techniques for 3D avatars and efficient language model inference. The advancements could enable more immersive visual experiences and allow complex AI systems to run on consumer devices such as the iPhone and iPad.
OpenAI buffs safety team and gives board veto power on risky AI.	A new safety advisory group has been established by OpenAI, and the board has been given veto power over all models.

Resources

Link	description
Introducing Ego-Exo4D: A foundational dataset for research on video learning and multimodal perception.	Ego-Exo4D, a foundational dataset and benchmark suite to support research on video learning and multimodal perception.
Coffee.	Coffee, which was released last week, integrates AI into current codebases to speed up front-end development. The project primarily focuses on a first-class DX, drawing on insights the Coframe team has gained from producing more than 80% of their front-end using AI.
DeepEval.	DeepEval is a simple-to-use, open-source evaluation framework for LLM applications. It is similar to Pytest but specialized for unit testing LLM applications. DeepEval evaluates performance based on metrics such as hallucination, answer relevancy, RAGAS, etc., using LLMs and various other NLP models locally on your machine.
Understanding GPU Memory 1: Visualizing All Allocations over Time.	Determining the reason behind memory leaks in Pytorch has proven to be one of the most difficult tasks for practitioners. Pytorch 2.1 has some incredible new features that provide insight into memory utilization. Classifying the utilization into well-known buckets (such as activations and gradients) is another application for it.
Fine Tuning Mistral 7B on Magic the Gathering Drafts.	Tips, examples, and thoughts from an exploration of the world of fine tuning
MMLU prompt templates.	The most effective prompting technique for MMLU at the moment is Microsoft's Medprompt+. The template, along with numerous other chain-of-thought style templates that are widely used in the evaluation community, was made available by Microsoft.
Amphion.	Amphion (/æmˈfaɪən/) is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development. Amphion offers a unique feature: visualizations of classic models or architectures.
Big Vision.	This codebase is designed for training large-scale vision models using Cloud TPU VMs or GPU machines. It is based on Jax/Flax libraries and uses tf. data and TensorFlow Datasets for scalable and reproducible input pipelines.
Thoughts on Jaxtyping.	In machine learning, shape problems are difficult to debug and can go undetected until the model is attempted to run. Checking shapes as a kind can help you get over most of these obstacles and advance faster.
Legaltech x AI: The Lightspeed View.	Lightspeed's perspective on the legal tech industry's use of AI is obvious and intriguing when viewed from the angle that what's good for VCs is good for everyone. Time will tell if they are on the right track or not, but there are some intriguing observations.
New Sequence Mixers.	The folks who brought us Mamba (and a ton of other models) have published a nice blog entry explaining simple sequence mixing topologies that provide some very significant speed increases over traditional Transformers.
microagents.	Agents Capable of Self-Editing Their Prompts / Python Code
LLMLingua.	LLMLingua utilizes a compact, well-trained language model (e.g., GPT2-small, LLaMA-7B) to identify and remove non-essential tokens in prompts. This approach enables efficient inference with large language models (LLMs), achieving up to 20x compression with minimal performance loss.
Distil-Whisper.	Distil-Whisper is a distilled version of Whisper that is 6 times faster, 49% smaller, and performs within 1% word error rate (WER) on out-of-distribution evaluation sets
M3DBench: Let's Instruct Large Models with Multi-modal 3D Prompts.	M3DBench introduces a comprehensive 3D instruction-following dataset that encompasses a variety of 3D vision-centric tasks, spanning fundamental abilities in real-world 3D environments.
WhisperPlus: Advancing Speech-to-Text Processing.	Advanced speech-to-text processing
tinyzero.	Easily train AlphaZero-like agents in any environment you want!
MossFormer2: Combining Transformer and RNN-Free Recurrent Network for Enhanced Time-Domain Monaural Speech Separation.	An improvement on the original MossFormer, the MossFormer2 model provides better monaural speech separation capabilities.
llama-recipes.	The 'llama-recipes' repository is a companion to the Llama 2 model. This repository aims to provide examples to quickly get started with fine-tuning for domain adaptation and how to run inference for the fine-tuned models.
LLaVA-Interactive.	LLaVA-Interactive is a large language-and-vision assistant demo, dedicated to demonstrating the possibilities of multimodal human-machine interaction: visual input, visual output, and visual interaction. It combines complementary skills from three models: visual chat of LLaVA, visual prompt for segmentation from SEEM, and visual prompt for image generation/editing from GLIGEN.
Whisper Turbo.	Whisper Turbo is a fast, cross-platform Whisper implementation, designed to run entirely client-side in your browser/electron app.

Perspectives

Link	description
The Where, When, and How of AI with Theory Ventures, Open AI, MotherDuck and Lamini.	Prominent inventors and venture capitalists discuss the latest developments in artificial intelligence, ranging from the use of LLMs to enterprise innovation. This is a helpful fast summary if the speed of "things you should know about AI" is a little overwhelming.
The Competition is Coming for Nvidia.	After a long, largely unimpeded run, NVIDIA’s challenge has finally arrived.
From Einstein to AI: how 100 years have shaped science.	Looking back a century reveals how much the research landscape has changed — and how unclear the consequences of scientific innovation can be.
ChatGPT and science: the AI system was a force in 2023 — for good and bad.	The poster child for generative AI software is a startling human mimic. It represents a potential new era in research but brings risks.
What was the Turing test actually about?.	It is important to develop metrics for the public scrutiny of today’s generative artificial intelligence.
Should scientists delegate their writing to ChatGPT?.	Scientists should exercise caution when using generative artificial intelligence (AI) tools such as ChatGPT to write grant applications
Mentor–trainee dialogue on proper use of AI tools.	The responsible use of artificial intelligence (AI) tools in education and academia is important on a micro- as well as a macro scale
My jaw hit the floor when I watched an AI master one of the world's toughest physical games in just six hours.	An AI just mastered Labyrinth in six hours, and I am questioning my own existence.
Meta’s CTO on how the generative AI craze has spurred the company to ‘change it up’.	Andrew Bosworth, Chief Technology Officer at Meta, discusses the company's future plans and the hype around artificial intelligence.
End of YearPay Report 2023.	Levels.fyi's annual compensation report. View top-paying companies, cities, titles & other trends.
Year One of Generative AI: Six Key Trends.	In this post, drawing on countless founder meetings and pitch decks, we distill our first-hand learnings into six trends that have defined the generative AI space throughout 2023 and that are set to shape its trajectory in 2024.
Marketplaces in the Age of AI.	For the past 20 years, marketplaces have dominated company models. This is a summary from a16z discussing their predictions on how AI would affect this kind of business. Customizing the experience for both sides of the marketplace is the main concept.

Back to index

ML news: Week 11 - 17 December

Research

Link	description
RAVE: Randomized Noise Shuffling for Fast and Consistent Video Editing with Diffusion Models.	RAVE is a technique for video editing that improves videos by utilizing pre-existing text-to-image diffusion models. With this method, it is possible to achieve excellent video edits without sacrificing the original composition and flow.
Language-driven Scene Synthesis using Multi-conditional Diffusion Model.	Textual cues give scene creation—which is influenced by things like human movement or room design—a new perspective. This repository presents a new method that effectively combines text, movement, and pre-existing objects: a multi-conditional diffusion model.
Audiobox: Unified Audio Generation with Natural Language Prompts.	Meta has hinted at an AI model for audio foundations. The paper, along with more samples and powerful demos, has been made available. Producing controlled audio content with styles derived from the same model was the primary objective of the project.
BioCLIP: A Vision Foundation Model for the Tree of Life.	A vision model with applications in biology in mind. On certain physiological tasks, it performs about 20% better than OpenAI's clip. There is also a training set of 10 million paired images and text.
Diversifying Spatial-Temporal Perception for Video Domain Generalization.	A novel model called the Spatial-Temporal Diversification Network (STDN) examines relationships over time as well as spatial elements within frames to identify a range of cues in movies.
Train Once, Get a Family: State-Adaptive Balances for Offline-to-Online Reinforcement Learning.	FamO2O is a framework that researchers have developed to improve the performance of existing offline-to-online reinforcement learning algorithms by figuring out how to best balance limitations and improvement depending on the state.
Promising or Elusive? Unsupervised Object Segmentation from Real-world Single Images.	This study explores the challenge of utilizing unsupervised models to segment items in real-world photographs.
Phi-2: The surprising power of small language models.	Azure's Phi 2 is the latest in a line of small language models that were mostly trained on synthetic data. The performance of 13B parameter models is matched by the 2.7B parameter model. The difficult part of this task is identifying and addressing "test set rephrasing," although the model is very effective in any scenario.
DiAD: A Diffusion-based Framework for Multi-class Anomaly Detection.	DiAD uses diffusion models to its advantage in order to find abnormalities. To precisely identify and pinpoint abnormalities in multi-class environments, it integrates a pixel-space autoencoder, a Semantic-Guided (SG) network, and a feature-space extractor in a novel way.
Learning Naturally Aggregated Appearance for Efficient 3D Editing.	A new technique called AGAP makes 3D editing easier. AGAP enables users to effortlessly update 3D scenes without having to re-optimize for each change by utilizing a 2D image known as a canonical image.
Sherpa3D: Boosting High-Fidelity Text-to-3D Generation via Coarse 3D Prior.	A novel framework called Sherpa3D enhances the production of 3D content via text prompts. By employing coarse 3D information to direct the creation process, it combines the advantages of 2D and 3D diffusion models. Thus, the constraints of current technologies are overcome and high-quality, diversified, and geometrically consistent 3D assets are produced.
Rethinking Compression: Reduced Order Modelling of Latent Features in Large Language Models.	This work presents a technique for reduced order modeling-based big language model compression that greatly minimizes memory and time requirements without requiring expensive hardware.
Towards a Generalized Multimodal Foundation Model.	With the help of FIND, AI models now have a flexible interface that improves their comprehension of datasets and visuals without changing the fundamental model.
HyperRouter: Towards Efficient Training and Inference of Sparse Mixture of Experts via HyperNetwork.	The HyperRouter technique dynamically modifies router characteristics to increase the effectiveness of training big language models.
FunSearch: Making new discoveries in mathematical sciences using Large Language Models.	By searching for “functions” written in computer code, FunSearch made the first discoveries in open problems in mathematical sciences using LLMs. scientific article.
Weak to Strong Generalization.	An analog to weak people aligning super intelligent models, this new discovery from the OpenAI super alignment team (with code) implies that you may utilize much weaker supervisor models to guide or align a considerably more powerful model. To regain much of the alignment performance of GPT-4, they employed GPT-2. Among the key differences between this approach and RLHF-like approaches is that the former provides a tractable route for significant improvement.
Chat-3D v2: Bridging 3D Scene and Large Language Models with Object Identifiers.	Object identifiers are included in Large Language Models by a novel research technique aimed at enhancing comprehension and providing answers about 3D situations. This approach, which focuses on finding and connecting items in a scene, has demonstrated encouraging outcomes in terms of improving AI's comprehension of intricate spatial connections.
SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention.	SwitchHead is a breakthrough in improving the effectiveness of AI models. Transformers' memory and processing requirements are decreased without sacrificing functionality.
Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models.	offers a method for self-training with feedback that can significantly lessen reliance on data created by humans; when model-generated data is used in conjunction with a reward function, LLM performance on problem-solving tasks is enhanced.

News

Link	description
French AI start-up Mistral secures €2bn valuation.	Eight-month-old group set to close roughly €400mn funding round as early as Friday, in new deal lead by Andreessen Horowitz. Meanwhile, it unveils its platform with new models, an embedding, and instruction-tuned models.
Google unveils AlphaCode 2, powered by Gemini.	Alongside its Gemini generative AI model, Google this morning took the wraps off of AlphaCode 2, an improved version of the code-generating AlphaCode
Introducing Stable LM Zephyr 3B.	Stable LM Zephyr 3B is a 3 billion parameter Large Language Model (LLM), 60% smaller than 7B models, allowing accurate, and responsive output on a variety of devices without requiring high-end hardware.
Better, Cheaper, Faster LLM Alignment with KTO.	a method called Kahneman-Tversky Optimization (KTO) that makes it easier and cheaper than ever before to align LLMs on your data without compromising performance.
Paving the way to efficient architectures: StripedHyena-7B.	StripedHyena builds on the many lessons learned in the past year on designing efficient sequence modeling architectures: H3, Hyena, HyenaDNA, and Monarch Mixer.
FTC looking into Microsoft's investment in OpenAI: Report.	Following UK regulators' probe, the Federal Trade Commission (FTC) is inspecting Microsoft's (MSFT) relationship with and investment in artificial intelligence developer OpenAI, according to Bloomberg.
Liquid AI, a new MIT spinoff, wants to build an entirely new type of AI.	An MIT spinoff co-founded by robotics luminary Daniela Rus aims to build general-purpose AI systems powered by a relatively new type of AI model called a liquid neural network.
Microsoft and Labor Unions Form ‘Historic’ Alliance on AI.	Microsoft Corp. is teaming up with labor unions to create “an open dialogue” on how artificial intelligence will impact workers.
Europe reaches a deal on the world’s first comprehensive AI rules.	European Union negotiators clinched a deal Friday on the world’s first comprehensive artificial intelligence rules, paving the way for legal oversight of AI technology that has promised to transform everyday life and spurred warnings of existential dangers to humanity.
OpenAI leaders warned of abusive behavior before Sam Altman’s ouster.	Sam Altman was briefly fired by OpenAI after a group of top leaders expressed concerns to the board about his claimed psychological abuse, which included inciting conflict amongst staff and causing turmoil. There were also allegations of dishonesty in Altman's board discussions. Threats of widespread resignations and resounding staff support led to Altman's restoration; yet, the incident has raised doubts about the company's future course.
MIT group releases white papers on governance of AI.	The series aims to help policymakers create better oversight of AI in society.
Google weighs Gemini AI project to tell people their life story using phone data, photos.	“Project Ellmann” is an internal Google proposal to use artificial intelligence to help users get a “bird’s-eye view” of their life stories. The idea would be to use LLMs like Gemini to ingest search results, spot patterns in a user’s photos, create a chatbot and “answer previously impossible questions” about a person’s life. The team also demonstrated “Ellmann Chat,” with the description “Imagine opening ChatGPT but it already knows everything about your life.”
a16z Open Source AI Grant recipients.	This program is designed to support a thriving open-source ecosystem around modern AI. We provide grant funding (not an investment) to developers and small teams who are building critical pieces of the open-source AI stack.
Google debuts Imagen 2 with text and logo generation.	Google’s making the second generation of Imagen, its AI model that can create and edit images given a text prompt, more widely available — at least to Google Cloud customers using Vertex AI who’ve been approved for access.
First Impressions with Google’s Gemini.	The Roboflow team has analyzed Gemini across a range of standard prompts that we have used to evaluate other LMMs, including GPT-4 with Vision, LLaVA, and CogVLM. Our goal is to better understand what Gemini can and cannot do well at the time of writing this piece.
OpenAI inks deal with Axel Springer on licensing news for model training.	OpenAI today announced that it’s reached an agreement with Axel Springer, the Berlin-based owner of publications including Business Insider and Politico, to train its generative AI models on the publisher’s content and add recent Axel Springer-published articles to OpenAI’s viral AI-powered chatbot ChatGPT.
OpenAI's Superalignment Fast Grants.	We’re launching $10M in grants to support technical research toward the alignment and safety of superhuman AI systems, including weak-to-strong generalization, interpretability, scalable oversight, and more.
A new old kind of R&D lab.	Respond AI is a new lab that seeks to find genuinely useful and productive applications for current models rather than creating new ones. Its goal is to do basic research to assist enterprises in enabling AI-enabled application cases.
Samsung unveils its generative AI model Samsung Gauss.	Samsung Gauss consists of language, code, and image models and will be applied to the company's various products in the future.

Resources

Link	description
Towards 100x Speedup: Full Stack Transformer Inference Optimization.	Deployment optimizations are getting more and more popular as open models prove beneficial for various enterprise needs. But the terrain is uneven and complicated. This article provides a good in-depth analysis of numerous common methods for accelerating language model serving.
Pearl - A Production-ready Reinforcement Learning AI Agent Library.	Pearl is a new production-ready Reinforcement Learning AI agent library open-sourced by the Applied Reinforcement Learning team at Meta.
Running Dolphin Locally with Ollama.	For llama cpp models, ollama functions similarly to package management. Its characteristics for quality-of-life usability make modeling simple, even while using a CPU. The two fantastic models, Samantha and Dolphin, which are good unfiltered models helpful for conversational activities, are demonstrated in this example.
LLM Visualization.	Welcome to the walkthrough of the GPT large language model! Here we'll explore the model nano-gpt, with a mere 85,000 parameters.
giskard.	The testing framework dedicated to ML models, from tabular to LLMs
BricksLLM: AI Gateway For Putting LLM In Production.	BricksLLM is a cloud-native AI gateway written in Go. Currently, it serves as a proxy for OpenAI. We let you create API keys that have rate limits, cost limits, and TTLs.
KwaiAgents.	KwaiAgents is a series of Agent-related works open-sourced by the KwaiKEG from Kuaishou Technology
Now add a walrus: Prompt engineering in DALL-E 3.	An experiment using DALL-E 3 that shows how various prompts produce a range of visuals and how additional prompts hone these images.
HuggingFace gets AMD support.	The new Mistral model, AMD compatibility, safe sensors by default, and more are included in Transformers 4.36.0!
AI Tamago.	A 100% local, LLM-generated, and driven virtual pet with thoughts, feelings, and feedback. Revive your fond memories of Tamagotchi!
Introducing gigaGPT: GPT-3 sized models in 565 lines of code.	GigaGPT is Cerebras’ implementation of Andrei Karpathy’s nanoGPT – the simplest and most compact code base to train and fine-tune GPT models.
Awesome CLIP in Medical Imaging.	This is a collection of awesome articles about CLIP in medical imaging
Roadmap To Learn Generative AI In 2024.	A repository with links and videos
Lookup Table meets Local Laplacian Filter: Pyramid Reconstruction Network for Tone Mapping.	LLF-LUT is an effective end-to-end framework for the HDR image tone mapping task performing global tone manipulation while preserving local edge details.
Obsidian: Worlds smallest multi-modal LLM. First multi-modal model in size 3B.	Obsidian-3B-V0.5 is a multi-modal AI model that has vision! Its smarts are built on Capybara-3B-V1.9 based on StableLM-3B-4e1t. Capybara-3B-V1.9 achieves state-of-the-art performance when compared to models of similar size, even beating some 7B models.
Mathematical Language Models: A Survey.	a survey on LLMs' development on mathematical activities; includes papers and resources on LLM research on tasks including theorem proving and math word problem solving, as well as prompting approaches.
A Survey of Large Language Models in Medicine: Principles, Applications, and Challenges.	a thorough analysis of more than 300 papers on LLMs in medicine; provides a synopsis of the concepts, uses, and difficulties faced by LLMs in the field.

Perspectives

Link	description
Excuse me, but the industries AI is disrupting are not lucrative.	Although the demo video for Google's Gemini AI model was remarkable, the model's real-time capabilities were questioned due to the use of pre-recorded material and edited responses, which resulted in a modest gain in Google's stock price. This caution is reflective of larger worries in the AI sector, as businesses set high standards but struggle to convert AI skills into meaningful economic gains. Current AI models are particularly good in domains that don't always generate large profits.
Interoperable Authentication Protocol.	Considering how quickly model capabilities are developing, coordinating communication between language models and users is essential. In order to address this, the Interoperable Authorization Protocol (IAP) establishes a consent management system and secure, flexible communication routes. In order to match AI operations with a variety of human values and objectives, this open-source methodology promotes cooperation within the AI community.
On Platform Shifts and AI.	While AI represents a technological shift, the discussion at the 2022 TCV Engage Summit focused on the fact that it requires a new distribution channel, which is critical for generating meaningful consumer prospects. Presently available AI breakthroughs are dependent on conventional channels of distribution, which benefits well-established businesses or creative start-ups; nevertheless, new channels of distribution may not materialize as expected.
The AI revolution in chemistry is not that far away.	Although the artificial intelligence (AI) revolution in chemistry has yet to happen, it is not that far off. The key question is what we can do to get there faster.
Can AI deliver advice that is judgment-free for science policy?	We acknowledge the potential of using artificial intelligence (AI) to inform science policy, but disagree with the suggestion that it can create judgment-free policy advice
How to make data open? Stop overlooking librarians.	Digital archivists are already experts at tackling the complex challenges of making research data open and accessible. We can help to smooth the transition.
Vertical AI.	The previous ten years of SaaS have been characterized by horizontal software, or software created to alleviate a user's pain regardless of the kind of user experiencing it. In contrast, vertical SaaS is intended for a specific user base, making it possible to customize the solution to meet their demands. According to a Greylock partner's prediction in this article, going the future, software development will be guided by consumers' expectations for customized services.
Why the AI Act was so hard to pass.	Over two years since it was first proposed, policymakers in Brussels were still debating core contents of the EU’s landmark AI regulations hours before reaching a deal.
Google’s Gemini Marketing Trick.	The world saw a jaw-dropping demo of Gemini this week. It just wasn’t the real deal.
Why Stability AI is launching a subscription fee.	Stability AI will charge commercial customers for the use of its most advanced models, pivoting away from being fully open source
The real research behind the wild rumors about OpenAI’s Q* project.	OpenAI hasn't said what Q* is but it has revealed plenty of clues.
Two Titans on the Future of AI (with Reid Hoffman & Vinod Khosla).	A double header from Cerebral Valley.

Back to index

ML news: Week 4 - 10 December

Research

Link	description
Diffusion Models Without Attention.	Modern diffusion models employ the attention mechanism in most cases, but not always. Recent advances in theory have accelerated the proliferation of interest in state spaces, leading to intriguing new applications.
MoMask: Generative Masked Modeling of 3D Human Motions.	A new initiative by the authors of seminal work in this field combines innovative encoder techniques to provide fine-grained control over the production of the final animation.
When StyleGAN Meets Stable Diffusion.	Enhancing identity preservation in produced pictures, a novel technique uses the enlarged StyleGAN embedding space W+ for text-to-image diffusion models.
MaXTron: Mask Transformer with Trajectory Attention for Video Panoptic Segmentation.	MaXTron is a simple yet effective unified meta-architecture for video segmentation, which enriches existing clip-level segmenters by introducing a within-clip tracking module and a cross-clip tracking module, thus achieving better temporally consistent segmentation results.
Mamba: Linear-Time Sequence Modeling with Selective State Spaces.	An additional article on state spaces that offers better performance and scalability. Here, they train a 3B parameter model, which beats out bigger 7B parameter Transformer models, by taking inspiration from the LSTM. official code.
MotionEditor: Editing Video Motion via Content-Aware Diffusion.	MotionEditor is a diffusion model that expertly strikes a balance between preserving the original material and manipulating motion in videos. With the introduction of a novel two-branch architecture with attention injection and a content-aware motion adaptor, modified movements may be seamlessly integrated while preserving the protagonist's look and the original background. official code.
Exploiting Diffusion Prior for Generalizable Pixel-Level Semantic Prediction.	Artificial intelligence-generated images now have more accurate semantic predictions because of a new technique called Diffusion Models as Prior (DMP). Even with minimal training data, this novel method outperforms existing approaches by shrewdly adapting pre-trained text-to-image models for a variety of tasks, such as semantic segmentation and 3D property estimation. official code.
IMMA: Immunizing text-to-image Models against Malicious Adaptation.	A novel way to prevent malicious adaptations of text-to-image models to produce harmful content is provided by the new IMMA approach. official code.
Language model self-teaching for domain adaptation.	You may either employ some retrieval strategies or fine-tune language models when attempting to apply them in areas that need knowledge of certain niches. Each has shortcomings. This innovative approach makes use of artificial data that is self-generated to improve knowledge throughout testing. In comparison to both fine-tuning and RAG, it demonstrates notable gains on typical adaption standards.
DiffiT: Diffusion Vision Transformers for Image Generation.	The Diffusion Vision Transformers (DiffiT) is a project that investigates the efficacy of vision transformers in diffusion-based generative learning. This model combines a new time-dependent self-attention module with a U-shaped encoder-decoder design.
Zero123++: A Single Image to Consistent Multi-view Diffusion Base Model.	This project introduces Zero123++, a model that applies diffusion concepts to produce consistent multi-view pictures from a single input image. Zero123++ tackles problems like as alignment and texture quality by using pre-trained 2D models.
Salient Object Detection in RGB-D Videos (RDVS dataset and DCTNet+ model).	The RDVS dataset, which contains a wide variety of RGB-D video sequences, and DCTNet+, a specialized network for RGB-D video object detection, which is outfitted with cutting-edge features for accurate prediction and enhanced performance over previous models, are the two main contributions revealed by this repository.
Style Aligned Image Generation via Shared Attention.	Amazing work by Google based on SDXL that distributes focus between generations to preserve unified looks. Importantly, this approach doesn't need to be adjusted.
Describing Differences in Image Sets with Natural Language.	This essay compares and contrasts two image collections using natural language. This is a brand-new, difficult challenge. The individual photos are captioned, rearranged, and summarized using a language model as part of the solution. official code.
BenchLMM: Benchmarking Cross-style Visual Capability of Large Multimodal Models.	In this work, BenchLMM, a benchmark for testing the resilience of large multimodal models such as GPT-4V and LLaVA against different image styles, is presented.
Let's Think Outside the Box !	This paper presents an approach to investigate Leap-of-Thought capabilities in multimodal LLMs using the Oogiri comedy-generating game. This method forces LLMs to use non-sequential thinking, which is an essential ability for coming up with original and amusing answers to a variety of multimodal material.
OneLLM: One Framework to Align All Modalities with Language.	OneLLM employs a universal encoder and a universal projection module to align multimodal inputs with LLM. It also utilizes modality tokens {modal} to switch between modalities.
Kandinsky 3.0.	We present Kandinsky 3.0, a large-scale text-to-image generation model based on latent diffusion, continuing the series of text-to-image Kandinsky models and reflecting our progress to achieve higher quality and realism of image generation. Compared to previous versions of Kandinsky 2.x, Kandinsky 3.0 leverages a two times larger UNet backbone, and a ten times larger text encoder and removes diffusion mapping. official code.

News

Link	description
OpenAI’s GPT store delayed to next year.	OpenAI’s GPT store will be delayed until next year, the company said in an email to people who signed up for its GPT Builder.
Amazon's Q generative AI chatbot allegedly leaks location of AWS data centers - report.	Amazon's newly launched artificial intelligence chatbot Amazon Q is “experiencing severe hallucinations and leaking confidential data,” internal documents warn.
Interview: Sam Altman on being fired and rehired by OpenAI.	“I totally get why people want an answer right now. But I also think it’s totally unreasonable to expect it.” In the meanwhile, OpenAI Agreed to Buy $51 Million of AI Chips From a Startup Backed by CEO Sam Altman and other details on the story emerge: New report illuminates why OpenAI board said Altman “was not consistently candid”.
Report: Stability AI Positioning Itself for Acquisition.	Stability AI, a British artificial intelligence (AI) startup, is reportedly considering selling the company amidst mounting pressure from investors over its financial position.
Report: Google delays Gemini launch from next week to January.	Google announced Gemini at I/O 2023 as its next-generation foundation model. According to a report today, Google was originally going to launch Gemini next week, but that has now been delayed until January.
The GPT to rule them all: Training for one trillion parameter model backed by Intel and US government has just begun.	LLM playfully dubbed 'ScienceGPT' is being trained from data from the Aurora supercomputer
Perplexity AI unveils ‘online’ LLMs that could dethrone Google Search.	Confusing AI can overthrow Google with its blend of current knowledge, a conversational AI chatbot interface, and a web index. Versions of the open-source models from Mistral and Meta that have been improved and enhanced have been made available by the firm. The models are meant to provide useful, accurate, and current information. These are the first-ever live LLM APIs with no knowledge cutoff, and they are based on online search data.
AI Alliance Launches as an International Community for Safe and Open AI .	More than 50 international organizations come together under the leadership of IBM and Meta to promote transparent, ethical AI development. Its main objectives are to advance hardware, establish AI standards, and advance AI knowledge and expertise. Major IT companies, academic organizations, and research centers are among the members. The Alliance places a strong emphasis on diversity, safety, and fair access to AI innovation.
Elon Musk's AI firm xAI files to raise up to $1 billion in equity offering.	Elon Musk's artificial intelligence startup xAI has filed with the U.S. securities regulator to raise up to $1 billion in an equity offering, according to a filing on Tuesday.
Google’s new AI experiment composes abstract musical clips inspired by instruments.	You may not hear the exact sound of the instrument you entered. you can try here.
Solve Intelligence helps attorneys draft patents for IP analysis and generation.	A platform that is AI-native and helps quickly create excellent patents is called Solve Intelligence. More than 25 IP firms worldwide have been employing them since their launch in July, and customers have reported 60–90% increases in efficiency. After completing Y Combinator, the startup just revealed the details of its $3 million seed financing.
Airbnb has acquired GamePlanner.AI.	Airbnb has purchased GamePlanner, an AI startup.AI, a business headed by Siamak Hodjat, an AI specialist, and Adam Cheyer, a co-founder of Siri. The group's main goals will be to integrate their technologies with Airbnb's platform and expedite certain AI projects. GamePlanner and Airbnb.AI are dedicated to leveraging AI to improve human-computer interaction.
Introducing Gemini: our largest and most capable AI model.	The field-wise network is a column-specific neural network to capture the information associated with the feature. The second component, on the other hand, is across-field network to choose the specialized operations for the dataset. Finally, the last step in the operation fusion network nonlinearly connects the chosen operations. In other words, it is an enhancement of DeepFM in which the network selects the best operations. Not all was good: Google’s best Gemini demo was faked.
Meta's AI image generator is available as a standalone website.	The company is testing dozens of new AI features in Facebook, Instagram and WhatsApp.
Microsoft’s Copilot is getting OpenAI’s latest models and a new code interpreter.	GPT-4 Turbo is on the way soon, alongside improvements to the DALL-E 3 model and deep search results for Bing.
Elon Musk told OpenAI to move faster right before he left the company in 2018: NYT.	Elon Musk said OpenAI needed to be quicker with its work before departing from the company, per NYT.
Purple Llama: Towards open trust and safety in the new world of generative AI.	Aiming to provide fair and equitable conditions for creating responsible and safe generative AI experiences, Purple Llama is a new initiative that is releasing models, assessments, and tools under permissive licenses for both commercial and research use. The Llama Guard model for identifying and thwarting cyberattacks, CyberSecEval for evaluating AI system security, and tools for insecure code detection and cyberattack compliance testing are among the initial products. By democratizing access to necessary resources, this project enables developers to design safe and ethical AI experiences.
X begins rolling out Grok, its ‘rebellious’ chatbot, to subscribers.	Grok, a ChatGPT competitor developed by xAI, Elon Musk’s AI startup, has officially launched on X, the site formerly known as Twitter.
Enabling next-generation AI workloads: Announcing TPU v5p and AI Hypercomputer.	Google unveiled Cloud TPU v5p, the company's most potent, adaptable, and scalable AI accelerator to yet. AI-powered products are trained and served using TPUs. Additionally, Google has revealed the AI Hypercomputer from Google Cloud, a supercomputer architecture that makes use of an integrated system of top-tier machine learning frameworks, open software, performance-optimized hardware, and adaptable consumption options. Systems-level co-design is used by AI Hypercomputer to increase productivity and efficiency in AI serving, tweaking, and training.
Long context prompting for Claude 2.1.	In reference, Anthropic's last release of Claude had 200k tokens. It only demonstrated 27% retrieval performance on certain common tasks, suggesting that it was afflicted by the "lost in the middle" issue that plagues language models in external evaluations. Retrieval performance rises to 98% when the prompt is modified to include the statement "Assistant: Here is the most relevant sentence in the context."

Resources

Link	description
weight-selection.	We introduce weight selection, a method for initializing models by selecting a subset of weights from a pre-trained larger model. With no extra cost, it is effective for improving the accuracy of a smaller model and reducing its training time needed to reach a certain accuracy level.
Nous-Hermes-2-Vision - Mistral 7B.	This vision model is a potent new open-source text and vision model that can operate on consumer hardware, and it is built on top of the best 7B language model with SigLIP integration. The incorporation of function calling is one of the neat ideas here. Due to a hallucination problem, the model remains in alpha.
LLM As A Function.	It's helpful to think about language models as functions with standard input and output when adding them to your code base. The author of React Native demonstrates a couple of methods for doing that in this blog post, along with the advantages of modeling your models in this manner.
aiconfig.	AIConfig saves prompts, models, and model parameters as source control-friendly configs. This allows you to iterate on prompts and model parameters separately from your application code.
Microsoft's Generative AI for Beginners.	A 12 Lesson course teaching everything you need to know to start building Generative AI applications
Unsloth.	Fast and memory-efficient LLM tuning.
Responsible Innovation Labs Launches First Pro-Innovation Responsible AI Protocol For Startups.	A voluntary industry-driven approach for responsible AI has been introduced by nonprofit Responsible Innovation Labs (RIL), especially for startups and their investors. The protocol is intended to make sure startups incorporate ethical AI practices as they grow, and it has the backing of 35 top venture capital funds. This action is in line with the Biden Administration's ethical AI goal.
mlx.	Thanks to unified memory, Apple has discreetly introduced a new Array framework that runs faster on Mac devices. It offers some GPU support in addition to being straightforward and tidy.
stable-fast.	An ultra-lightweight inference performance optimization framework for HuggingFace Diffusers on NVIDIA GPUs.
Introducing the OpenAI Switch Kit: Move from closed to open-source AI in minutes.	With just a few lines of code, you can make your project open source.
Optimizing LLMs for Real-World Applications.	Lightspeed provides information from TitanML and Google regarding the specifics of enhancing LLMs through fine-tuning or prompting.
Efficient SAM Example.	This script provides an example of how to get visualization results from EfficientSAM using ready-to-use torch script
CopilotKit.	Integrate AI-powered Textareas and in-app chatbots into React web applications.
Mamba-Chat.	Mamba-Chat is the first chat language model based on a state-space model architecture, not a transformer. The model is based on Albert Gu's and Tri Dao's work Mamba: Linear-Time Sequence Modeling with Selective State Spaces
mlx-llama.	People have already enabled Llama 2 models to operate on the new framework, just one day after Apple published the MLX framework. or if you prefer Mistral.
UIDraw.	Draw and build a website on your phone.
SEO GPT by Writesonic.	Boost your website’s SEO instantly right inside ChatGPT.

Perspectives

Link	description
The future of AI in software development.	Inbal Shani, the Chief Product Officer of GitHub, talks about the role AI plays in software development and makes the case that AI-driven code production will increase developer productivity rather than replace it. She delves into GitHub's Copilot's performance measures, philosophy, and innovation-promoting practices. The future of AI in the IT sector is clarified by this discussion.
OpenAI & Grand Strategy.	The importance of grand strategy in the IT industry is emphasized in this essay, which compares the lofty goals of contemporary tech executives to historical victories and exhorts them to behave and think like historical leaders in order to match capabilities with ambitions. It uses the recent happenings at OpenAI as an illustration of a grand strategy that works and challenges prospective leaders to develop plans that match their skills with their goals for meaningful, constructive change.
AI and Trust.	In the discussion, the speaker emphasizes the differences between social and interpersonal trust and warns about the potential for profit-driven organizations to take advantage of our inclination to view AI as friends rather than as a service. In order to guarantee that AI continues to be a reliable and advantageous service for society, they urge government intervention through transparency legislation and regulations targeted at the people who create AI. They also ask for the creation of public AI models.
AI Doomers are worse than wrong - they're incompetent.	This essay attacks the OpenAI AI doomer movement for making calculated mistakes and unnecessarily speeding up AI development instead of protecting it.
How AI Changes Workflows.	GitHub has acknowledged the impact of AI on developer workflows and is "re-founding" Copilot. With AI-assisted procedures like autocomplete code, it will boost productivity and might potentially completely reorganize operations. This change permits customized workflows, but it also necessitates striking a compromise between flexibility and the capacity to offer extensive client assistance.
Teach Llamas to Talk: Recent Progress in Instruction Tuning.	After instruction tweaking was implemented, the utility of language models significantly increased. Synthetic data pipelines are one of the numerous recent innovations that improve and streamline the process.
Two Titans on the Future of AI (with Reid Hoffman & Vinod Khosla).	An excellent synopsis of Reid Hoffman and Vinod Khosla's 45+ minute lectures, in which they each delved into a range of subjects from AI to manifestos for "techno-optimists." Both titans have insightful opinions about the future and the best ways to manage the tech sector.
Is AI leading to a reproducibility crisis in science?	Scientists worry that ill-informed use of artificial intelligence is driving a deluge of unreliable or useless research.
Generative AI could revolutionize health care — but not if control is ceded to big tech.	Large language models such as that used by ChatGPT could soon become essential tools for diagnosing and treating patients. To protect people’s privacy and safety, medical professionals, not commercial interests, must drive their development and deployment.
ChatGPT one year on: who is using it, how and why?.	In just a year, ChatGPT has permeated scientific research. Seven scientists reveal what they have learned about how the chatbot should — and shouldn’t — be used.
ML system design: 300 case studies to learn from.	How do companies like Netflix, Airbnb, and Doordash apply machine learning to improve their products and processes? We put together a database of 300 case studies from 80+ companies that share practical ML use cases and learnings from designing ML systems.

Back to index

ML news: Week 27 November - 3 December

Research

Link	description
SegVol: Universal and Interactive Volumetric Medical Image Segmentation.	Clinical analysis has entered a new era with the release of SegVol, a universal model for medical picture segmentation. SegVol is highly proficient at segmenting a wide range of anatomical categories, having been trained on a large set of CT images. official code.
Visual In-Context Prompting.	This novel strategy supports a variety of cues and environments, significantly improving performance in segmentation tasks and demonstrating outstanding outcomes in open-ended challenges. official code.
Starling-7B: Increasing LLM Helpfulness & Harmlessness with RLAIF.	Researchers at Berkeley used synthetic preference data to train a brand-new, cutting-edge 7B parameter model. This blog discusses the unique difficulties in training reward models (such as how an example's score might change depending on where it is in the list) and how they overcome them. Both the training reward model and the generated model are made public.
Segmentation-Based Parametric Painting.	A novel method has been devised by researchers to convert pictures into paintings that emulate human characteristics and aesthetics.
Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers.	A groundbreaking study explores the potential of shallow feed-forward networks to replace attention mechanisms in Transformer models. Shallow feed-forward networks can emulate the behavior of attention mechanisms effectively, with similar performances. This research opens new avenues in neural network design, potentially simplifying complex models.
MEDITRON-70B: Scaling Medical Pretraining for Large Language Models.	Large language models (LLMs) can potentially democratize access to medical knowledge. In this work, we improve access to large-scale medical LLMs by releasing MEDITRON: a suite of open-source LLMs with 7B and 70B parameters adapted to the medical domain.
DiffSLVA: Harnessing Diffusion Models for Sign Language Video Anonymization.	A novel technique for maintaining linguistic content in sign language films while maintaining anonymity is DiffSLVA. By eliminating the need for exact position prediction, this method, which makes use of pre-trained diffusion models and a dedicated module for face expressions, addresses earlier shortcomings.
Efficient Dataset Distillation via Minimax Diffusion.	Generative diffusion techniques have been used in a novel way to produce surrogate datasets that are much less computationally intensive and far more representative and varied.
Enhancing Perceptual Quality in Video Super-Resolution through Temporally-Consistent Detail Synthesis using Diffusion Models.	A new method for video super-resolution (VSR) called StableVSR uses a temporal conditioning module together with diffusion models to improve the quality of upscaled movies.
Animatable Gaussians: Learning Pose-dependent Gaussian Maps for High-fidelity Human Avatar Modeling.	This research suggests "Animatable Gaussians," a cutting-edge method that blends 3D Gaussian splatting with 2D CNNs to produce more realistic and intricate human avatars from films.
Illuminating protein space with a programmable generative model.	Evolution has produced a range of diverse proteins, and now a generative model called Chroma can expand that set by allowing the user to design new proteins and protein complexes with desired properties and functions.
Bridging the Gap: A Unified Video Comprehension Framework for Moment Retrieval and Highlight Detection.	UVCOM is a new framework that better handles the distinct requirements of Video Moment Retrieval (MR) and Highlight Detection (HD).
DeepSeek-LLM.	The Deepseek coder model demonstrated remarkable code synthesis capability. Its current chat LLM, with 67B parameters, works noticeably better than Llama 2's 70B.
llamafile.	llamafile lets you distribute and run LLMs with a single file. Introducing llamafile.
Millions of new materials discovered with deep learning.	AI tool GNoME finds 2.2 million new crystals, including 380,000 stable materials that could power future technologies
Seamless: Multilingual Expressive and Streaming Speech Translation.	An innovative technique that transforms automated voice recognition is called Seamless. Conversations feel more natural thanks to this sophisticated model, which not only translates across 76 languages but also maintains the speaker's distinct prosody and voice style.
Language-conditioned Detection Transformer.	A novel open-vocabulary detection system called DECOLA is presented by researchers. It performs exceptionally well at recognizing things outside of its training dataset. official code.
Diffusion-MU-Attack.	This project proposes an evaluation system to assess the reliability of safety-driven unlearning techniques in diffusion models through the use of adversarial prompts.
AI system self-organises to develop features of brains of complex organisms.	Cambridge scientists have shown that placing physical constraints on an artificially-intelligent system – in much the same way that the human brain has to develop and operate within physical and biological constraints – allows it to develop features of the brains of complex organisms in order to solve tasks.

News

Link	description
Anthropic slashes AI pricing amid rising competition.	For tokens created after the most recent version of Claude was released, Anthropic added a pricing reduction. Both closed and open model pressure are the source of this. In addition, the model now can digest up to 200k tokens, hallucinates half as often, and can search the web, more info here.
Gen AI for the Genome: LLM Predicts Characteristics of COVID Variants.	A new demo lets users explore visualizations of the genome-scale language model by Argonne National Laboratory, NVIDIA, and other collaborators.
Codegen raises new cash to automate software engineering tasks.	Codegen has successfully raised a significant amount of money for some truly incredible automated software development technology. It links GitHub PRs for automated engineering to Jira boards.
Kyutai is a French AI research lab with a $330 million budget that will make everything open source.	This new lab, called Kyutai, will be a privately funded nonprofit working on artificial general intelligence. It will work with PhD students, postdocs, and researchers on research papers and open-source projects.
Amazon and Salesforce Expand Partnership to Add New AI Capabilities.	Salesforce and AWS have extended their collaboration to facilitate customers' management of data on both platforms and their integration of generative AI technology into their workflows and apps.
Tesla starts releasing to employees FSD v12 – a critical update to self-driving effort.	Tesla has started releasing to employees its FSD v12 update, which is apparently critical to Tesla’s achieving its self-driving goal. The biggest difference with the update is how vehicle controls will be taken over by neural nets rather than being hard-coded by engineers.
ChatGPT with voice is now available to all free users.	Download the app on your phone and tap the headphones icon to start a conversation.
Announcing the MLCommons AlgoPerf Training Algorithms Benchmark Competition.	A recent competition called AlgoPerf attempts to optimize for wall clock time. This implies that you can earn real money if you can develop a technique that outperforms current settings (faster than ADAM, for example). Some of the biggest AI companies in the world today support this fascinating task.
Introducing SDXL Turbo: A Real-Time Text-to-Image Generation Model.	SDXL Turbo achieves state-of-the-art performance with a new distillation technology, enabling single-step image generation with unprecedented quality, reducing the required step count from 50 to just one. code and weights.
OpenAI unlikely to offer board seat to Microsoft, other investors - source.	ChatGPT owner OpenAI is not expected to offer Microsoft and other investors including Khosla Ventures and Thrive Capital seats on its new board, a person familiar with the matter told Reuters on Tuesday. OpenAI has a new initial board. OpenAI seems to have returned to normality, however Some OpenAI employees are still feeling uneasy and looking for other jobs despite Sam Altman's return, report says.
Sports Illustrated Published Articles by Fake, AI-Generated Writers.	Articles created by fictitious AI authors have been covertly published by Sports Illustrated.
Together AI raises $102M series A.	This investment, led by Kleiner Perkins, will support the expansion of the training and inference solutions, which have had tremendous uptake since Together's June debut.
Amazon Q.	A generative model was trained by Amazon to help AWS platform users. Additionally, it will be used to general business support. The model is a proprietary mechanism that responds to inquiries from users on different facets of Amazon's backend infrastructure.
Voltage Park launches massive new cloud for AI development.	With a 24k H100 mega cluster, Voltage Park is a new cloud service that powers processes for clients like Character AI. With less than $2 per hour for each GPU, it appears to have an industry-leading price.
These ex-Apple employees are bringing AI to the desktop.	After selling Workflow to Apple in 2017, the co-founders are back with a new startup that wants to reimagine how desktop computers work using generative AI.
Martian’s tool automatically switches between LLMs to reduce costs.	With $9 million in funding, AI experts Etan Ginsberg and Shriyash Upadhyay founded the startup Martian. Martian developed a "model router" tool that intelligently routes requests to the most appropriate model for the job, maximizing the utilization of huge language AI models like GPT-4 and reducing costs. According to the founders, this strategy encourages fundamental research as opposed to competitive research.

Resources

Link	description
ziplora-pytorch.	Low-rank learning matrices, or LoRAs, alter model behavior at a lower cost than traditional fine-tuning. This paper suggests a practical method for combining LoRAs while preserving their unique information. original paper
DuckTrack: Accurate Computer Activity Tracking.	It can be a little difficult to extract image, audio, and keystroke data from your computer. This library's goal is to train digital agents by simplifying that procedure.
direct-preference-optimization.	Using extremely identical data, direct preference optimization is a reliable substitute for RLHF. An implementation of the approach can be studied in this repository to gain knowledge about it.
Kandinsky Video — a new text-to-video generation model.	This paper presents a new two-stage latent diffusion text-to-video generation architecture based on the text-to-image diffusion model. code.
SD-T2I-360PanoImage.	A novel circular blending technique has been developed by researchers to solve the enduring problem of producing smooth 360-degree panoramic pictures. Their novel methods for creating panoramic panoramas from text and individual photos heavily rely on this methodology.
insanely-fast-whisper.	Transcribe 150 minutes (2.5 hours) of audio in less than 98 seconds.
Agency: The Go Way to AI.	Library designed for developers eager to explore the potential of Large Language Models (LLMs) and other generative AI through a clean, effective, and Go-idiomatic approach.
CoachLM.	CoachLM presents a cutting-edge AI method for improving training datasets for LLMs. This approach dramatically increases the efficacy of instruction-following in LLMs by improving datasets in a novel way—by modifying rather than eliminating low-quality samples.
multimodal-maestro.	Effective prompting for Large Multimodal Models like GPT-4 Vision or LLaVA.
Tanuki.	Easily build LLM-powered apps that get cheaper and faster over time.
Accelerating Generative AI with PyTorch II: GPT, Fast.	The PyTorch team describes in this blog post how to significantly accelerate language model inference using native Pytorch code. The article explains how to obtain more than 200 tokens from Llama 2 7B every second.
Qwen-Audio.	An audio understanding model that is universal has been released by the Alibaba Cloud Group. It is capable of a variety of audio-related tasks, such as text question answering and music comprehension and speaker recognition.
3D to Photo.	3D to Photo is an open-source package by Dabble, that combines threeJS and Stable diffusion to build a virtual photo studio for product photography. Load a 3D model into the browser and virtual shoot it in any kind of scene you can imagine. The app currently uses Stable Diffusion 1.5-inpainting, hosted on Replicate.
5 Ways To Leverage AI in Tech with Freshworks CIO Prasad Ramakrishnan.	Prasad Ramakrishnan, CIO of Freshworks, goes over a few practical applications of AI for startups. This article outlines five ways that businesses may utilize AI to grow and address challenges, from improving user experience to onboarding and optimizing data platforms.

Perspectives

Link	description
The Q* hypothesis: Tree-of-thoughts reasoning, process reward models, and supercharging synthetic data.	According to a recent leak, an internal breakthrough combining search and reinforcement learning was the reason behind the OpenAI leadership scandal. This article presents one notion that sheds light on what this new approach could be genuinely doing.
Inside OpenAI, a rift between billionaires and altruistic researchers unravelled over the future of artificial intelligence.	a comprehensive overview of the OpenAI disaster. Generations to come will study this. This synopsis, which covers all of the intriguing facets and consequences, is essential reading for anyone who hasn't followed the full story.
What is OpenAI, Really?.	It’s been five incredibly turbulent days at the leading AI tech company, with the exit and then return of CEO Sam Altman. As we dig into what went wrong, an even bigger question looms: what is OpenAI?
Why I Just Resigned From My Job In Generative AI.	Stability AI's Audio team leader left because of differences in the company's position on using copyrighted material to train generative AI models. The proponent of generative AI feels that it is not fair use to train models on copyrighted material without permission, as this could put them in competition with original creations.
Exploring the Growing Convergence Between Blockchain and AI.	This research, which surveyed more than 600 IT professionals worldwide, delves into the main causes of the increasing demand for AI, the most well-liked applications, and the main obstacles preventing its wider deployment. Interestingly, it refutes the idea that blockchain and artificial intelligence are incompatible with technology.
Reshaping the tree: rebuilding organizations for AI.	By automating tasks and decision-making, the integration of AI in enterprises is altering conventional work processes and empowering teams to operate more productively. Organizations must adjust as AI develops quickly by promoting team experimentation with the technology, getting ready for new developments, and moving quickly to maintain their competitive edge.
God Help Us, Let's Try To Understand AI Monosemanticity.	By replicating a larger AI within a smaller one, Anthropic researchers have devised a mechanism to comprehend the intricate workings of AI. What they have discovered is that AI neural networks are capable of encoding information in a sophisticated way, much like physics' superposition. Through the application of this method to a basic AI, they found that it could represent discrete concepts, such as "God," in discrete features. They further conjecture that the same methodology may yield deeper insights into artificial and biological neural systems, which could result in safer and more effective AI development.
A Data-Driven Look at the Rise of AI.	This article from the Cerebral Valley AI Summit examines the development of AI using data and is heavily illustrated with slides. We all hear it all the time, but there is a ton of evidence to support it. The development of developer interest and its eventual collapse are of particular importance.
How Much Does it Cost to Use an LLM?.	Different models cost different amounts. Also, the size of the context window is an important factor. But how much?
The 10-Year “Overnight” Success Story of Casetext.	When Casetext first launched in 2013, it was a crowdsourced legal library, similar to "Wikipedia meets Reddit" for legal matters. A decade later, Casetext stands as one of the greatest AI achievements to date, able to compress weeks' worth of laborious legal work into hours or minutes. It was purchased for $650 million just a few months ago. Between those two points, what transpired?
ChatGPT generates fake data set to support scientific hypothesis.	Researchers say that the model behind the chatbot fabricated a convincing bogus database, but a forensic examination shows it doesn’t pass for authentic.
What the OpenAI drama means for AI progress — and safety.	A debacle at the company that built ChatGPT highlights concern that commercial forces are acting against the responsible development of artificial-intelligence systems.
Adjust the format of papers to improve description by AI.	The chatbot ChatGPT and other tools based on large language models (LLMs) can make scientific research more efficient, but they can also introduce mistakes when they describe scientific work. I suggest that small changes to the format of scientific papers could improve the training of LLMs.
AI under the microscope: the algorithms powering the search for cells.	Deep learning is driving the rapid evolution of algorithms that can automatically find and trace cells in a wide range of microscopy experiments.
ChatGPT's training data can be exposed via a "divergence attack".	Large language models, like ChatGPT, have been shown to be able to memorize and unintentionally disclose particular training data, raising privacy concerns—especially with bigger models.
In The Age Of AI, Google Experiments With Bold Changes To Search.	The excitement surrounding Q*, the purported AI breakthrough from OpenAI, illustrates the tech community's propensity to quickly change focus and conjecture about the next great development in AI, frequently with no information—a phenomenon known as "Shiny Object Syndrome."
A global hit: AI translation tools help singers break down borders.	While some producers view the cost as prohibitive, YouTube, Mr. Beast, and a South Korean label are among the companies using AI to dub video content into many languages.
ChatGPT is winning the future — but what future is that?.	OpenAI didn’t mean to kickstart a generational shift in the technology industry. But it did. Now all we have to decide is where to go from here.

Back to index

ML news: Week 20-26 November

Research

Link	description
MILA: Memory-Based Instance-Level Adaptation for Cross-Domain Object Detection.	Cross-domain object detection is challenging, and it involves aligning labeled source and unlabeled target domains. we propose a memory-based instance-level domain adaptation framework. Our method aligns a target instance with the most similar source instance of the same category retrieved from memory storage. official code.
TopoMLP: An Simple yet Strong Pipeline for Driving Topology Reasoning.	TopoMLP is a system that detects and analyzes traffic features and road centerlines to comprehend road scenes and identify drivable courses for self-driving automobiles. official code.
Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model.	In this study, several data optimization strategies that need less computational overhead to enable knowledge transfer across models are examined.
StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models.	StyleTTS 2 is a text-to-speech model that combines huge speech language models with adversarial training and style diffusion to produce human-level voice synthesis. official code.
Orca 2: Teaching Small Language Models How to Reason.	A few months ago, we introduced Orca, a 13-billion language model that demonstrated strong reasoning abilities by imitating the step-by-step reasoning traces of more capable LLMs.Orca 2 significantly surpasses models of similar size (including the original Orca model) and attains performance levels similar to or better than models 5-10 times larger, as assessed on complex tasks that test advanced reasoning abilities in zero-shot settings.
Proving Test Set Contamination in Black Box Language Models.	a thorough examination of the data that was utilized to train language models. Its findings imply that a large number of closed-source models most likely did not train on widely used benchmarks.
Amazon Reportedly Training AI With Twice As Many Parameters As GPT-4 .	The model will have a whopping 2 trillion parameters, which are the variables that determine the output of a given model, making it one of the largest currently in development.

News

Link	description
Discord is shutting down its AI chatbot Clyde.	Discord users won’t be able to chat to Clyde from December 1st onwards.
OpenAI has put ChatGPT Plus sign-ups on pause.	After announcing premium-tier users can build their own chatbots, CEO Sam Altman says its Plus subscription has exceeded capacity
OpenAI Staff Threatens Exodus, Jeopardizing Company’s Future.	A board member who was part of Sam Altman’s ouster as chief executive joined a majority of the company’s staff in calling for the decision’s reversal.
Sam Altman is still trying to return as OpenAI CEO.	Altman’s move to Microsoft isn’t a done deal, and Ilya Sutskever’s flip to supporting Altman means two board members need to change their minds.
Salesforce looks to poach outbound OpenAI staff with "full cash" compensation offer.	OpenAI researchers leaving the firm in protest could be offered a lifeline at Salesforce
Amazon’s offering free courses on generative AI.	From the company that brought you AWS certification comes a new ‘AI Ready’ education track to help train aspiring professionals on Amazon’s AI tech.
Eye On AI: Bain Capital Ventures Launches BCV Labs In Search Of New AI Deals.	BCV Labs is a new AI incubator and technical community founded by Bain Capital Ventures that provides money, office space, events, GPU credits, fellowship program, and recruiting help.
Microsoft rebrands its AI-powered Bing Chat as Copilot.	The company has also announced more Copilot AI features for its 365 apps.
Sam Altman to return as CEO of OpenAI.	After an attempted coup by OpenAI’s board that lasted five days, Altman is returning alongside co-founder Greg Brockman.
Microsoft and Nvidia are making it easier to run AI models on Windows.	Microsoft’s new Windows AI Studio lets developers access and configure AI models, such as Microsoft’s Phi, Meta’s Llama 2, and Mistral.
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding.	Automating autoregressive language model inference may be done in a variety of ways. One method that has generated excitement is the use of draft models. Although it may take longer, this needs two models. On the other hand, you may reduce the requirement for a draft model and accelerate creation linearly by producing related ngrams from the same model.
OpenAI drops a big new ChatGPT feature with a joke about its CEO drama.	ChatGPT’s voice feature lets you ask it a question by saying it aloud — and now it’s available for free.
Emmett Shear threatening to leave OpenAI if board can’t prove Sam Altman’s wrongdoing.	Former Twitch CEO Emmett Shear took a role at OpenAI following the ousting of Sam Altman but is reportedly threatening to leave unless the board can show evidence of Altman’s wrongdoing.
Artificial intelligence finds ways to develop new drugs.	A new AI model developed by chemists at ETH Zurich can not only predict where a pharmaceutically active molecule can be chemically modified but also how best to do it. This makes it possible to identify new pharmaceutical ingredients more quickly and improve existing ones in a targeted manner.
OpenAI researchers warned board of AI breakthrough ahead of CEO ouster, sources say.	Ahead of OpenAI CEO Sam Altman’s four days in exile, several staff researchers wrote a letter to the board of directors warning of a powerful artificial intelligence discovery that they said could threaten humanity

Resources

Link	description
Neural-Cherche.	Neural-Cherche is a library designed to fine-tune neural search models such as Splade, ColBERT, and SparseEmbed on a specific dataset.
The Data Engineering Handbook.	This repo has all the resources you need to become an amazing data engineer.
tensorli.	Absolute minimalistic implementation of a GPT-like transformer using only numpy (<650 lines).
THE RISE OF “WET” ARTIFICIAL INTELLIGENCE.	Combining AI with traditional wet lab work creates a virtuous circle from lab to data and back to the lab.
Video-LLaVA.	Video-LLaVA exhibits remarkable interactive capabilities between images and videos, despite the absence of image-video pairs in the dataset. It achieves state-of-the-art performance in video summarization and captioning.
make-real-starter.	Recently, tldraw released a popular tool that lets people quickly design software using a paint-like interface. GPT-V is then used to write code for the design's online version. It produces reliable and functional code and operates remarkably well. It also accepts commands in plain language.
AI Exploits.	A collection of real-world AI/ML exploits for responsibly disclosed vulnerabilities
Collaborative Word-based Pre-trained Item Representation for Transferable Recommendation.	The recently proposed CoWPiRec method enhances recommender systems using text-based item representations combined with collaborative filtering information. Using word graphs for item interactions, this novel approach has demonstrated better performance in a range of recommendation circumstances, including solving the cold-start issue.
RustGPT.	A web ChatGPT clone entirely crafted using Rust and HTMX.
Stable Video Diffusion Image-to-Video Model Card.	Stable Video Diffusion (SVD) Image-to-Video is a diffusion model that takes in a still image as a conditioning frame, and generates a video from it.
LangChain for Go.	Building applications with LLMs through composability, with Go
Reinforcement Learning for Generative AI: A Survey.	Comprehensive review across various application areas like NLP, computer vision, and more exciting and emerging domains. Insights into RL's flexibility in introducing new training approaches.Future directions for the evolution of generative AI.

Perspectives

Link	description
OpenAI’s identity crisis and the battle for AI’s future.	Last weekend some news happened in OpenAI, this blog post is about discussing some open questions.
A Data-Driven Look at the Rise of AI.	2023, The AI Revolution: Coatue's Sri Viswanath breaks down this year's developments in AI.
AI: The Coming Revolution.	Coatue highlight four points for the future: AI has potential to break through the hype and meaningfully improve our world. Open source is the heartbeat of AI, but not all open source is created equally. Builders and investors need to understand the new, AI-centric tech stack. The best of AI is yet to come
OpenAI’s Misalignment and Microsoft’s Gain.	After co-founders Sam Altman and Greg Brockman resigned from OpenAI due to internal issues and the company's failing non-profit strategy, Microsoft acquired key staff and intellectual property from OpenAI, significantly changing the AI field.
AGI's Impact on Tech, SaaS Valuations.	Thought experiments on how AGI affects SaaS companies of all shapes and sizes
Oops! We Automated Bullshit.	ChatGPT is a bullshit generator. To understand AI, we should think harder about bullshit
Explaining the SDXL latent space.	Using a smaller latent space for diffusion was one of the advances of the original Stable Diffusion model. This indicates that the diffusion occurs on a compressed image representation rather than on pixels. This article explores many interpretations of that space for SDXL.
Sudden Disturbances in Rapidly Moving Objects : The Implications of the OpenAI Fiasco.	The unexpected threat to OpenAI's dominating position in the developer ecosystem creates a chance for smaller businesses to step in and take advantage of a fresh opening. Microsoft will probably emerge victorious in the AI race, but it's possible that Anthropic and other model-layer businesses may capitalize on the disruption.
AI should focus on equity in pandemic preparedness.	Over-reliance on AI could inadvertently prioritize certain viruses or populations, leading to inequities in vaccine and disease research.
How AI is expanding art history.	From identifying disputed artworks to reconstructing lost masterpieces, artificial intelligence is enriching how we interpret our cultural heritage.
How AI shapes the life sciences: an interview with Oliver Stegle.	Oliver Stegle explains how AI-based tools have the potential to transform our ability to better understand the complexity of life and how these tools will shape the future of scientific exploration

Back to index

ML news: Week 12-19 November

Research

Link	description
3DStyle-Diffusion: Pursuing Fine-grained Text-driven 3D Stylization with 2D Diffusion Models.	In order to provide more control over appearance and geometry, this research integrates 2D diffusion models into the 3DStyle-Diffusion model, a revolutionary technique for comprehensive stylization of 3D meshes. It functions by first employing implicit MLP networks to parameterize the texture of a 3D mesh into reflectance and illumination. After that, a pre-trained 2D diffusion model is used to maintain geometric consistency and match the produced pictures with the text prompt. official code.
Cross-modal Prompts: Adapting Large Pre-trained Models for Audio-Visual Downstream Task.	Dual-Guided Spatial-Channel-Temporal (DG-SCT) attention mechanism to enhances pre-trained audio-visual models for multi-modal tasks.
Generalized Biomolecular Modeling and Design with RoseTTAFold All-Atom.	RoseTTAFold All-Atom (RFAA), a deep network addressing the limitations of current protein structure modeling tools by accurately representing complete biological assemblies, including covalent modifications and interactions with small molecules. RFAA demonstrates comparable accuracy to AlphaFold2 in protein structure prediction, excels in flexible small molecule docking, and predicts covalent modifications and assemblies involving nucleic acids and small molecules. Additionally, the authors present RFdiffusion All-Atom (RFdiffusionAA), a fine-tuned model for generating binding pockets around small and non-protein molecules, showcasing experimental validation with proteins binding to therapeutic, enzymatic, and optically active molecules.
FinGPT: Large Generative Models for a Small Language.	This study tackles the challenges of creating large language models (LLMs) for Finnish, a language spoken by less than 0.1% of the world population.
Watermarking Vision-Language Pre-trained Models for Multi-modal Embedding as a Service.	VLPMarker, a secure and robust backdoor-based embedding watermarking method for vision-language pre-trained models (VLPs), which effectively injects triggers into VLPs without interfering with model parameters, providing high-quality copyright verification and minimal impact on performance, while also enhancing resilience against various attacks through a collaborative copyright verification strategy based on both backdoor triggers and embedding distribution.
Visualizing the Diversity of Representations Learned by Bayesian Neural Networks.	ExplainableAI methods and their applications to Bayesian Neural Networks
MonoDiffusion: Self-Supervised Monocular Depth Estimation Using Diffusion Model.	In this work, a novel framework for self-supervised monocular depth estimation called MonoDiffusion is presented. It takes a fresh approach to the problem by treating iterative denoising. Instead of employing real depth ground-truth for training, it makes use of a faux ground-truth diffusion process led by a teacher model that has already been taught. official code.
Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answering.	The paper discusses the deployment challenges of large language models (LLMs) in real-world scenarios, particularly in domain-specific question answering (QA) with the integration of domain knowledge graphs. The authors introduce KnowPAT, a novel pipeline that employs style and knowledge preference sets, coupled with a new alignment objective, to improve LLMs for practical use in domain-specific QA, as evidenced by superior performance in experiments against 15 baseline methods. official code.
DeepMind AI accurately forecasts weather — on a desktop computer.	The machine-learning model takes less than a minute to predict future weather worldwide more precisely than other approaches. original article
Role play with large language models.	Casting dialogue-agent behaviour in terms of role play allows us to draw on familiar folk psychological terms, without ascribing human characteristics to language models that they in fact lack.
Fine-tuning Language Models for Factuality.	ChatGPT's widespread acceptance was made possible by a breakthrough in model optimization based on preferences. By using comparable technologies, model accuracy and factual accuracy can be increased, leading to a 50% reduction in medical recall errors.
Ultra-Efficient On-Device Object Detection on AI-Integrated Smart Glasses with TinyissimoYOLO.	This group trained an ultra-small YOLO computer vision model and developed new RISC-V hardware specifically for vision, allowing for real-time object identification at very low latency and low power consumption.
SentAlign: Accurate and Scalable Sentence Alignment.	an accurate sentence alignment tool designed to handle very large parallel document pairs. It can efficiently handle tens of thousands of sentences. official code.
Large Language Models are Temporal and Causal Reasoners for Video Question Answering.	LLMs make errors in VQA when they focus too much on the language and ignore the video content, this article aims to solve this official code.

News

Link	description
Google in talks to invest ‘hundreds of millions’ into AI startup Character.AI.	Character.AI's chatbots, with various roles and tones to choose from, have appealed to users ages 18 to 24, who contributed about 60% of its website traffic.
Introducing AI to FigJam.	FigJam, Figma's digital whiteboard application, now incorporates AI support to help streamline and improve design interactions.
LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills.	An open-source approach that integrates language and vision is called LLaVa. The updated version gives the instruction-tuned model access to tools for creating and altering images, among other things.
ai-town-rwkv-proxy.	Hundreds of agents in AI Town, an incredible experiment, go about their everyday lives as prompt states in language models. Compared to typical Transformers, the RWKV model is a linear language model that uses less resources. This repository runs AI Town on your local computer using this less expensive model.
Nvidia is launching a new must-have AI chip — as customers still scramble for its last one.	The new class-leading H200 has more memory capacity and bandwidth, speeding up its work with generative AI and LLMs.
OpenAI reveals new details about its AI development roadmap and fundraising plans.	OpenAI LP is working on GPT-5 and plans to raise more capital from Microsoft Corp. to support its development efforts, Chief Executive Officer Sam Altman has disclosed in a new interview.
Xbox partners with Inworld AI to build generative AI tools for game development.	Xbox and Inworld AI are working together to build AI-driven technologies that will enhance game developers' narratives and character creation features. As part of the collaboration, an AI character runtime engine and an AI design copilot will be created to help game creators create immersive gaming experiences. They believe these technologies will accelerate game creation, improve immersion, and encourage boundless innovation.
New techniques efficiently accelerate sparse tensors for massive AI models.	Complimentary approaches — “HighLight” and “Tailors and Swiftiles” — could boost the performance of demanding machine-learning tasks.
OpenAI’s six-member board will decide ‘when we’ve attained AGI’	According to OpenAI, the six members of its nonprofit board of directors will determine when the company has “attained AGI”
Giant AI Platform Introduces ‘Bounties’ for Deepfakes of Real People.	Users of the contentious "bounties" function of Civitai, an AI model sharing site, may now commission and profit from the production of AI-generated photographs.
You.com launches new APIs to connect LLMs to the web.	When OpenAI connected ChatGPT to the internet, it supercharged the AI chatbot’s capabilities. Now, the search engine You.com wants to do the same for every large language model (LLM) out there.
Microsoft and OpenAI partnership unveils new AI opportunities.	Microsoft said at OpenAI's DevDay that it will launch the new GPT-4 Turbo on Azure OpenAI Service before year's end, offering more control and cost savings. Businesses' AI skills will be enhanced by OpenAI's Custom Models initiative, which will easily interact with Microsoft's ecosystem.
Nous-Capybara-34B V1.9.	This is trained on the Yi-34B model with 200K context length, for 3 epochs on the Capybara dataset (multi-turn data with more than 1000 tokens per conversation)
AI writes summaries of preprints in bioRxiv trial.	Large language model creates synopses of papers aimed at various reading levels to help scientists sift through the literature.
Catch me if you can! How to beat GPT-4 with a 13B model.	Announcing Llama-rephraser: 13B models reaching GPT-4 performance in major benchmark. What's the trick behind it? Well, rephrasing the test set is all you need!
IBM debuts $500 million enterprise AI venture fund	IBM is dedicating $500 million to invest in generative AI startups focused on business customers.
Microsoft is finally making custom chips — and they’re all about AI.	The Azure Maia 100 and Cobalt 100 chips are the first two custom silicon chips designed by Microsoft for its cloud infrastructure
Google's AI-powered search feature goes global with a 120-country expansion.	The SGE update includes additional language support for Spanish, Portuguese, Korean and Indonesian.
Universe 2023: Copilot transforms GitHub into the AI-powered developer platform.	GitHub is announcing general availability of GitHub Copilot Chat and previews of the new GitHub Copilot Enterprise offering, new AI-powered security features, and the GitHub Copilot Partner Program.
Deepmind’s animation gallery.	A variety of animations and artwork have been made available by Google's deepmind research department to help people comprehend various AI systems. The animations are visually stunning but also a little strange.
Deep mind announce music generation model.	Today, in partnership with YouTube, we’re announcing Google DeepMind’s Lyria, our most advanced AI music generation model to date. Any content published by our Lyria model will be watermarked with SynthID.
META introduces Emu Video and Emu Edit, our latest generative AI research milestones.	A generative model frequently produces an output image that isn't exactly what you were hoping for. It is really difficult to alter that image using the same model, though. Meta made a crucial discovery: editing capabilities can arise when all generations are treated as instructions. This is a really good improvement, especially when combined with the model architecture's newfound simplicity.
Microsoft launches a deepfakes creator at Ignite 2023 event.	One of the more unexpected products to launch out of the Microsoft Ignite 2023 event is a tool that can create a photorealistic avatar of a person and animate that avatar saying things that the person didn’t necessarily say.
YouTube will show labels on videos that use AI	YouTube is now requiring creators to mark videos that are made using AI, and the platform will show labels to viewers.
Sam Altman fired as CEO of OpenAI.	In a sudden move, Altman is leaving after the company’s board determined that he ‘was not consistently candid in his communications.’ President and co-founder Greg Brockman has also quit. apparently, they asked him to come back but he is now hired by Microsoft.
Google delays launch of AI model Gemini, a potential rival to OpenAI's GPT-4.	Google is delaying the launch of its new large language model called Gemini, a potential rival to AI models from Microsoft (MSFT)-backed OpenAI
The Escalating AI Arm Race: Inside the High-Stakes Talent Wars with OpenAI and Google.	OpenAI recruiters are pitching annual compensation packages around $5-10 million for senior researchers who jump ship depending on their role and expertise.
Meta disbanded its Responsible AI team.	A new report says Meta’s Responsible AI team is now working on other AI teams.

Resources

Link	description
The Alignment Handbook.	The HuggingFace's Alignment Handbook aims to fill that gap by providing the community with a series of robust training recipes that span the whole pipeline.
versatile_audio_super_resolution.	Pass your audio in, AudioSR will make it high fidelity!
tarsier.	Vision utilities for web interaction agents. A number of teams are working on creating agents that can interact with web items through vision thanks to the development of potent new vision models. A standard toolset is introduced by Tarsier (e.g., element tagging). Any vision system will work to help you navigate the website and take action. It also has browsing facilities for language models without eyesight.
Extra-fast Bark for generating long texts.	In this notebook, we'll show you how to generate very long texts very quickly using Bark, Flash Attention 2 and batching.
OpenGPTs.	This is an open source effort to create a similar experience to OpenAI's GPTs. It builds upon LangChain, LangServe and LangSmith.
Tamil-Llama: A Family of LLaMA-based LLMs focused on Tamil Language.	This repository contains the code and models for "Tamil-Llama", a project focused on enhancing the performance of language models for the Tamil language.
GPT4V-AD-Exploration.	In our report, we explore the revolutionary GPT-4V, a visionary in the field of autonomous driving.
BestGPTs.	Top ranked OpenAI GPTs
Hallucination Leaderboard.	This evaluates how often an LLM introduces hallucinations when summarizing a document.
draw-a-ui	This is an app that uses tldraw and the gpt-4-vision api to generate html based on a wireframe you draw.
AMBER: An Automated Multi-dimensional Benchmark for Multi-modal Hallucination Evaluation.	a new benchmark designed to assess and reduce hallucinations in Multi-modal Large Language Models (MLLMs)
instructor	Structured extraction in Python, powered by OpenAI's function calling api, designed for simplicity, transparency, and control.
GPU-Accelerated LLM on a $100 Orange Pi.	This post shows GPU-accelerated LLM running smoothly on an embedded device at a reasonable speed. Additionally, we are able to run a Llama-2 13b model at 1.5 tok/sec on a 16GB version of the Orange Pi 5+ under $150.
LLM Sherpa.	LLM Sherpa provides strategic APIs to accelerate large language model (LLM) use cases.
The Developer's Guide to Production-Grade LLM Apps.	dvanced Techniques for Maximizing LLM Performance
Accelerating Generative AI with PyTorch: Segment Anything, Fast.	This blog shows how to get META SAM 8x faster, just using PyTorch features: quantization, nested tensors and Triton
ai-exploits.	This repository, ai-exploits, is a collection of exploits and scanning templates for responsibly disclosed vulnerabilities affecting machine learning tools.
Music ControlNet.	ControlNet represented an innovative approach to providing image synthetics models with fine-grained control. There is now a model for music generation that is fairly similar and allows you to manage several aspects such as pitch and pronunciation.
GPT-4 Turbo Note Taker.	Fast and simple, Tactiq’s AI Note Taker with GPT-4 Turbo lets you turn your meetings into actionable notes - so that you're always taking the right action and getting more out of your meetings.
Chroma.	Chroma is a generative model for designing proteins programmatically.
A Survey on Language Models for Code.	gives a summary of LLMs for code, covering 500 relevant works, more than 30 assessment tasks, and more than 50 models.

Perspectives

Link	description
Adversarial Attacks on LLMs.	This blog post discusses the many new assaults that language model systems are facing. It has good details regarding several attack types as well as some successful mitigations that teams have discovered.
AI and Open Source in 2023.	A comprehensive review of the major developments in the AI research, industry, and open-source space that happened in 2023.
How investors see your start up?	A general partner at Angular Ventures divides the application concepts we are seeing into three major categories in an attempt to make sense of all the nascent AI firms. This exclusively examines application-layer businesses; it ignores model-layer companies.
retool's state of AI 2023.	Retool surveyed 1,500 tech workers
Language models can use steganography to hide their reasoning, study finds.	large language models (LLMs) can master “encoded reasoning,” a form of steganography. This intriguing phenomenon allows LLMs to subtly embed intermediate reasoning steps within their generated text in a way that is undecipherable to human readers.
Why teachers should explore ChatGPT’s potential — despite the risks.	Many students now use AI chatbots to help with their assignments. Educators need to study how to include these tools in teaching and learning — and minimize pitfalls.
The future is quantum: universities look to train engineers for an emerging industry.	With quantum technologies heading for the mainstream, undergraduate courses are preparing the workforce of the future.
The Future of Music: How Generative AI Is Transforming the Music Industry.	AI-generated music has the potential to become our primary source of music in the future and influence our listening preferences. This might mark the beginning of music's "Midjourney moment."
AI Doomers Are Finally Getting Some Long Overdue Blowback.	Now, those who predicted AI will bring about our collective extinction must reconsider their claims. The "AI doom" really mainly benefited the large players, and there are plenty of chances for the open source AI movements.
There's a model for democratizing AI.	The request for recommendations made by OpenAI on integrating democratic procedures in AI decision-making comes out as constrictive and prefers to handle delicate political matters without accepting accountability, which could limit the application and efficacy of democracy in AI governance.
Copilot is an Incumbent Business Model.	Though its ultimate disruptive potential rests in redesigning workflows, a challenge that might open substantially larger market opportunities, the Copilot AI business model improves current workflows for efficiency without generating new markets or upending lower ends.

Back to index

ML news: Week 6-12 November

Research

Link	description
RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketches.	Hand-drawn sketches as a modality for goal specification in visual imitation learning. You sketch the robot execute, in other words, you can communicate with the robot with a sketch. here is the official article.
Cheating Depth: Enhancing 3D Surface Anomaly Detection via Depth Simulation	RGB-based surface anomaly detection methods have advanced significantly. However, certain surface anomalies remain practically invisible in RGB alone, necessitating the incorporation of 3D information. This new approach 3D data with RGB outperforms traditional methods for surface anomaly detection. official code.
Gaussian Mixture Solvers for Diffusion Models.	Recently, diffusion models have achieved great success in generative tasks. Gaussian mixture solvers improve the model both in speed and quality official code.
PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis.	This paper introduces PIXART-alpha, a Transformer-based T2I diffusion model whose image generation quality is competitive with state-of-the-art image generators. The model uses three elements: T5 text encodings, cross attention, and a diffusion transformer
Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch.	In this paper, we uncover that Language Models (LMs), either encoder- or decoder-based, can obtain new capabilities by assimilating the parameters of homologous models without retraining or GPUs. official code.
An Efficient Self-Supervised Cross-View Training For Sentence Embedding	Cross-View Training (SCT) allows efficient sentence embedding with small language models official code.
A Systematic Review of Deep Graph Neural Networks: Challenges, Classification, Architectures, Applications & Potential Utility in Bioinformatics	Apart from presenting all existing GNN models, mathematical analysis and comparison of the variants of all types of GNN have been highlighted in this survey. Graph neural networks are investigated for their potential real-world applications in various fields, focusing on Bioinformatics.
How AI could lead to a better understanding of the brain	Early machine-learning systems were inspired by neural networks — now AI might allow neuroscientists to get to grips with the brain’s unique complexities.
How AI can help to save endangered species	Scientists are using artificial intelligence to fight biodiversity loss by analyzing vast amounts of data, monitoring ecosystems, and spotting trends over time.
Pretraining Data Mixtures Enable Narrow Model Selection Capabilities in Transformer Models	An article from Google providing experimental evidence that the transformer (and therefore LLMs) cannot generalize beyond the training data. This is an indication that the transformer will be not the architecture leading us to artificial general intelligence (AGI)
RobustMat: Neural Diffusion for Street Landmark Patch Matching under Challenging Environments	For autonomous vehicles (AVs), visual perception techniques based on sensors like cameras play crucial roles in information acquisition and processing. In various computer perception tasks for AVs, it may be helpful to match landmark patches taken by an onboard camera with other landmark patches captured at a different time or saved in a street scene image database. The authors using spatial information and neural differential equations have created an approach to improve landmark matching. official code
I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models.	Video synthesis has recently made remarkable strides benefiting from the rapid development of diffusion models. However, it still encounters challenges in terms of semantic accuracy, clarity, and spatio-temporal continuity. This new approach is composed of two steps: preserve the static image's content and refine details and resolution.
Rethinking Benchmark and Contamination for Language Models with Rephrased Samples.	We know that better data improves LLM training, here is a better way to clean your data. The authors have published the decontaminator tool here.
Hallucination in LLMs.	We begin with an innovative taxonomy of LLM hallucinations, then delve into the factors contributing to hallucinations. Subsequently, we present a comprehensive overview of hallucination detection methods and benchmarks
Simplifying Transformer Blocks.	Combining signal propagation theory and empirical observations, we motivate modifications that allow many block components to be removed with no loss of training speed, including skip connections, projection or value parameters, sequential sub-blocks, and normalization layers. official code
LLaVA-Med: Large Language and Vision Assistant for BioMedicine	LLaVA-Med was initialized with the general-domain LLaVA and then continuously trained in a curriculum learning fashion (first biomedical concept alignment then full-blown instruction-tuning). We evaluated LLaVA-Med on standard visual conversation and question-answering tasks. official repository

News

Link	description
Google Research scholar program.	The Research Scholar Program aims to support early-career professors who are pursuing research in fields relevant to Google.
OpenAI DevDay Buzz Includes Alleged Leak Of New ChatGPT Prototype.	"highlights: OpenAI could introduce major updates for developers, making it cheaper and faster to build AI-based applications. A rumored "Team" plan for ChatGPT could offer unlimited high-speed GPT-4, advanced data analysis, and more."
Google is extending its Vulnerability Rewards Program (VRP) to include generative AI.	Today, we’re expanding our VRP to reward attack scenarios specific to generative AI. As part of expanding VRP for AI, we’re taking a fresh look at how bugs should be categorized and reported.
Paper Digest: NeurIPS 2023 Highlights.	Paper digest has analyzed more than 500/3500 papers. Interesting, but many articles are already been published for a while
HelixNet.	HelixNet is a Deep Learning architecture consisting of 3 x Mistral-7B LLMs. It has an actor, a critic, and a regenerator. The authors also used AI synthetic data. This approach showed impressive results. The model is available on HuggingFace
ChatGPT Plus members can upload and analyze files in the latest beta.	ChatGPT Plus members can also use modes like Browse with Bing without manually switching, letting the chatbot decide when to use them.
OpenAI Dev day recap.	A recap by OpenAI:New GPT-4 Turbo model that is more capable, cheaper and supports a 128K context window, New Assistants API that makes it easier for developers to build their own assistive AI apps. New multimodal capabilities in the platform, including vision, image creation (DALL·E 3), and text-to-speech (TTS)
xAI PromptIDE	Integrated development environment for prompt engineering and interpretability research, released by xAI
ChatGPT continues to be one of the fastest-growing services ever	In less than a year, it’s hit 100 million weekly users, and over 2 million developers are currently building on the company’s API, including the majority of Fortune 500 companies.
Xbox partners with Inworld AI to build AI tools for game development.	Microsoft’s Xbox and Inworld AI have partnered to create AI-powered game development tools for narrative and character creation.
Nvidia Is Piloting a Generative AI for Its Engineers.	ChipNeMo summarizes bug reports, gives advice, and writes design-tool scripts
YouTube to test generative AI features.	Users may test out a new conversational tool that utilizes artificial intelligence (AI) to respond to inquiries about YouTube content and provide suggestions, as well as a new feature that summarizes subjects in video comments, as part of the premium package offered to pay subscribers.
Google Announces Expansion of AI Partnership with Anthropic.	Partnership includes important new collaborations on AI safety standards, committing to the highest standards of AI security, and use of TPU v5e accelerators for AI inference
Cohere Introduced Embed v3	Embed v3 offers state-of-the-art performance per trusted MTEB and BEIR benchmarks. it is multilingual (100+ languages), works well with noisy data, retrieval-augmentation generation (RAG) systems, searches in a language or cross-language searches
Microsoft has over a million paying Github Copilot users	"We have over 1 million paid copilot users in more than 37,000 organizations that subscribe to copilot for business," said Nadella, "with significant traction outside the United States."
Meta's audiocraft can also generate stereo music	Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor/tokenizer, along with MusicGen, a simple and controllable music generation LM with textual and melodic conditioning.
Hugging Face has a two-person team developing ChatGPT-like AI models	Hugging Face's H4 team is focused on developing open-source ChatGPT
Samsung is joining the AI arms race, too	Samsung’s live translate feature, which the company is calling “AI Live Translate Call,” will be built into the company’s native phone app. Samsung says “audio and text translations will appear in real-time as you speak” and that the translations will happen on the device.
Introducing Adept Experiments	Adept is building AI agents and now they are opening access to test them
Introducing GPTs	You can now create custom versions of ChatGPT that combine instructions, extra knowledge, and any combination of skills. Highlight: Starting today, you can create GPTs and share them publicly. Later this month, we’re launching the GPT Store, featuring creations by verified builders. Once in the store, GPTs become searchable and may climb the leaderboards. We will also spotlight the most useful and delightful GPTs we come across in categories like productivity, education, and “just for fun”. In the coming months, you’ll also be able to earn money based on how many people are using your GPT.
Google Cloud demonstrates the world’s largest distributed training job for large language models across 50000+ TPU v5e chips	Google Cloud TPU Multislice Training was built from the ground up to address the challenges of distributed ML training in orchestration, compilation, and end-to-end optimization. We demonstrated the benefits of Cloud TPU Multislice Training with what we believe is the largest publicly disclosed LLM distributed training job in the world (in terms of the number of chips used for training) on a compute cluster of 50,944 Cloud TPU v5e chips on the JAX ML framework, utilizing both BF16 and INT8 quantized training.
OpenAI Data Partnerships.	We’re interested in large-scale datasets that reflect human society and that are not already easily accessible online to the public today. We can work with any modality, including text, images, audio, or video. We’re particularly looking for data that expresses human intention (e.g. long-form writing or conversations rather than disconnected snippets), across any language, topic, and format.

Resources

Link	description
DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference.	new software for competing with vLLM and text-generation interfaces for the fast serving of language models.
qdrant.	Qdrant (read: quadrant) is a vector similarity search engine and vector database. It provides a production-ready service with a convenient API to store, search, and manage points—vectors with an additional payload Qdrant is tailored to extended filtering support.
Video2Music	Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model. official article.
Hacking Google Bard - From Prompt Injection to Data Exfiltration.	Great post that explains what are the novel risk with generative AI plugins
RedPajama-Data-v2.	a new version of the RedPajama dataset, with 30 trillion filtered and deduplicated tokens (100+ trillion raw) from 84 CommonCrawl dumps covering 5 languages, along with 40+ pre-computed data quality annotations that can be used for further filtering and weighting. A dataset bigger than the one used for GPT-4 and already preprocessed
LLM4Rec.	The proposed CLLM4Rec is the first recommender system that tightly combines the ID-based paradigm and LLM-based paradigm and leverages the advantages of both worlds.
consistencydecoder.	OpenAI has released an Improved decoding for stable diffusion vaes. Consistency decoder has reached the SOTA and it is nice they released also for stable diffusion
TopicGPT.	We introduce TopicGPT, a prompt-based framework that uses large language models (LLMs) to uncover latent topics within a provided text collection. TopicGPT produces topics that align better with human categorizations compared to competing methods. official article.
FACTOR.	an effective tool to detect deep fakes even without training. FACTOR leverages the discrepancy between false facts and their imperfect synthesis within deepfakes. By quantifying the similarity using the truth score, computed via cosine similarity, FACTOR effectively distinguishes between real and fake media, enabling robust detection of zero-day deepfake attacks.
CogVLM.	CogVLM is a powerful open-source visual language model (VLM). CogVLM-17B has 10 billion vision parameters and 7 billion language parameters.
langroid	Langroid is an intuitive, lightweight, extensible, and principled Python framework to easily build LLM-powered applications. You set up Agents, equip them with optional components (LLM, vector-store and methods), assign them tasks, and have them collaboratively solve a problem by exchanging messages.
OVIR-3D.	D object retrieval from text prompts using 2D image fusion. his work provides a straightforward yet effective solution for open-vocabulary 3D instance retrieval, which returns a ranked set of 3D instance segments given a 3D point cloud reconstructed from an RGB-D video and a language query.
JaSPICE: Automatic Evaluation Metric Using Predicate-Argument Structures for Image Captioning Models.	an automatic evaluation metric called JaSPICE, which evaluates Japanese captions based on scene graphs. There is a gap between performance of models for english captioning and other languages, this clever approach promises to reduce the gap
awesome-openai-vision-api-experiments.	A set of examples showing how to use the OpenAI vision API to run inference on images, video files and webcam streams.
punica.	Low rank adapation (LoRA) is a parameter efficient way to add new knowledge to a pretrained LLM. Although the pretrained LLM takes 100s of GB storage, a LoRA finetuned model only adds 1% storage and memory overhead. Punica enables running multiple LoRA finetuned models at the cost of running one.
LongQLoRA.	LongQLoRA is a memory-efficient and effective method to extend context length of Large Language Models with less training GPUs. On a single 32GB V100 GPU, LongQLoRA can extend the context length of LLaMA2 7B and 13B from 4096 to 8192 and even to 12k.
Lidar-Annotation-is-All-You-Need.	a smarter method for self-driving cars to recognize roads by using lidar technology.
LM4VisualEncoding.	Pretrained transformers from LLMs, despite being trained solely on textual data, are surprisingly strong encoders for purely visual tasks in the absence of language. Our exploration shows the potential of LLMs as general-purpose encoders for visual data, as opposed to the previous usages of either pure encoders for text embeddings or decoders for tokenized outputs. official article.
vimGPT.	Browse the web with GPT-4V and Vimium. Vimium is a Chrome extension that lets you navigate the web with only your keyboard. You could use Vimium to give the model a way to interact with the web.
Announcing a New Way to Create AI Employees	the first platform letting you build a team of AI employees working together to perform any task. The idea is to build an agent that you can call and ask to perform a task

Perspectives

Link	description
Data Pipeline Attacks.	An excerpt from Secure Intelligent Machines. In the future attacks will be focused on poisoning data or other components of the data pipeline. This blog post describes this issue and potential mitigation issues
Could Cruise be the Theranos of AI? And is there a dark secret at the core of the entire driverless car industry?	Cruise is a driverless car company bought by General Motors. However, it seems that remote human interventions is needed in many cases
Will generative AI transform business?	Industries expect demand for quality control and human oversight of AI-generated content to grow
A minor ChatGPT update is a warning to founders: Big Tech can blow up your startup at any time.	Wrapping chatGPT as a core business is not a great idea. chatGPT can now interact with PDF and let you ask questions which is blowing the business of small start-ups. It's a bleak reminder that swift rule changes by Big Tech firms can wreak havoc on smaller players.
Pixel Perfect: How AI Unlocks Creativity.	AI, and creators are gaining momentum. Using the right tactics can increase it
Almost an Agent: What GPTs can do.	GPT is almost an agent, but what actually an agent can do? For instance, write a scientific article by itself
Are language models good at making predictions?	It seems so. The article suggests GPT-4 really is better at making predictions for politics than for science or technology, even once the hardness of the questions are accounted for.
OpenAI Is A Lot More Vulnerable Than You Think.	All the press, money, and awards in the world won’t prevent OpenAI from the cold reality of competition.
ChatGPT use shows that the grant-application system is broken.	The fact that artificial intelligence can do much of the work makes a mockery of the process. It’s time to make it easier for scientists to ask for research funding.
The world’s week on AI safety: powerful computing efforts launched to boost research.	UK and US governments establish efforts to democratize access to supercomputers that will aid studies on AI systems.
Is AI the Next Crypto? Insights from 2M HN comments.	Both crypto and AI have been heavily debated on Hacker News, with discussions going back years. By looking at trends in HN commenter opinions we might find interesting similarities and differences.
AI companies have all kinds of arguments against paying for copyrighted content.	The biggest companies in AI aren’t interested in paying to use copyrighted material as training data.
AI could cause ‘catastrophic’ financial crisis, says Yuval Noah Harari	Historian and Sapiens author says sophistication of technology makes it difficult to forecast its dangers
Nvidia Envy: understanding the GPU gold rush.	In 2023, thousands of companies and countries begged Nvidia to purchase more GPUs. Can the exponential demand endure?
AI is about to completely change how you use computers.	Bill Gates in his blog (yes, he has a blog) discuss how AI will revolutionize software interaction
Self Supervised Learning Market Size Thrives with AI Systems That Discover Patterns and Insights Independently	Self Supervised Learning market growth surges due to AI's ability to autonomously learn from unlabelled data, enhancing efficiency and innovation
Yoko Taro Foresees the End of Video Games as We Know Them	Yoko Taro says the rise of AI will give birth to a new era of video games in which the line between developer and player is blurred into nonexistence.
How Generative AI Will Transform Knowledge Work	Generative AI can be a boon for knowledge work, but only if you use it in the right way. New generative AI-enabled tools are rapidly emerging to assist and transform knowledge work in industries ranging from education and finance to law and medicine.

Back to index

ML news: Week 30 October - 5 November

Research

Link	description
An Emulator for Fine-Tuning Large Language Models using Small Language Models	What would happen if we combined the knowledge learned by a large model during pre-training with the knowledge learned by a small model during fine-tuning (or vice versa)? Our experiments with EFT show that scaling up fine-tuning tends to improve helpfulness while scaling up pre-training tends to improve factuality.
Nearest Neighbor Guidance for Out-of-Distribution Detection	Detecting out-of-distribution (OOD) or unfamiliar data samples is crucial for machine learning models deployed in open-world environments. NNguide can help the model in this setting, especially in identifying unknown data. Code for the benchmark,Code for the method
Towards Monosemanticity: Decomposing Language Models With Dictionary Learning	Using a sparse autoencoder, we extract a large number of interpretable features from a one-layer transformer.
AlphaFold update	AlphaFold’s update by Isomorphic (a spin-off from Google). A more powerful model that expands coverage beyond proteins. Other related information: Comment by DeepMind, official article
Mask Propagation for Efficient Video Semantic Segmentation	a method for segmenting video content that reduces computational load by focusing on keyframes and then predicting masks
Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V	how well GPT-4 with Vision (GPT-4V) answers questions related to medical images? This study analyzes exactly this offcial code
Learning From Mistakes Makes LLM Better Reasoner	Consider a human student who failed to solve a math problem, he will learn from what mistake he has made and how to correct it. Mimicking this error-driven learning process, LeMa fine-tunes LLMs on mistake-correction data pairs generated by GPT-4. Analysis of the article
AI ‘breakthrough’: neural net has human-like ability to generalize	Systematic generalization is demonstrated by people’s ability to effortlessly use newly acquired words in new settings. official article
Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks	This article benchmarks different pre-trained models on different computer vision tasks official code, analysis of the article
The Foundation Model Transparency Index	Stanford measured how transparent companies are true their Large Language Models (LLMs) and other foundation models. The results? there is a lot to improve. deep dive
SoulChat: Improving LLMs' Empathy, Listening, and Comfort Abilities through Fine-tuning with Multi-turn Empathy Conversations	Researchers developed a new method to improve empathy capabilities of large language models. This is can be very important for psychological counseling or medical application official code
Towards Foundation Models for Knowledge Graph Reasoning	A foundation model for knowledge graphs which was actually missingblog post from the authors
Evaluating Large Language Models: A Comprehensive Survey	A comprehensive overview about the evaluation of LLMs
Deep Learning for Day Forecasts from Sparse Observations	a state-of-the-art neural weather model; MetNet-3 makes predictions up to 24 hours ahead for precipitation, wind, temperature, and dew point.

News

Link	description
Google commits to invest $2 billion in OpenAI competitor Anthropic	Google agreed to invest up to $2 billion in Anthropic, the artificial intelligence startup founded by ex-OpenAI executives, CNBC has confirmed.
Amazon rolls out AI-powered image generation	Amazon Ads has introduced an AI-powered image generation feature in beta. Without technical skills, brands can now create more engaging ads
Multi-modal prompt injection image attacks against GPT-4V	Multi-modal prompt injection image attacks against GPT-4V. GPT4-V is the new mode of GPT-4 that allows you to upload images as part of your conversations. It’s absolutely brilliant. It also provides a whole new set of vectors for prompt injection attacks.
Biden releases AI executive order directing agencies to develop safety guidelines	The executive order builds on non-binding agreements the White House made with AI companies.
A group behind Stable Diffusion wants to open source emotion-detecting AI	The group wants to open source the Empathic project. This in order to improve AI-human interaction
Kuo: Apple Could Spend $4.75 Billion on AI Servers in 2024	Apple is expected to spend several billion on hardware to support its artificial intelligence development in 2024. Tim Cook has commented that they are spending quite a bit of money on AI (more details here)
Artists Lose First Round of Copyright Infringement Case Against AI Art Generators	While a federal judge advanced an infringement claim against Stability AI, he dismissed the rest of the lawsuit.
Hackers Are Weaponizing AI to Improve a Favorite Attack	Phishing attacks are already devastatingly successful. What happens when artificial intelligence makes them even harder to spot?
Chinese tech giant Alibaba launches upgraded AI model to challenge Microsoft, Amazon	Alibaba on Tuesday launched the latest version of its artificial intelligence model (Tongyi Qianwen 2.0, its latest large language model), as the Chinese technology giant looks to compete with U.S. rivals like Amazon and Microsoft.
Microsoft pushes the boundaries of small AI models with big breakthrough	Microsoft researchers shared that the model, Phi 1.5, is now “multimodal,” meaning it can view and interpret images. Phi 1.5 is open source.
New techniques efficiently accelerate sparse tensors for massive AI models	Researchers from MIT and NVIDIA have developed two techniques that accelerate the processing of sparse tensors, a type of data structure that’s used for high-performance computing tasks. The complementary techniques could result in significant improvements to the performance and energy efficiency of systems like the massive machine-learning models that drive generative artificial intelligence.
Stability AI’s latest tool uses AI to generate 3D models	Stability AI, the startup behind the text-to-image AI model Stable Diffusion, thinks 3D model creation tools could be the next big thing in generative AI.
UK invests $273 million in AI supercomputer as it seeks to compete with U.S., China	The U.K. government said Wednesday that it will invest £225 million, or $273 million, into an AI supercomputer, highlighting the country’s ambition to lead in the technology as it races to catch up to the U.S. and China.
The Beatles Just Released Their Final Song With The Help Of AI	More than 50 years after their breakup, The Beatles have released their final song — and used AI to bring John Lennon's voice back to life.
Elon Musk's first AI product is a chatbot named Grok	Elon Musk's first AI product is here, and it's a chatbot called Grok — not to be confused with rizzed-up Baby Gronk.

Resources

Link	description
Audioflare	An all-in-one AI audio playground using Cloudflare AI Workers to transcribe, analyze, summarize, and translate any audio file.
JudgeLM: Fine-tuned Large Language Models are Scalable Judges	JudgeLM is an open platform for training, serving, and evaluating scalable large language model
Deep learning in Rust	Rust is a popular language and Burn is a framework for using ML in Rust. Now, you have a free book to learn about burn in rust.
LLM Collection	a collection and summary of notable and foundational LLMs
Leveraging Embeddings and Clustering Techniques in Computer Vision	How to use CLIP to cluster images
Training LLMs at Scale with AMD MI250 GPUs	Everyone uses NVIDIA, this post discusses how to train a LLM with AMD GPU
ICTC: Image Clustering Conditioned on Text Criteria	New methodology for performing image clustering based on user-specified criteria in the form of text paper
Insanely Fast Whisper	Transcribe 300 minutes (5 hours) of audio in less than 10 minutes - with OpenAI's Whisper Large v2.
magentic	Easily integrate Large Language Models into your Python code. Simply use the @prompt decorator to create functions that return structured output from the LLM. Mix LLM queries and function calling with regular Python code to create complex logic.
PUCA: Patch-Unshuffle and Channel Attention for Enhanced Self-Supervised Image Denoising	a new self-supervised denoising approach with incredible performances
LangChain Templates	LangChain Templates are the easiest and fastest way to build a production-ready LLM application. These templates serve as a set of reference architectures for a wide variety of popular LLM use cases.
how-to guide for LLaMA	META has released a guide on how to get started with LLaMA
Fine-tuning Mistral on your own data	In this notebook and tutorial, we will fine-tune the Mistral 7B model with just 1 dollar
Amazon release Mistral 7B with longer context window	Amazon has used RoPE to extend the model context length to 32K. However, there is already a Mistral version with 128K (by Nous using the Yarn method) which you can find here
Tiger toolkit	open-source resource for developers to create AI models and language applications tailored to their needs.
parameter-efficient-MOE	Cohere has released the code base for training an efficient mixture of experts (MOE)
ChatGPT-Powered Hierarchical Comparisons for Image Classification	Conventional image classification approaches typically evaluate their performance on the same set of categories as their training data. However, this evaluation paradigm fails to capture the challenges in real-world scenarios, where classes in the test set are not overlapped with the training set. For this reason here is a simple method using ChatGPT to create hierarchical classes official code
talk-llama	Talk with an LLaMA AI in your terminal
What's In My Big Data?	WIMBD platform analyzes content in text corpora, revealing duplicates, low-quality content, PII, toxicity, and benchmark contamination. code will be released here

Perspectives

Link	description
Thanks to AI, the future of programming may involve YELLING IN ALL CAPS	Politeness and emphasis play a surprising role in AI-model communications. Some OpenAI internal prompts are leaked, showing that using caps-lock for important words and adding please is a surprisingly efficient technique
Is AI alignment on track? Is it progressing... too fast?	We do not have concrete benchmarks about alignment, this is feeding a narrative of fear and doom. but it is true? Without serious study, we cannot know, this blog post discusses it in detail
The White House Is Preparing for an AI-Dominated Future	The Atlantic perspective on the new bill: "President Biden’s big swing on AI is as impressive and confusing as the technology itself."
The Half-Life of the AI Stack	The infrastructure layer in AI is rapidly changing
Ilya Sutskever, OpenAI’s chief scientist, on his hopes and fears for the future of AI	Interviewer to one of the most famous AI researcher
How Amazon and Berkshire got too big	a perspective about threats the business growth
Seismic Waves of Gen Z Behavior	A perspective on how Generation Z is changing the industries and the market.
Andrew NG warns big tech mount on AI fear to stop competition	A leading AI expert and Google Brain co-founder said Big Tech companies were stoking fears about the technology's risks to shut down competition. Yann LeCun is also discussing the same here
Biden’s AI Order May Have Wide Impact For Startups	The new order can have a deep impact for start-up
What AI means for your product strategy	1 hour podcast about how AI will impact product strategy
4 Ways AI Is Changing Marketing	How can AI be harnessed to drive more effective and efficient marketing? Forbes is discussing this
Sifting Through the Noise	We are in the age of information overload and soon we can be flooded with AI-generated content, how we survive?
How AI detectors can destroy innocent writers' livelihoods	The massive false positive rate of general AI detectors had a devastating effect on freelance writer Michael Berben: being falsely accused of cheating, he lost his job.

Back to index

ML news: Week 23-29 October

Research

Link	description
Geographical erasure in language generation	LLMs encode a vast amount of knowledge but it is not representative of all countries, Amazon shows how to mitigate this unbalance
Entangled Preferences: The History and Risks of Reinforcement Learning and Human Feedback	A deep dive in the history of RLHF, potential issues and suggestions for new lines of research
AgentTuning: Enabling Generalized Agent Abilities for LLMs	Open-source models are inferior as AI agents when you need them as efficient controllers for complex tasks. This paper highlights how to create efficient agent LLaMA-2 models
The Foundation Model Transparency Index	Stanford's new index rates the transparency of 10 foundation model companies and finds them lacking. The new index analyses 100 parameters, showing there is room for improvements
BotChat: Evaluating LLMs' Capabilities of Having Multi-Turn Dialogues	evaluation of the ability of large language models (LLMs) to engage in human-like multi-turn conversations.
SALMONN: Towards Generic Hearing Abilities for Large Language Models	SALMONN understands text and audio at the same time, and can be used for speech recognition and speech translation. official code
FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling	While you can generate easily an image with diffusion creating a video is much more complex (consistency), this work allows generations up to 512 frames long paper, code
PDFTriage: Question Answering over Long, Structured Documents	Finding information from PDFs (web pages or other multi-page structured documents) is more difficult than for regular text. Therefore researchers at Adobe Research have developed a model that is able to consider both the text and the structure of the document
VidChapters-7M: Video Chapters at Scale	Segmenting long videos into chapters enables users to quickly navigate to the information of their interest. Here the authors collected VidChapters-7M, a dataset of 817K user-chaptered videos including 7M chapters in total.
RLMRec: Representation Learning with Large Language Models for Recommendation	In this article the authors enhanced a recommendation system with an LLM, resulting in better recommendations. code here
CommonCanvas: An Open Diffusion Model Trained with Creative-Commons Images	We assemble a dataset of Creative-Commons-licensed (CC) images, which we use to train a set of open diffusion models that are qualitatively competitive with Stable Diffusion 2 (SD2). official code
LLM-FP4: 4-Bit Floating-Point Quantized Transformers	We propose LLM-FP4 for quantizing both weights and activations in large language models (LLMs) down to 4-bit floating-point values, in a post-training manner. Existing post-training quantization (PTQ) solutions are primarily integer-based and struggle with bit widths below 8 bits.official code
Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time	For a specific input, only a small fraction of attention heads and MLP neurons are needed, while the rest can be "silenced" without changing the output. Deja Vu to speed up inference for large language models. exploiting "contextual sparsity" (finding small subsets of model parameters that are sufficient to compute the same output for a given input.). This is unlike prior pruning methods that permanently remove parameters. official code
ConvNets Match Vision Transformers at Scale	Many researchers believe that ConvNets perform well on small or moderately sized datasets, but are not competitive with Vision Transformers when given access to datasets on the web scale. The authors invested the same computer budget on a CNN to make a fair comparison with the vision transformers and they matched the performance
Llemma: An Open Language Model For Mathematics	a large language model for mathematics, the authors show how using a small model in continuous pretraining you can beat bigger models on Math and STEM. deep dive
Zephyr: Direct Distillation of LM Alignment	a 7B parameter model with competitive performance to ChatGPT on AlpacaEval

News

Link	description
New Nvidia AI agent, powered by GPT-4, can train robots	Eureka, a new AI agent (powered by GPT-4) can teach complex skills to robots
‘Mind-blowing’ IBM chip speeds up AI	IBM has developed a brain-inspired computer chip that could supercharge artificial intelligence (AI) by working faster with much less power
“Math is hard” — if you are an LLM – and why that matters	LLM success on math is still limited, especially if you just rely on a LLM
Apple Rumored to Follow ChatGPT With Generative AI Features on iPhone as Soon as iOS 18	Apple plans to start implementing generative AI technology on the iPhone and iPad in late 2024 at the earliest according to analysts
Reddit can survive without search	Reddit and other companies may stop crawlers (and be not find anymore on google search) if they do not find an agreement in generative AI
This new data poisoning tool lets artists fight back against generative AI	A new tool lets artists add invisible changes to the pixels in their art before they upload it online so that if it’s scraped into an AI training set, it can cause the resulting model to break in chaotic and unpredictable ways.
AI risk must be treated as seriously as climate crisis, says Google DeepMind chief	Demis Hassabis calls for greater regulation to quell existential fears over tech with above-human levels of intelligence
Claude accessibility is expanded to 95 countries
IBM Presents NorthPole	a new chip much faster for AI and much more energy efficient
Perplexity raises new funding at $500 million valuation	Perplexity is developing an AI-powered search engine competing with the likes of OpenAI’s ChatGPT and Google’s Bard. According to recent reports, Perplexity has been generating annual recurring revenue of $3 million as of this month.
AI rapidly diagnoses brain tumours during surgery	A machine-learning method to assess DNA can accurately classify brain tumours in real time. This rapid analysis might help surgeons to identify the tumour type when operating and to adjust their surgical strategy accordingly.
AI executive order on October 30	The Biden Administration is reportedly set to unveil a broad executive order on artificial intelligence next week.
Lenovo and NVIDIA Announce Hybrid AI Solutions to Help Enterprises Quickly Adopt GenAI	New End-to-End Solutions Include Accelerated Systems, AI Software and Expert Services to Build and Deploy Domain-Specific AI Models with Ease

Resources

Link	description
caption-usampling	DALL-3 power is derived from better data quality, this library can allow you to upsample your dataset
SolidGPT	Chat everything with your code repository, ask repository-level code questions, and discuss your requirements. AI Scan and learning your code repository, provide you code repository level answer
GoLLIE 34B	zero-shot Information Extraction model for extracting information from unstructured data (CSV, JSON, and so on)
Arithmo-Mistral-7B	Mistral 7B fine-tuned on math
GraphMaker	a diffusion model capable of generating highly realisitc large attributed graphs. original article
Meta’s Habitat 3.0 simulates real-world environments for intelligent AI robot training	Researchers from Meta Platforms Inc.’s Fundamental Artificial Intelligence Research team said today they’re releasing a more advanced version of the AI simulation environment Habitat, which is used to teach robots how to interact with the physical world.
SAM-Med3D	the most comprehensive study to modify SAM for 3D medical images. Curated the most extensive volumetric medical dataset to date for training, boasting 131K 3D masks and 247 categories. paper
deepsparse	DeepSparse is a CPU inference runtime that takes advantage of sparsity to accelerate neural network inference.
ExecuTorch	PyTorch Edge: Enabling On-Device Inference Across Mobile and Edge Devices with ExecuTorch
Spelltest: AI-to-AI Testing for LLM Based Applications	Today's AI-driven applications largely depend on Large Language Models (LLMs) like GPT-4 to deliver innovative solutions. However, ensuring that they provide relevant and accurate responses in every situation is a challenge. Spelltest addresses this by simulating LLM responses using synthetic user personas and an evaluation technique to evaluate these responses automatically(but still requires human supervision).
polyfire-js	An all-in-one managed backend for AI apps. Build AI apps from the frontend, very fast
ToRA: A Tool-Integrated Reasoning Agent	ToRA is a series of Tool-integrated Reasoning Agents designed to solve challenging mathematical reasoning problems by interacting with tools, e.g., computation libraries and symbolic solvers. ToRA series seamlessly integrates natural language reasoning with the utilization of external tools, thereby amalgamating the analytical prowess of language and the computational efficiency of external tools.
Adala	Adala offers a robust framework for implementing agents specialized in data processing, with an emphasis on diverse data labeling tasks.

Perspectives

Link	description
Emotional labor and its consequences	Emotional labor is what differentiate us from AI
The Techno-Optimist Manifesto	A blog post that has ignited a strong debate in Silicon Valley about the positive impact of technology
Peak Data	a blog post discussing what will happen if the internet is filled only with AI-generated data, this will lead probably to the collapse of AI model trained on these data
Five Areas of AI Opportunity According to Snowflake’s Ahmad Khan	Lightspeed recently hosted the latest in its Generative AI series in Los Angeles, a fireside chat with Ahmad Khan, Head of AI/ML Strategy at Snowflake
An AI revolution is brewing in medicine. What will it look like?	Emerging generalist models could overcome some limitations of first-generation machine-learning tools for clinical use.
The Convergence of Data & Software Engineering in the Age of AI	This convergence signals how far data teams have evolved into core engineering teams. Machine learning’s demand for data has accelerated this movement because AI needs data to function.
Managing AI Risks in an Era of Rapid Progress	Some of the biggest names in the field (Hinton, Bengio and so on) discuss the potential threats of AI and how to manage them

Back to index