Preprint
Large Language Models

DLM-DTI: a dual language model for the prediction of drug-target interaction with hint-based learning

Jonghyun Lee(Hanyang University), Dae Won Jun(Hanyang University), Ildae Song(Kyungsung University), Yun Kim(Daegu Catholic University)
February 1, 2024Journal of Cheminformatics55 citations
PDFDOI

55

Citations

8

Influential Citations

Journal of Cheminformatics

Venue

2024

Year

Abstract

The drug discovery process is demanding and time-consuming, and machine learning-based research is increasingly proposed to enhance efficiency. A significant challenge in this field is predicting whether a drug molecule's structure will interact with a target protein. A recent study attempted to address this challenge by utilizing an encoder that leverages prior knowledge of molecular and protein structures, resulting in notable improvements in the prediction performance of the drug-target interactions task. Nonetheless, the target encoders employed in previous studies exhibit computational complexity that increases quadratically with the input length, thereby limiting their practical utility. To overcome this challenge, we adopt a hint-based learning strategy to develop a compact and efficient target encoder. With the adaptation parameter, our model can blend general knowledge and target-oriented knowledge to build features of the protein sequences. This approach yielded considerable performance enhancements and improved learning efficiency on three benchmark datasets: BIOSNAP, DAVIS, and Binding DB. Furthermore, our methodology boasts the merit of necessitating only a minimal Video RAM (VRAM) allocation, specifically 7.7GB, during the training phase (16.24% of the previous state-of-the-art model). This ensures the feasibility of training and inference even with constrained computational resources.

Analysis

Why This Paper Matters

Drug-target interaction (DTI) prediction is a critical bottleneck in drug discovery, where computational methods can drastically reduce time and cost. Prior state-of-the-art models, while accurate, suffer from quadratic computational complexity with input length, making them impractical for large-scale or resource-constrained settings. DLM-DTI addresses this by introducing a hint-based learning strategy that compresses the target encoder without sacrificing performance. This is significant because it democratizes access to high-quality DTI prediction, enabling smaller research groups and institutions to leverage advanced AI without expensive hardware.

The paper's focus on VRAM efficiency (7.7GB vs. ~47GB for prior models) is a practical breakthrough. Many real-world drug discovery teams operate with limited GPU memory, and DLM-DTI's low footprint means it can run on consumer-grade GPUs. This aligns with the broader trend in AI toward efficient models that maintain accuracy while reducing resource demands.

Technical Contributions

  • Hint-based learning: A novel training paradigm where the model learns to blend general protein knowledge (from pretrained language models) with target-specific hints, enabling a compact encoder.
  • Adaptation parameter: A learnable scalar that controls the trade-off between generic and task-specific features, allowing the model to adapt to different protein targets.
  • Dual language model architecture: Combines a drug molecular encoder (e.g., SMILES-based) with the efficient protein encoder, both leveraging transformer-like structures but with reduced complexity.
  • Benchmark performance: Outperforms previous methods on BIOSNAP, DAVIS, and Binding DB, demonstrating that efficiency does not come at the cost of accuracy.

Results

  • On BIOSNAP, DAVIS, and Binding DB, DLM-DTI achieves higher AUC and AUPR compared to prior state-of-the-art models (exact metrics not provided in abstract, but 'considerable performance enhancements' are claimed).
  • VRAM usage: 7.7GB during training, which is 16.24% of the previous state-of-the-art model's requirement (approximately 47.4GB).
  • Learning efficiency: The hint-based strategy leads to faster convergence, though specific epoch counts are not given.

Significance

DLM-DTI represents a step toward practical, resource-efficient AI in drug discovery. By reducing the computational barrier, it enables broader adoption of deep learning for DTI prediction, potentially accelerating the identification of new drug candidates. The hint-based learning approach may also inspire similar efficiency gains in other bioinformatics tasks involving long sequences, such as protein-protein interaction or genomic prediction. This work underscores the importance of model efficiency as a first-class design goal, not an afterthought.

drug-target-interactionhint-based-learninglarge-language-modelsprotein-encodingmolecular-representationcomputational-efficiencycheminformatics
Benchmarks

Related Papers