AIntibody: an experimentally validated in silico antibody discovery design challenge | Nature Biotechnology

Nature Biotechnology (2024)Cite this article Metrics details Science is frequently subject to the Gartner hype cycle1: emergent technologies spark intense initial enthusiasm with the recruitment of

Nature Biotechnology (2024)Cite this article

Metrics details

Science is frequently subject to the Gartner hype cycle1: emergent technologies spark intense initial enthusiasm with the recruitment of dedicated scientists. As limitations are recognized, disillusionment often sets in; some scientists turn away, disappointed in the inability of the new technology to deliver on initial promise, while others persevere and further develop the technology. Although the value (or not) of a new technology usually becomes clear with time, appropriate benchmarks can be invaluable in highlighting strengths and areas for improvement, substantially speeding up technology maturation. A particular challenge in computational engineering and artificial intelligence (AI)/machine learning (ML) is that benchmarks and best practices are uncommon, so it is particularly hard for non-experts to assess the impact and performance of these methods. Although multiple papers have highlighted best practices and evaluation guidelines2,3,4, the true test for such methods is ultimately prospective performance, which requires experimental testing.

In the 1990s, several groups attempted to predict the structures of proteins from amino acid sequences, and the success and value of different models were assessed ad hoc. The Critical Assessment of Structure Prediction (CASP)5 biannual competition was established in 1994 to compare the performance of various algorithms. In this competition, teams used experimental methods (predominantly X-ray crystallography and NMR) to determine protein structure, but supplied only the protein sequences to the modeling community. Expert modeling teams then used either a combination of human expertise and computational methods or fully automated methods to predict the correct structure from the sequence. Teams were allowed to provide up to five models per target. An independent panel compared the predicted and experimentally determined structures. Widely viewed as the ‘protein structure prediction world championship’, the competition demonstrated incremental improvements in structural predictions until AlphaFold6 won dramatically in 20187 and 20208. Although AlphaFold has not competed since, many methods today are inspired by the AlphaFold architecture, collectively demonstrating the power of deep learning algorithms for protein structure prediction. Other, similar competitions aimed at predicting antibody structure from sequence9,10 were initiated but have not been held since 2014.

This is a preview of subscription content, access via your institution

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

$29.99 / 30 days

cancel any time

Subscribe to this journal

Receive 12 print issues and online access

$209.00 per year

only $17.42 per issue

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Linden, A. & Fenn, J. Understanding Gartner’s hype cycles (Strategic Analysis Report no. R-20-1971). Gartner Inc. 88, 1423 (2003).

Google Scholar

Lewis Chinery, A. M. H. et al. Preprint at bioRxiv https://doi.org/10.1101/2024.03.26.586756 (2024).

Wossnig, L., Furtmann, N., Buchanan, A., Kumar, S. & Greiff, V. Drug Discov. Today 29, 104025 (2024).

Article CAS PubMed Google Scholar

Bender, A. et al. Nat. Rev. Chem. 6, 428–442 (2022).

Article PubMed Google Scholar

Moult, J. Curr. Opin. Biotechnol. 7, 422–427 (1996).

Article CAS PubMed Google Scholar

Senior, A. W. et al. Nature 577, 706–710 (2020).

Article CAS PubMed Google Scholar

Senior, A. W. et al. Proteins 87, 1141–1148 (2019).