Rēs ipSAE loquuntur : What’s wrong with AlphaFold’s ipTM score and how to fix it

Abstract

AlphaFold’s
ipTM
metric is used to predict the accuracy of structural predictions of protein-protein interactions (PPIs) and the probability that two proteins interact. Many AF2/AF3 users have experienced the phenomenon that if they trim full-length sequence constructs (e.g. from UniProt) to the interacting domains (or domain+peptide), their
ipTM
scores change, even though the structure prediction of the interaction is unchanged. The reason this happens is due to the mathematical formulation of
ipTM
in AF2/AF3, which scores the interactions of whole chains. When one chain is a single ordered domain but the other contains lots of disorder or irrelevant domains,
ipTM
is higher in the presence of those residues than it is when they are trimmed. If both chains in a PPI complex contain large amounts of disorder or accessory domains that do not form the primary domain-domain or domain/peptide interaction, the
ipTM
score can be significantly lower in the presence of those regions than when they are trimmed. The score then does not accurately represent the accuracy of the structure prediction nor whether the two proteins actually interact. We have solved this problem by: 1) including only residue pairs in the
ipTM
metric that have good predicted aligned error (
PAE
) scores; 2) by adjusting the
d
0
parameter (a function of the length of the query sequences) in the TM score equation to include only the number of residues with good interchain
PAE
s to the aligned residue; and 3) using the
PAE
value itself and not the probability distributions over the aligned error to calculate the pairwise residue-residue
pTM
values that go into the
ipTM
calculation. The first two are crucial in calculating high
ipTM
s for domain-domain and domain-peptide interactions even in the presence of many hundreds of residues in disordered regions and/or accessory domains. The third allows us to require only the common output json files of AF2 and AF3 (including the server output) without having to change the AlphaFold code and without affecting the accuracy. It also works on npz file output from Boltz. We show in a benchmark that the new score, called
ipSAE
(interaction prediction Score from Aligned Errors), is able to separate true from false complexes more efficiently than AlphaFold2’s
ipTM
score. The resulting program is freely available at
https://github.com/dunbracklab/IPSAE
.