Supportability

last update; Jun. 18. 2020

Supportability

(1) Purpose

Although strength of coexpression is represented by MR value, the coexpression may be artifact in a platform. A measure, supportability, is introduced to quantify reproducibility of a coexpressed gene list of interest.

History on ATTED-II

The supportability is previously called as reliability. With the refinement at Sep 2015, we renamed the reliability to supportability for the following two reasons.

We are trying to evaluate quality of coexpression data from various aspects. Reliability is not suitable to mean one of such measures.
Coexpression supported by another platform is reliable, but the converse is not true. Namely, coexpression without any supports can be occured when appropriate reference is not available.

Period	Term	Calculation of coex list similarity	Null distribution	Threshold
Period	Term	Calculation of coex list similarity	Null distribution	☆	☆☆	☆☆☆
2013-08 ~ 2015-08	Reliability	COXSIM(100)	Common for all platform virtually including 10000 genes	E-04	E-12	E-20
2015-09 ~ now	Supportability	COXSIM(1%)	Enpirical COXSIM distributions between an Arabidopsis (Ath-r) gene and a rice (Osa-r) gene irrespective of gene homology.	E-01	E-02	E-03

(2) Basic idea

When a gene list is repeatedly observed in indipendent platforms, the coexpressed gene list can be regarded as reliable.

Example of well-supported coexpressed gene list: DHCR7 (Hsa)
Example of less-supported coexpressed gene list: CCND3 (Hsa)

There are two possible ways to compare coexpression for reliability assessment. One is comparison of gene pairs (A), and the other is comparison of gene lists (B).

We employ the B-type (gene list) comparison because pseudo coexpression is mainly caused by inappropriate probes with weak hybridization or cross-hybridization and thus pseudo coexpression appears not only one gene pair but also all gene pairs from the problematic guide gene.

(3) Degree of coincidence of two coexpressed gene lists

Basic idea

We introduced a similarity measure COXSIM, which is the weighted concordance rate between the coexpressed gene listf from a guide gene g of interest (list_g) and that from a reference guide gene r (list_r). COXSIM is a function of guide gene g, guide gene r and threshold k.

, where n(i, list_g, list_r) is the number of common genes (orthologous genes in the case using platforms in different species) found in the top i coexpressed gene lists.

Excluding orphan genes for the gene list comparison

However, there are genes in a platform for list_g that do not have corresponding genes in a platform for list_r. When such genes appear at high ranks in list_g, the coincidence of the two lists decreases. To avoid the effect of the absence of the corresponding genes in the reference platform, genes that lack corresponding genes in the reference platform are excluded from list_g, leaving list_g→r. In the same way, genes in list_r that lack corresponding genes in list_g are excluded, resulting list_r→g. Subsequently, we examined the top k coexpressed genes in list_g→r with the reference gene list, list_r→g.

Selection of k

As k, we use 1% of the number of the genes in list_g→r.

We previously used 100 for k, meaning that we checked the gene correspondence of the top 100 coexpressed genes, in accordance with the default representation of a coexpressed gene list on COXPRESdb. However, the use of a common threshold for all platforms causes different stringencies of the coexpression thresholds. For example, the Sce platform for S. cerevisiae has 4,461 genes for coexpression analysis, whereas the Hsa platform for human has 19,803 genes. The former has four to five times higher probability to randomly include a particular gene in the top k rank, and thus overestimates the significance for the coincidence of the gene lists. Therefore, we have modified the number of genes from the top k to the top 1% of all genes in list_g→r.

(4) Selection of the most appropriate reference guide gene / gene list

Since the best reference guide gene is unknown, we checked all possible reference guide genes. The reference guide gene set R is composed of all available orthologous genes for different species. When multiple platforms are available for the species including the guide gene g, the same gene in the other platforms is also included in the reference guide gene set R. The COXSIM values are calculated between the target guide gene g and every reference gene r in R. The reference gene r_max that gives the maximum COXSIM value is regarded as the best reference guide gene.

(5) Calculation of p-value

To assess statistical significance, the maxCOXSIM value is compared with the null distribution generated under the same number of genes in list_g→r.

(6) Discretization of maxCOXSIM significance

On the ATTED-II, the significance level, which we call the supportability, is shown as the number of stars according to the following p-value threshold.

p-value threshold	Representation
1E-01	☆
1E-02	☆☆
1E-03	☆☆☆

(7) Result

Platform	☆☆☆	☆☆	☆	No star	No reference	Total
Ath-m	4080	8014	5619	2063	1060	20836
Ath-r	3977	8377	6834	5309	799	25296
Bra-r	545	4737	9697	17620	2832	35431
Gma-m	1107	3815	6863	3710	407	15902
Gma-r	1675	6803	12919	18211	3179	42787
Osa-m	1709	5239	6793	5391	1493	20625
Osa-r	1470	4674	6478	4526	400	17548
Ppo-m	465	2267	4844	8202	6131	21909
Sly-m	257	1229	2323	1880	97	5786
Sly-r	654	2824	5657	12358	1702	23195
Vvi-m	247	1118	2756	5194	249	9564
Zma-m	1052	3455	3617	2354	591	11069
Zma-r	1408	5112	7420	7298	1354	22592

Limitation

To use the supportability as a measures of reliability, comparison should be done between species having same genetic systems about the function of interest. Namly comparison between two platforms in the same species or closely related species is required. The table shows the ratio of supportability levels in each platform, indicating that Arabidopsis, soybean, tomato, rice and maize have higher ratio of the three stars. This does not directly mean that the absolute quality of these coexpression data is high, because they just have ideal reference platforms.