The Ontology of Biological and Clinical Statistics (OBCS)-based statistical method standardization and meta-analysis of host responses to yellow fever vaccines

Abstract

Background: The community-based Ontology of Biological and Clinical Statistics (OBCS) represents and standardizes biological and clinical data and statistical methods.

Methods: Both OBCS and the Vaccine Ontology (VO) were used to ontologically model various components and relations in a typical host response to vaccination study. Such a model was then applied to represent and compare three microarray studies of host responses to the yellow fever vaccine YF-17D. A literature meta-analysis was then conducted to survey yellow fever vaccine response papers and summarize statistical methods, using OBCS.

Results: A general ontological model was developed to identify major components in a typical host response to vaccination. Our ontology modeling of three similar studies identified common and different components which may contribute to varying conclusions. Although these three studies all used the same vaccine, human blood samples, similar sample collection time post vaccination, and microarray assays, statistically differentially expressed genes and associated gene functions differed, likely due to the differences in specific variables (e.g., microarray type and human variations). Our manual annotation of 95 papers in human responses to yellow fever vaccines identified 38 data analysis methods. These statistical methods were consistently represented and classified with OBCS. Eight statistical methods not available in existing ontologies were added to OBCS.

Conclusions: The study represents the first single use case of applying OBCS ontology to standardize, integrate, and use biomedical data and statistical methods. Our ontology-based meta-analysis showed that different experimental results might be due to different experimental assays and conditions, sample variations, and data analysis methods.

Graphical abstract

Keywords

OBCS / ontology / vaccine / host response to vaccination / statistical data analysis

INTRODUCTION

Statistics has always played a crucial role in protocol-driven biomedical research. In the era of “Big Data”, this role has become even more prominent, since statistics serves as the central tool in basic biological research and clinical trials. If the data, data processing and analysis, and statistical results of such information-driven research are to be understandable, all of these components should be annotated in a consistent and logically well-defined way. Unfortunately, most statistical studies in biological and clinical research have not been described in this way. Traditional statistical studies performed by independent investigators may have their own standards to process and analyze data. However, such standards are usually not available to other studies performed in different groups. This will affect big data analysis among different studies performed by multiple investigators, since these individually identified standards are only applicable to individual groups, with a lack of a universally standardized method that can be used across multiple related groups. In addition, different data types, statistical methods, and result types have inherent semantic relations, which are not presented in traditional standardized statistical studies. This creates an obstacle to the successful standardization of statistics in the sort of complex projects of analysis and automated inference now typical when very big (and heterogeneous) data are frequently generated and stored. The lack of such a formal representation prevents careful inspection, comparison and reproduction of statistical methods and statistical tests applied to data transformation and analysis, thereby reducing the ability to extract meaningful data from large data sets in the ways required by good scientific practice.

To address the obstacles described above, the Ontology of Biological and Clinical Statistics (OBCS; https://github.com/obcs/obcs) was recently developed as a community-based ontology of statistics in the biological and clinical domains [1]. An ontology is the human-and computer-interpretable representation of the types of entities in a specific scientific domain and of the relations between these types. Since the creation of the first version of the Gene Ontology in 1998 [2], hundreds of ontologies have been created and deposited in resources such as NCBO BioPortal [3] and Ontobee [4]. These ontologies achieved considerable success initially in biological data representation, and have led to a paradigm change in the way terminologies are treated now also in the field of health informatics [5]. The role of ontologies in biomedical data standardization, sharing, and analysis in the big data era has become increasingly important [5–8]. OBCS extends the Ontology of Biomedical Investigations (OBI) [9] and targets the domain of statistics in the biological and clinical fields. Co-developed by some 20 different disciplinary communities, OBI provides a set of logically defined terms covering a broad range of biomedical processes, experimental conditions, and data analysis methods [9]. As an ontology of the Open Biological and Biomedical Ontologies (OBO) Foundry ontology library [10], the OBCS development follows the OBO Foundry principles including openness and collaboration [10]. The original OBCS paper introduced the general OBCS development strategy, top level design, design pattern, and concise examples in biological and clinical areas [11]. With different demonstrations, OBCS allows not only consistent representation, classification, comparison and integration across heterogeneous datasets, but also computer-assisted automatic reasoning.

In this paper, we will focus on the single domain of vaccinology and illustrate how OBCS can be used to represent and model the studies that have generated data of host immune responses to a specific type of vaccine. For such a study, it is ideal to have a domain-specific ontology. The Vaccine Ontology (VO) represents various entities of vaccines, vaccine components, vaccinations, host responses to vaccines, etc [12–14]. In conjunction with classes from VO, we discuss how OBCS is used to represent vaccine-specific statistical data process and analysis.

Our vaccine-specific study is focused on vaccines against yellow fever, a re-emerging infectious disease caused by the yellow fever virus (a RNA virus of the genusFlavivirus) [15,16]. Spread by the bite of an infected female mosquito, yellow fever is endemic to tropic regions of Africa and America and affects humans and nonhuman primates. Common symptoms include fever, chills, loss of appetite, nausea, muscle pains, and headaches. Liver damage, jaundice, and kidney problems often occur. The live attenuated yellow fever vaccine 17D (YF-17D) was developed by Theiler and Smith in 1937 [17], and substrains derived from the original 17D strain are still used for vaccinations today [16].Theiler was awarded the Nobel Prize for his life-saving yellow fever vaccine research in 1951, so far the only Nobel Prize awarded for viral vaccine research [18]. Roukens and Visser reported in 2008 that vaccination with YF-17D induced low-grade viraemia in half of the vaccinees and elicited protective neutralizing antibody levels in 99% of the vaccinees [19]. As an effective vaccine against yellow fever viral infection, YF-17D has been used as a vaccine model for analysis of protective vaccine immune mechanism [20–23].

For a comprehensive OBCS application demonstration, OBCS, together with VO and other biomedical ontologies, are used in this study to represent metadata, experimental data, and statistical data analyses of example data sets from yellow fever vaccine research studies.

RESULTS

General ontological modeling of vaccine immune response study and data analysis using OBCS, OBI, and VO

Figure 1 provides a general ontological representation of how OBCS combined with other ontologies is used to represent a typical study of host responses to vaccination and how data can be analyzed. Specifically, an organism (e.g., human) who has the vaccine host role (VO: vaccine host role) is first vaccinated with a specific vaccine, which has the vaccine function (including two subclass functions: preventive vaccine function and therapeutic vaccine function). At a time before or after the vaccination process, there is a specimen collection process (OBI: specimen collection process) which results in the output of a specimen. After material processing (OBI: material processing), the processed specimen’ (OBI: processed specimen) is used as an input of an experimental assay (OBI: assay). Such an assay will generate new data sets (IAO: data sets). The data sets can be used for data transformation in statistics (OBCS: data transformation in statistics). The resulting transformed data sets (OBCS: transformed data set), especially the comparison of transformed data sets which are about different specimens collected at different times relative to an vaccination, are then used for statistical data analysis (OBCS: statistical data analysis), which will generate a specific data item (IAO: data item) (e.g.,p-value), which can be used to draw a conclusion (OBI: drawing a conclusion based on data) (Figure 1).

As shown in the general data modeling, different components are involved. All these components and the relations among these components in Figure 1 are represented using terms in existing ontologies, mainly OBCS, OBI and VO. Many terms from OBI and IAO (The Information Artifact Ontology, an ontology used for representation of information entities, such as data, document, and protocol, https://github.com/information-artifact-ontology/IAO/) have already been imported to OBCS and/or VO. Note that in vaccine research, a typical study is to compare the host responses before and after vaccination. In this case, time relative to vaccination is an independent variable in a statistical analysis. The time immediately before a vaccination is usually defined as day 0. A time after vaccination can then be defined accordingly. It is also typical to analyze the host responses at different days after vaccination.

OBCS application in representing a typical study of host responses to vaccination

Gaucher et al. published a paper in 2008 that reports their applying functional genomics and polychromatic flow cytometry to define the signatures of human immune responses to yellow fever vaccine 17D (YF-17D), in a total of 43 volunteers at different time points (up to 1 year) after vaccination [22]. For the transcriptomics study, RNA samples obtained from the whole blood of 15 vaccinated individuals on day 0, 3, 7, 10, 14, 28, and 60 after vaccination were collected, amplified and hybridized onto Illumina microarray chips. Their data analysis identified 594 genes whose expression profiles changed significantly between day 0 and any time point after vaccination [22].

Figure 2 represents the functional genomics study performed by Gaucher et al. [22] that was generated based on the Figure 1 general model. In this use case, the vaccine is YF-VAX (made with a specific YF-17D strain), and the vaccine host is human. The time of specimen collection from blood specimen includes day 0 (before vaccination) and day 3,7,10,etc. after vaccination. The extracted RNAs were used for transcription profiling, which generated a large amount of gene expression data sets. After quantile transformation (a data transformation method), the results were used through LIMMA data analysis [24], which generated differentially expressed gene lists. Based on the gene lists and following Gene Set Enrichment Analysis (GSEA) [25], new results about transcription factors were generated and used to draw further conclusions based on data (Figure 2).

OBCS-based comparison of three studies of host responses to YF-17D vaccinations

In addition to the above study of human responses to YF-17D vaccination, there have been two similar studies [23,26]. Specifically, in the study of YF-VAX-vaccinated human responses by Schereret al. [23], with 20 human subjects (11 females and 9 males) between the ages of 18 and 32 and in good health. Interestingly, their study identified 615 genes exhibiting significant changes in gene expression in peripheral blood mono-nuclear cells (PBMCs) at 4–7 days after vaccination with YF-VAX. The YF-VAX host response from this study was identified with a suppression of ribosomal and translation factors [23]. It appears that although many differentially expressed genes were shared, the results from this study differed significantly from the study by Gaucheret al. [22]. In the third study reported by Querec et al. [26], 15 healthy humans not previously vaccinated with YF-17D participated in the vaccination study. Blood samples were collected at days 0, 1, 3, 7 and 21 after vaccination. The transcriptional profiling of total PBMCs from these 15 subjects was conducted using Affymetrix Human Genome U133 Plus 2.0 Arrays. Interestingly, only 97 genes were identified to be modulated by YF-17D vaccination [26].

These three publications all examined host immune responses to YF-17D vaccine using array approaches to detect gene expression changes in blood cells from different groups of human subjects. Surprisingly, the results from these three studies differed [22,23,26]. Particularly, the numbers of statistically differentially expressed genes in these studies were found different: 594 [22], 615 [23], and 97 [26] (Table 1). We hypothesized that such differences might be due to some differences in experimental designs, microarray data analysis methods, and experimental conditions. Table 1 compares these differences in the three studies in detail. While our comparative study showed lots of common information among three studies, our study also identified some differences. For example, the study by Gaucheret al. [22] used bead chips manufactured by Illumina; however, the study by Schereret al. [23] used Agilent’s glass slide arrays that are Cy3/Cy5 two-color arrays, and Querecet al. [26] used Affymetrix Human Genome U133 Plus 2.0 microarrays. In addition, due to different kinds of array used, the raw data process software and following statistical analysis methods are different (shown in Table 1).

Ontology-based meta-analysis of yellow fever vaccine investigation studies

The above two sections introduced three YF-17D vaccine studies. It is clear that they are similar, but also have some differences in their details. To identify similar studies, we performed a systematic literature data analysis annotation. By searching the keyword “human response to yellow fever vaccine” in PubMed, we found 245 related papers. After cleaning and filtering (excluding the invalid papers such as non-English papers, abstract papers, papers with unclear statistical methods, etc.), 95 valid papers were identified (Supplementary Table 1). We went through all the papers and represented the statistical analysis methods consistently using ontology terms. Our mapping of these methods to OBCS identified 8 terms unavailable in OBCS. These new terms were added to OBCS using the standard OBCS development procedure [1]. The latest OBCS release is available at http://purl.obolibrary.org/obo/obcs.owl. The ontology can be visualized on the ontology server Ontobee [4] at: http://www.ontobee.org/ontology/OBCS. The annotation resulted in a total of 38 statistical methods used in these papers (Table 1). These 38 statistical methods cover different types of data transformation and statistical data analysis.

To identify the relations among these 38 methods, we used the Ontofox tool [27] to generate an OBCS subset that contains these methods and their associated parent terms (Figure 3). In contrast to Table 1, the hierarchical display in Figure 3 clearly shows more specific relations among different methods. For example, Figure 3 classifies three specific ANOVA tests (one-way, two-way, and repeated measure ANOVA) under the parent ANOVA test, which is defined as a statistical hypothesis test. Figure 3 also groups five statistical tests under the parent test non-parametric test, which does not require the data to fit a normal distribution.

DISCUSSION

In this study, we focus on the specific vaccine host response domain, and demonstrate how the OBCS, combined with the domain-specific Vaccine Ontology (VO), can be used to represent experimental data and metadata, and statistical data analyses of host immune responses to various immunizations with yellow fever vaccine data sets. Our ontology-based methods start with an initial general modeling of host immune response to vaccine study (Figure 1), followed with one typical yellow fever vaccine study (Figure 2), then a comparison between three similar studies (Table 1), and eventually with a meta-analysis of all possible studies reported in PubMed (Table 2). From about 100 peer-reviewed papers, our manual survey and annotations identified 38 statistical methods used in these studies. Our study demonstrates that OBCS, combined with terms from OBI, VO, and other ontologies can be used to comprehensively model and represent experimental studies and related data analysis methods from a single biomedical domain, in a consistent and robust way.

Three published studies, one reported by Gaucher et al. 2008 [22], one by Scherer et al. 2007 [23], and the last by Querec et al. [26], all used microarrays to detect gene expression profiles in human subjects vaccinated with the same yellow fever vaccine (YF-17D) [22,23]. These studies all used microarray approaches to examine the gene expression profiles in blood cells from human subjects. Although the same vaccine (YF-Vax) and the same LIMMA data analysis were used in two studies [22,23], these two studies generated quite different results. While we can say this is likely due to human variation since the human subjects differ, it is also likely that there were many differences other than human variation. Our ontology-based modeling identified many differences in experimental operations (e.g., different microarray format) and the processing of raw data. Such a study confirmed our hypothesis that these three studies used different experimental designs, had different experimental conditions, and relied upon different data analysis methods and software. While such a conclusion of hypothesis testing might be considered trivial to vaccine research experts, the ontology-based systematic analysis provides a reasonable semantic platform and basis for more advanced computer-assisted analyses (e.g., associations between statistical methods and questions to be addressed, and identification of optimal statistical methods) in the future.

Our meta-analysis of various statistical methods in the yellow fever vaccine host response studies identified nearly 40 statistical methods. Some methods, such as Student’st-test, Fisher’s exact test, Mann-Whitney Utest, ANOVA, and Chi-square test, are commonly used (Table 2). Our OBCS-based classification clearly classifies these methods. The identification and classification of these statistical methods facilitate our better understanding of what and how statistical methods were used in reported yellow fever vaccine studies, and whether these statistical methods applied to similar experimental conditions/designs might have resulted in different results. In the future, we plan to extend our study to other biomedical scenarios and use cases and detect possible associations between statistical methods and experimental/clinical assays.

Although OBCS and VO are two different ontologies, they can be conveniently integrated and merged. One main reason is that both ontologies use the Basic Formal Ontology (BFO) as the upper level ontology [28]. BFO is the default upper level ontology for OBO library ontologies. Over 100 biomedical ontologies, including OBCS and OBI, also use BFO as their upper level ontology. Therefore, the usage of BFO makes all BFO-aligned ontologies well interoperable. Similar to the combined usage OBCS with VO, OBCS can be combined with other biomedical ontologies for the ontological representation of research in different biomedical domains.

Our vaccine case study demonstrates that OBCS, together with VO, can play an important role in standardizing, representing, and analyzing data in vaccine research to facilitate automated and reproducible comparisons of related studies. The OBCS ontology-based data representation can be applied to many areas. First, OBCS combined with a domain specific ontology can serve as a semantic framework that links all pieces of data and data analysis methods together in a logical manner. Second, such an ontology strategy can easily identify inconsistent results and whether the inconsistency is due to different analysis methods. Third, we can also apply the OBCS ontology to build up standardized statistical methods to analyze various data types. Fourth, an OBCS-based system can be used to generate a metadata ontology system to represent ontology-annotated experimental data. In the future, we plan to generate such a metadata ontology based on OBCS and other OBO Foundry ontologies and apply it to standardize all the terms and perform advanced data and metadata-analysis of large datasets combined from multiple studies. Such ontology-based approaches will enhance our discovery of new knowledge based on integration of data from multiple experiments in the same domain.

OBCS supports precision medicine. Precision medicine requires that specific details be modeled to identify the contributing factors to a specific health outcome. As shown in this study, OBCS-based ontology modeling allows the identification of specific variables and their values in various studies within the same research or clinical domain such as the host responses to yellow fever vaccinations shown in this study.

METHODS

Ontological modeling of host responses to vaccinations

The ontological modeling of host responses to vaccinations is primarily based on the OBCS and VO. The standard procedure in ontology modeling was used in this study [1,12–14]. After the general modeling was generated, we applied the modeling to studies of host responses to yellow fever vaccination identified from the PubMed literature database. The YF-17D vaccination was emphasized in this study. We also focused on the topic of different results obtained from similar experimental design, particularly the responses of human subjects to the same YF-17D vaccinations as reported from the three studies [22,23,26]. More studies were conducted as a meta-analysis described below.

Ontology-based meta-analysis of host responses to yellow fever vaccinations

The PubMed literature database was searched by using the keyword “human response to yellow fever vaccine”. Manual annotation was performed to identify statistical methods described in publications, which were then mapped to OBCS. Those terms that were not available in OBCS were added to OBCS using standard OBCS development procedure [9]. The Protégé OWL Editor [29] was used to edit OBCS. The HermiT reasoner (http://hermit-reasoner.com/) tool was used for reasoning over OBCS to detect possible inconsistencies or conflicts.

Generation of an OBCS subset containing related statistics terms used in the study

All related OBCS terms used in this study were output as a subset of OBCS using the Ontofox tool (http://ontofox.hegroup.org) [27]. The generated OBCS subset was displayed using the Protégé OWL Editor version 5.0 beta [30].

References

[1] Zheng, J., Harris,  M. R., Masci,  A. M., Lin,  Y., Hero, A. ,  Smith, B.  and  He, Y. (2016) The Ontology of Biological and Clinical Statistics (OBCS) for standardized and reproducible statistical analysis. J. Biomed. Semantics, 7, 53160;

[2] Ashburner, M., Ball,  C. A., Blake,  J. A., Botstein,  D., Butler, H. ,  Cherry, J. M. ,  Davis, A. P. ,  Dolinski, K. ,  Dwight, S. S. ,  Eppig, J. T. ,  (2000) Gene ontology: tool for the unification of biology. Nat. Genet., 25, 25–29160;

[3] Salvadores, M., Alexander,  P. R., Musen,  M. A. and Noy,  N. F. (2013) BioPortal as a dataset of linked biomedical ontologies and terminologies in RDF. Semant Web, 4, 277–284

[4] Ong, E., Xiang,  Z., Zhao, B. ,  Liu, Y. ,  Lin, Y. ,  Zheng, J. ,  Mungall, C. ,  Courtot, M. ,  Ruttenberg, A.  and  He, Y. (2017) Ontobee: A linked ontology data server to support ontology term dereferencing, linkage, query and integration. Nucleic Acids Res., 45, D347–D352160;

[5] Schulz, S., Balkanyi,  L., Cornet, R.  and  Bodenreider, O.  (2013) From concept representations to ontologies: a paradigm shift in health informatics? Healthc. Inform. Res., 19, 235–242

[6] Blake, J. A. and Bult, C. J. (2006) Beyond the data deluge: data integration and bio-ontologies. J. Biomed. Inform., 39, 314–320

[7] Hoehndorf, R., Schofield,  P. N. and Gkoutos,  G. V. (2015) The role of ontologies in biological and biomedical research: a functional perspective. Brief. Bioinform., 16, 1069–1080

[8] Bodenreider, O. (2009) Biomedical ontologies in action: role in knowledge management, data integration and decision support. Yearb. Med. Inform., 67–79 

[9] Bandrowski, A., Brinkman,  R., Brochhausen, M. ,  Brush, M. H. ,  Bug, B. ,  Chibucos, M. C. ,  Clancy, K. ,  Courtot, M. ,  Derom, D. ,  Dumontier, M. ,  (2016) The ontology for biomedical investigations. PLoS One, 11, e0154556

[10] Smith, B., Ashburner,  M., Rosse, C. ,  Bard, J. ,  Bug, W. ,  Ceusters, W. ,  Goldberg, L. J. ,  Eilbeck, K. ,  Ireland, A. ,  Mungall, C. J. ,  (2007) The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat. Biotechnol., 25, 1251–1255

[11] Zheng, J., Harris,  M. R., Masci,  A. M., Lin,  Y., Hero, A. ,  Smith, B.  and  He, Y. (2016) The Ontology of Biological and Clinical Statistics (OBCS) for standardized and reproducible statistical analysis. J. Biomed. Semantics, 7, 53

[12] He, Y., Cowell,  L., Diehl, A. D. ,  Mobley, H. L. ,  Peters, B. ,  Ruttenberg, A. ,  Scheuermann, R. H. ,  Brinkman, R. R. ,  Courtot, M. ,  Mungall, C. ,  (2009) VO: Vaccine Ontology. In The 1st International Conference on Biomedical Ontology (ICBO-2009). Nature Precedings: 

[13] Özgür, A. ,  Xiang, Z. ,  Radev, D. R.  and  He, Y. (2011) Mining of vaccine-associated IFN-γ gene interaction networks using the Vaccine Ontology. J. Biomed. Semantics, 2, S8

[14] Lin, Y. and He,  Y. (2012) Ontology representation and analysis of vaccine formulation and administration and their effects on vaccine immune responses. J. Biomed. Semantics, 3, 17

[15] Beasley, D. W. ,  McAuley, A. J.  and  Bente, D. A.  (2015) Yellow fever virus: genetic and phenotypic diversity and implications for detection, prevention and therapy. Antiviral Res., 115, 48–70

[16] Gardner, C. L.  and  Ryman, K. D.  (2010) Yellow fever: a reemerging threat. Clin. Lab. Med., 30, 237–260

[17] Theiler, M. and Smith, H. H. (1937) The use of yellow fever virus modified by in vitro cultivation for human immunization. J. Exp. Med., 65, 787–800

[18] Norrby, E. (2007) Yellow fever and Max Theiler: the only Nobel Prize for a virus vaccine. J. Exp. Med., 204, 2779–2784

[19] Roukens, A. H.  and  Visser, L. G.  (2008) Yellow fever vaccine: past, present and future. Expert Opin. Biol. Ther., 8, 1787–1795

[20] Pulendran, B. (2009) Learning immunology from the yellow fever vaccine: innate immunity to systems vaccinology. Nat. Rev. Immunol., 9, 741–747 

[21] Pulendran, B., Li,  S. and Nakaya, H. I.  (2010) Systems vaccinology. Immunity, 33, 516–529

[22] Gaucher, D., Therrien,  R., Kettaf, N. ,  Angermann, B. R. ,  Boucher, G. ,  Filali-Mouhim, A. ,  Moser, J. M. ,  Mehta, R. S. ,  Drake, D. R. 3rd ,  Castro, E. ,  (2008) Yellow fever vaccine induces integrated multilineage and polyfunctional immune responses. J. Exp. Med., 205, 3119–3131

[23] Scherer, C. A. ,  Magness,  C. L. ,  Steiger, K. V. ,  Poitinger, N. D. ,  Caputo, C. M. ,  Miner, D. G. ,  Winokur, P. L. ,  Klinzman, D. ,  McKee, J. ,  Pilar, C. ,  (2007) Distinct gene expression profiles in peripheral blood mononuclear cells from patients infected with vaccinia virus, yellow fever 17D virus, or upper respiratory infections. Vaccine, 25, 6458–6473

[24] Smyth, G. K.Ritchie,  M., Thorne, N. , and  Wettenhall, J.  (2005) Limma: Linear Models for Microarray Data. In Statistics for Biology and Health. pp. 397–420. New York: Springer

[25] Subramanian, A., Tamayo,  P., Mootha, V. K., Mukherjee, S., Ebert,  B. L., Gillette,  M. A., Paulovich,  A., Pomeroy, S. L. ,  Golub, T. R. ,  Lander, E. S. ,  (2005) Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proc. Natl. Acad. Sci. USA, 102, 15545–15550

[26] Querec, T. D. ,  Akondy, R. S. , Lee, E. K. ,  Cao, W. ,  Nakaya, H. I. ,  Teuwen, D. ,  Pirani, A. ,  Gernert, K. ,  Deng, J. ,  Marzolf, B. ,  (2009) Systems biology approach predicts immunogenicity of the yellow fever vaccine in humans. Nat. Immunol., 10, 116–125

[27] Xiang, Z., Courtot,  M., Brinkman, R. R. ,  Ruttenberg, A.  and  He, Y. (2010) OntoFox: web-based support for ontology reuse. BMC Res. Notes, 3, 175160;

[28] Arp, R., Smith,  B. and Spear, A. D.  (2015) Building Ontologies With Basic Formal Ontology. Cambridge: MIT Press

[29] Musen, M. A. (2015) The protégé project: a look back and a look forward. AI Matters, 1, 4–12

[30] The protege ontology editor. 

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag GmbH Germany