##############################################################################
##############################################################################
###
### Running command:
###
###   F:\biocbuild\bbs-3.18-bioc\R\bin\R.exe CMD check --no-multiarch --install=check:goSorensen.install-out.txt --library=F:\biocbuild\bbs-3.18-bioc\R\library --no-vignettes --timings goSorensen_1.4.0.tar.gz
###
##############################################################################
##############################################################################


* using log directory 'F:/biocbuild/bbs-3.18-bioc/meat/goSorensen.Rcheck'
* using R version 4.3.3 (2024-02-29 ucrt)
* using platform: x86_64-w64-mingw32 (64-bit)
* R was compiled by
    gcc.exe (GCC) 12.3.0
    GNU Fortran (GCC) 12.3.0
* running under: Windows Server 2022 x64 (build 20348)
* using session charset: UTF-8
* using option '--no-vignettes'
* checking for file 'goSorensen/DESCRIPTION' ... OK
* checking extension type ... Package
* this is package 'goSorensen' version '1.4.0'
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking whether package 'goSorensen' can be installed ... OK
* checking installed package size ... OK
* checking package directory ... OK
* checking 'build' directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking R files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking contents of 'data' directory ... OK
* checking data for non-ASCII characters ... OK
* checking data for ASCII and uncompressed saves ... OK
* checking files in 'vignettes' ... OK
* checking examples ... OK
Examples with CPU (user + system) or elapsed time > 5s
                   user system elapsed
hclustThreshold  954.81  21.35  979.83
buildEnrichTable  17.93   1.89   20.74
* checking for unstated dependencies in 'tests' ... OK
* checking tests ...
  Running 'test_gosorensen_funcs.R'
  Running 'test_nonsense_genes.R'
 OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes in 'inst/doc' ... OK
* checking running R code from vignettes ... SKIPPED
* checking re-building of vignette outputs ... SKIPPED
* checking PDF version of manual ... OK
* DONE

Status: OK

##############################################################################
##############################################################################
###
### Running command:
###
###   F:\biocbuild\bbs-3.18-bioc\R\bin\R.exe CMD INSTALL goSorensen
###
##############################################################################
##############################################################################


* installing to library 'F:/biocbuild/bbs-3.18-bioc/R/library'
* installing *source* package 'goSorensen' ...
** using staged installation
** R
** data
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (goSorensen)

R version 4.3.3 (2024-02-29 ucrt) -- "Angel Food Cake"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(goSorensen)



Attaching package: 'goSorensen'

The following object is masked from 'package:utils':

    upgrade

> 
> # A contingency table of GO terms mutual enrichment
> # between gene lists "atlas" and "sanger":
> data(tab_atlas.sanger_BP3)
> tab_atlas.sanger_BP3
                 Enriched in sanger
Enriched in atlas TRUE FALSE
            TRUE    38    31
            FALSE    2   452
> ?tab_atlas.sanger_BP3
> class(tab_atlas.sanger_BP3)
[1] "table"
> 
> # Sorensen-Dice dissimilarity on this contingency table:
> ?dSorensen
> dSorensen(tab_atlas.sanger_BP3)
[1] 0.3027523
> 
> # Standard error of this Sorensen-Dice dissimilarity estimate:
> ?seSorensen
> seSorensen(tab_atlas.sanger_BP3)
[1] 0.05058655
> 
> # Upper 95% confidence limit for the Sorensen-Dice dissimilarity:
> ?duppSorensen
> duppSorensen(tab_atlas.sanger_BP3)
[1] 0.3859598
> # This confidence limit is based on an assimptotic normal N(0,1)
> # approximation to the distribution of (dSampl - d) / se, where
> # dSampl stands for the sample dissimilarity, d for the true dissimilarity
> # and se for the sample dissimilarity standard error estimate.
> 
> # Upper confidence limit but using a Student's t instead of a N(0,1)
> # (just as an example, not recommended -no theoretical justification)
> df <- sum(tab_atlas.sanger_BP3[1:3]) - 2
> duppSorensen(tab_atlas.sanger_BP3, z.conf.level = qt(1 - 0.95, df))
[1] 0.3870921
> 
> # Upper confidence limit but using a bootstrap approximation
> # to the sampling distribution, instead of a N(0,1)
> set.seed(123)
> duppSorensen(tab_atlas.sanger_BP3, boot = TRUE)
[1] 0.3941622
attr(,"eff.nboot")
[1] 10000
> 
> # Some computations on diverse data structures:
> badConti <- as.table(matrix(c(501, 27, 36, 12, 43, 15, 0, 0, 0),
+                             nrow = 3, ncol = 3,
+                             dimnames = list(c("a1","a2","a3"),
+                                             c("b1", "b2","b3"))))
> tryCatch(nice2x2Table(badConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(badConti): Not a 2x2 table>
> 
> incompleteConti <- badConti[1,1:min(2,ncol(badConti)), drop = FALSE]
> incompleteConti
    b1  b2
a1 501  12
> tryCatch(nice2x2Table(incompleteConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(incompleteConti): Not a 2x2 table>
> 
> contiAsVector <- c(32, 21, 81, 1439)
> nice2x2Table(contiAsVector)
[1] TRUE
> contiAsVector.mat <- matrix(contiAsVector, nrow = 2)
> contiAsVector.mat
     [,1] [,2]
[1,]   32   81
[2,]   21 1439
> contiAsVectorLen3 <- c(32, 21, 81)
> nice2x2Table(contiAsVectorLen3)
[1] TRUE
> 
> tryCatch(dSorensen(badConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> 
> # Apparently, the next order works fine, but returns a wrong value!
> dSorensen(badConti, check.table = FALSE)
[1] 0.05915493
> 
> tryCatch(dSorensen(incompleteConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> dSorensen(contiAsVector)
[1] 0.6144578
> dSorensen(contiAsVector.mat)
[1] 0.6144578
> dSorensen(contiAsVectorLen3)
[1] 0.6144578
> dSorensen(contiAsVectorLen3, check.table = FALSE)
[1] 0.6144578
> dSorensen(c(0,0,0,45))
[1] NaN
> 
> tryCatch(seSorensen(badConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> tryCatch(seSorensen(incompleteConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> seSorensen(contiAsVector)
[1] 0.04818012
> seSorensen(contiAsVector.mat)
[1] 0.04818012
> seSorensen(contiAsVectorLen3)
[1] 0.04818012
> seSorensen(contiAsVectorLen3, check.table = FALSE)
[1] 0.04818012
> tryCatch(seSorensen(contiAsVectorLen3, check.table = "not"), error = function(e) {return(e)})
<simpleError in seSorensen.numeric(contiAsVectorLen3, check.table = "not"): Argument 'check.table' must be logical>
> seSorensen(c(0,0,0,45))
[1] NaN
> 
> tryCatch(duppSorensen(badConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> tryCatch(duppSorensen(incompleteConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> duppSorensen(contiAsVector)
[1] 0.6937071
> duppSorensen(contiAsVector.mat)
[1] 0.6937071
> set.seed(123)
> duppSorensen(contiAsVector, boot = TRUE)
[1] 0.6922658
attr(,"eff.nboot")
[1] 10000
> set.seed(123)
> duppSorensen(contiAsVector.mat, boot = TRUE)
[1] 0.6922658
attr(,"eff.nboot")
[1] 10000
> duppSorensen(contiAsVectorLen3)
[1] 0.6937071
> # Bootstrapping requires full contingency tables (4 values)
> set.seed(123)
> tryCatch(duppSorensen(contiAsVectorLen3, boot = TRUE), error = function(e) {return(e)})
<simpleError in duppSorensen.numeric(contiAsVectorLen3, boot = TRUE): Bootstraping requires a numeric vector of 4 frequencies>
> duppSorensen(c(0,0,0,45))
[1] NaN
> 
> # Equivalence test, H0: d >= d0 vs  H1: d < d0 (d0 = 0.4444)
> ?equivTestSorensen
> equiv.atlas.sanger <- equivTestSorensen(tab_atlas.sanger_BP3)
> equiv.atlas.sanger

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab_atlas.sanger_BP3
(d - d0) / se = -2.801, p-value = 0.002547
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3859598
sample estimates:
Sorensen dissimilarity 
             0.3027523 
attr(,"se")
standard error 
    0.05058655 

> getTable(equiv.atlas.sanger)
                 Enriched in sanger
Enriched in atlas TRUE FALSE
            TRUE    38    31
            FALSE    2   452
> getPvalue(equiv.atlas.sanger)
    p-value 
0.002547349 
> 
> tryCatch(equivTestSorensen(badConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> tryCatch(equivTestSorensen(incompleteConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> equivTestSorensen(contiAsVector)

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  contiAsVector
(d - d0) / se = 3.5287, p-value = 0.9998
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.6937071
sample estimates:
Sorensen dissimilarity 
             0.6144578 
attr(,"se")
standard error 
    0.04818012 

> equivTestSorensen(contiAsVector.mat)

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  contiAsVector.mat
(d - d0) / se = 3.5287, p-value = 0.9998
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.6937071
sample estimates:
Sorensen dissimilarity 
             0.6144578 
attr(,"se")
standard error 
    0.04818012 

> set.seed(123)
> equivTestSorensen(contiAsVector.mat, boot = TRUE)

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  contiAsVector.mat
(d - d0) / se = 3.5287, p-value = 0.9996
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.6922658
sample estimates:
Sorensen dissimilarity 
             0.6144578 
attr(,"se")
standard error 
    0.04818012 

> equivTestSorensen(contiAsVectorLen3)

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  contiAsVectorLen3
(d - d0) / se = 3.5287, p-value = 0.9998
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.6937071
sample estimates:
Sorensen dissimilarity 
             0.6144578 
attr(,"se")
standard error 
    0.04818012 

> 
> tryCatch(equivTestSorensen(contiAsVectorLen3, boot = TRUE), error = function(e) {return(e)})
<simpleError in equivTestSorensen.numeric(contiAsVectorLen3, boot = TRUE): Bootstraping requires a numeric vector of 4 frequencies>
> 
> equivTestSorensen(c(0,0,0,45))

	No test performed due non finite (d - d0) / se statistic

data:  c(0, 0, 0, 45)
(d - d0) / se = NaN, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                   NaN 
attr(,"se")
standard error 
           NaN 

> 
> # Sorensen-Dice computations from scratch, directly from gene lists
> data(allOncoGeneLists)
> ?allOncoGeneLists
> data(humanEntrezIDs)
> # First, the mutual GO node enrichment tables are built, then computations
> # proceed from these contingency tables.
> # Building the contingency tables is a slow process (many enrichment tests)
> normTest <- equivTestSorensen(allOncoGeneLists[["atlas"]], allOncoGeneLists[["sanger"]],
+                               listNames = c("atlas", "sanger"),
+                               onto = "BP", GOLevel = 5,
+                               geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> normTest

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -7.3786, p-value = 8e-14
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3641617
sample estimates:
Sorensen dissimilarity 
             0.3411306 
attr(,"se")
standard error 
    0.01400189 

> 
> # To perform a bootstrap test from scratch would be even slower:
> # set.seed(123)
> # bootTest <- equivTestSorensen(allOncoGeneLists[["atlas"]], allOncoGeneLists[["sanger"]],
> #                               listNames = c("atlas", "sanger"),
> #                               boot = TRUE,
> #                               onto = "BP", GOLevel = 5,
> #                               geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # bootTest
> 
> # It is much faster to upgrade 'normTest' to be a bootstrap test:
> set.seed(123)
> bootTest <- upgrade(normTest, boot = TRUE)
> bootTest

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -7.3786, p-value = 9.999e-05
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3642245
sample estimates:
Sorensen dissimilarity 
             0.3411306 
attr(,"se")
standard error 
    0.01400189 

> # To know the number of planned bootstrap replicates:
> getNboot(bootTest)
[1] 10000
> # To know the number of valid bootstrap replicates:
> getEffNboot(bootTest)
[1] 10000
> 
> # There are similar methods for dSorensen, seSorensen, duppSorensen, etc. to
> # compute directly from a pair of gene lists.
> # They are quite slow for the same reason as before (many enrichment tests).
> # dSorensen(allOncoGeneLists[["atlas"]], allOncoGeneLists[["sanger"]],
> #           listNames = c("atlas", "sanger"),
> #           onto = "BP", GOLevel = 5,
> #           geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # seSorensen(allOncoGeneLists[["atlas"]], allOncoGeneLists[["sanger"]],
> #            listNames = c("atlas", "sanger"),
> #            onto = "BP", GOLevel = 5,
> #            geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> #
> # duppSorensen(allOncoGeneLists[["atlas"]], allOncoGeneLists[["sanger"]],
> #              listNames = c("atlas", "sanger"),
> #              onto = "BP", GOLevel = 5,
> #              geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> #
> # set.seed(123)
> # duppSorensen(allOncoGeneLists[["atlas"]], allOncoGeneLists[["sanger"]],
> #              boot = TRUE,
> #              listNames = c("atlas", "sanger"),
> #              onto = "BP", GOLevel = 5,
> #              geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # etc.
> 
> # To build the contingency table first and then compute from it, may be a more flexible
> # and saving time strategy, in general:
> ?buildEnrichTable
> tab <- buildEnrichTable(allOncoGeneLists[["atlas"]], allOncoGeneLists[["sanger"]],
+                         listNames = c("atlas", "sanger"),
+                         onto = "BP", GOLevel = 5,
+                         geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> 
> tab
                 Enriched in sanger
Enriched in atlas TRUE FALSE
            TRUE   507   480
            FALSE   45  9116
> 
> # (Here, an obvious faster possibility would be to recover the enrichment contingency
> # table from the previous normal test result:)
> tab <- getTable(normTest)
> tab
                 Enriched in sanger
Enriched in atlas TRUE FALSE
            TRUE   507   480
            FALSE   45  9116
> 
> tst <- equivTestSorensen(tab)
> tst

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -7.3786, p-value = 8e-14
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3641617
sample estimates:
Sorensen dissimilarity 
             0.3411306 
attr(,"se")
standard error 
    0.01400189 

> set.seed(123)
> bootTst <- equivTestSorensen(tab, boot = TRUE)
> bootTst

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -7.3786, p-value = 9.999e-05
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3642245
sample estimates:
Sorensen dissimilarity 
             0.3411306 
attr(,"se")
standard error 
    0.01400189 

> 
> dSorensen(tab)
[1] 0.3411306
> seSorensen(tab)
[1] 0.01400189
> # or:
> getDissimilarity(tst)
Sorensen dissimilarity 
             0.3411306 
attr(,"se")
standard error 
    0.01400189 
> 
> duppSorensen(tab)
[1] 0.3641617
> getUpper(tst)
   dUpper 
0.3641617 
> 
> set.seed(123)
> duppSorensen(tab, boot = TRUE)
[1] 0.3642245
attr(,"eff.nboot")
[1] 10000
> getUpper(bootTst)
   dUpper 
0.3642245 
> 
> # To perform from scratch all pairwise tests (or other Sorensen-Dice computations)
> # is even much slower. For example, all pairwise...
> # Dissimilarities:
> # # allPairDiss <- dSorensen(allOncoGeneLists,
> # #                          onto = "BP", GOLevel = 5,
> # #                          geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # # allPairDiss
> #
> # # Still time consuming but faster: build all tables computing in parallel:
> # allPairDiss <- dSorensen(allOncoGeneLists,
> #                          onto = "BP", GOLevel = 5,
> #                          geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db",
> #                          parallel = TRUE)
> # allPairDiss
> 
> # Standard errors:
> # seSorensen(allOncoGeneLists,
> #            onto = "BP", GOLevel = 5,
> #            geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> #
> # Upper confidence interval limits:
> # duppSorensen(allOncoGeneLists,
> #              onto = "BP", GOLevel = 5,
> #              geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # All pairwise asymptotic normal tests:
> # allTests <- equivTestSorensen(allOncoGeneLists,
> #                               onto = "BP", GOLevel = 5,
> #                               geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # getPvalue(allTests, simplify = FALSE)
> # getPvalue(allTests)
> # p.adjust(getPvalue(allTests), method = "holm")
> # To perform all pairwise bootstrap tests from scratch is (slightly)
> # even more time consuming:
> # set.seed(123)
> # allBootTests <- equivTestSorensen(allOncoGeneLists,
> #                                   boot = TRUE,
> #                                   onto = "BP", GOLevel = 5,
> #                                   geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # Not all bootstrap replicates may conduct to finite statistics:
> # getNboot(allBootTests)
> 
> # Given the normal tests (object 'allTests'), it is much faster to upgrade
> # it to have the bootstrap tests:
> # set.seed(123)
> # allBootTests <- upgrade(allTests, boot = TRUE)
> # getPvalue(allBootTests, simplify = FALSE)
> 
> # Again, the faster and more flexible possibility may be:
> # 1) First, build all pairwise enrichment contingency tables (slow first step):
> # allTabsBP.4 <- buildEnrichTable(allOncoGeneLists,
> #                                 onto = "BP", GOLevel = 5,
> #                                 geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # allTabsBP.4
> 
> # Better, directly use the dataset available at this package, goSorensen:
> data(allTabsBP.4)
> allTabsBP.4
$cangenes
$cangenes$atlas
                    Enriched in atlas
Enriched in cangenes TRUE FALSE
               TRUE     0     0
               FALSE  420  3383
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 4


$cis
$cis$atlas
               Enriched in atlas
Enriched in cis TRUE FALSE
          TRUE    80     3
          FALSE  340  3380

$cis$cangenes
               Enriched in cangenes
Enriched in cis TRUE FALSE
          TRUE     0    83
          FALSE    0  3720
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 4


$miscellaneous
$miscellaneous$atlas
                         Enriched in atlas
Enriched in miscellaneous TRUE FALSE
                    TRUE   198    21
                    FALSE  222  3362

$miscellaneous$cangenes
                         Enriched in cangenes
Enriched in miscellaneous TRUE FALSE
                    TRUE     0   219
                    FALSE    0  3584
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 4

$miscellaneous$cis
                         Enriched in cis
Enriched in miscellaneous TRUE FALSE
                    TRUE    70   149
                    FALSE   13  3571


$sanger
$sanger$atlas
                  Enriched in atlas
Enriched in sanger TRUE FALSE
             TRUE   209    24
             FALSE  211  3359

$sanger$cangenes
                  Enriched in cangenes
Enriched in sanger TRUE FALSE
             TRUE     0   233
             FALSE    0  3570
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 4

$sanger$cis
                  Enriched in cis
Enriched in sanger TRUE FALSE
             TRUE    68   165
             FALSE   15  3555

$sanger$miscellaneous
                  Enriched in miscellaneous
Enriched in sanger TRUE FALSE
             TRUE   151    82
             FALSE   68  3502


$Vogelstein
$Vogelstein$atlas
                      Enriched in atlas
Enriched in Vogelstein TRUE FALSE
                 TRUE   220    32
                 FALSE  200  3351

$Vogelstein$cangenes
                      Enriched in cangenes
Enriched in Vogelstein TRUE FALSE
                 TRUE     0   252
                 FALSE    0  3551
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 4

$Vogelstein$cis
                      Enriched in cis
Enriched in Vogelstein TRUE FALSE
                 TRUE    68   184
                 FALSE   15  3536

$Vogelstein$miscellaneous
                      Enriched in miscellaneous
Enriched in Vogelstein TRUE FALSE
                 TRUE   156    96
                 FALSE   63  3488

$Vogelstein$sanger
                      Enriched in sanger
Enriched in Vogelstein TRUE FALSE
                 TRUE   217    35
                 FALSE   16  3535


$waldman
$waldman$atlas
                   Enriched in atlas
Enriched in waldman TRUE FALSE
              TRUE   264    39
              FALSE  156  3344

$waldman$cangenes
                   Enriched in cangenes
Enriched in waldman TRUE FALSE
              TRUE     0   303
              FALSE    0  3500
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 4

$waldman$cis
                   Enriched in cis
Enriched in waldman TRUE FALSE
              TRUE    77   226
              FALSE    6  3494

$waldman$miscellaneous
                   Enriched in miscellaneous
Enriched in waldman TRUE FALSE
              TRUE   203   100
              FALSE   16  3484

$waldman$sanger
                   Enriched in sanger
Enriched in waldman TRUE FALSE
              TRUE   181   122
              FALSE   52  3448

$waldman$Vogelstein
                   Enriched in Vogelstein
Enriched in waldman TRUE FALSE
              TRUE   192   111
              FALSE   60  3440


attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 4
attr(,"class")
[1] "tableList" "list"     
> class(allTabsBP.4)
[1] "tableList" "list"     
> # 2) Then perform all required computatios from these enrichment contingency tables...
> # All pairwise tests:
> allTests <- equivTestSorensen(allTabsBP.4)
> allTests
$cangenes
$cangenes$atlas

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 



$cis
$cis$atlas

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = 8.807, p-value = 1
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.7262589
sample estimates:
Sorensen dissimilarity 
             0.6819085 
attr(,"se")
standard error 
    0.02696312 


$cis$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 



$miscellaneous
$miscellaneous$atlas

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -2.8406, p-value = 0.002252
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4174355
sample estimates:
Sorensen dissimilarity 
             0.3802817 
attr(,"se")
standard error 
    0.02258792 


$miscellaneous$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$miscellaneous$cis

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = 2.5804, p-value = 0.9951
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.5950555
sample estimates:
Sorensen dissimilarity 
             0.5364238 
attr(,"se")
standard error 
    0.03564549 



$sanger
$sanger$atlas

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -3.8566, p-value = 5.748e-05
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3959452
sample estimates:
Sorensen dissimilarity 
             0.3598775 
attr(,"se")
standard error 
    0.02192764 


$sanger$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$sanger$cis

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = 3.5799, p-value = 0.9998
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.6271347
sample estimates:
Sorensen dissimilarity 
             0.5696203 
attr(,"se")
standard error 
    0.03496631 


$sanger$miscellaneous

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -4.3974, p-value = 5.479e-06
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3739718
sample estimates:
Sorensen dissimilarity 
             0.3318584 
attr(,"se")
standard error 
    0.02560313 



$Vogelstein
$Vogelstein$atlas

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -4.6585, p-value = 1.593e-06
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3802668
sample estimates:
Sorensen dissimilarity 
             0.3452381 
attr(,"se")
standard error 
    0.02129595 


$Vogelstein$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$Vogelstein$cis

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = 4.4076, p-value = 1
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.6498536
sample estimates:
Sorensen dissimilarity 
             0.5940299 
attr(,"se")
standard error 
    0.03393844 


$Vogelstein$miscellaneous

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -4.2339, p-value = 1.148e-05
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3790962
sample estimates:
Sorensen dissimilarity 
             0.3375796 
attr(,"se")
standard error 
    0.02524032 


$Vogelstein$sanger

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -23.128, p-value < 2.2e-16
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.1292852
sample estimates:
Sorensen dissimilarity 
             0.1051546 
attr(,"se")
standard error 
    0.01467036 



$waldman
$waldman$atlas

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -9.3848, p-value < 2.2e-16
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3003348
sample estimates:
Sorensen dissimilarity 
             0.2697095 
attr(,"se")
standard error 
    0.01861884 


$waldman$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$waldman$cis

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = 4.9573, p-value = 1
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.6529946
sample estimates:
Sorensen dissimilarity 
             0.6010363 
attr(,"se")
standard error 
    0.03158842 


$waldman$miscellaneous

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -11.029, p-value < 2.2e-16
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.2553636
sample estimates:
Sorensen dissimilarity 
             0.2222222 
attr(,"se")
standard error 
    0.02014852 


$waldman$sanger

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -5.1402, p-value = 1.372e-07
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3629683
sample estimates:
Sorensen dissimilarity 
             0.3246269 
attr(,"se")
standard error 
    0.02330993 


$waldman$Vogelstein

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -6.0739, p-value = 6.243e-10
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.000000 0.345029
sample estimates:
Sorensen dissimilarity 
             0.3081081 
attr(,"se")
standard error 
    0.02244631 



attr(,"class")
[1] "equivSDhtestList" "list"            
> class(allTests)
[1] "equivSDhtestList" "list"            
> set.seed(123)
> allBootTests <- equivTestSorensen(allTabsBP.4, boot = TRUE)
> allBootTests
$cangenes
$cangenes$atlas

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 



$cis
$cis$atlas

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = 8.807, p-value = 1
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.000000 0.725535
sample estimates:
Sorensen dissimilarity 
             0.6819085 
attr(,"se")
standard error 
    0.02696312 


$cis$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 



$miscellaneous
$miscellaneous$atlas

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -2.8406, p-value = 0.004
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.000000 0.418077
sample estimates:
Sorensen dissimilarity 
             0.3802817 
attr(,"se")
standard error 
    0.02258792 


$miscellaneous$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$miscellaneous$cis

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = 2.5804, p-value = 0.9933
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.000000 0.595412
sample estimates:
Sorensen dissimilarity 
             0.5364238 
attr(,"se")
standard error 
    0.03564549 



$sanger
$sanger$atlas

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -3.8566, p-value = 3e-04
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3960626
sample estimates:
Sorensen dissimilarity 
             0.3598775 
attr(,"se")
standard error 
    0.02192764 


$sanger$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$sanger$cis

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = 3.5799, p-value = 0.9996
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.6278561
sample estimates:
Sorensen dissimilarity 
             0.5696203 
attr(,"se")
standard error 
    0.03496631 


$sanger$miscellaneous

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -4.3974, p-value = 2e-04
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3765829
sample estimates:
Sorensen dissimilarity 
             0.3318584 
attr(,"se")
standard error 
    0.02560313 



$Vogelstein
$Vogelstein$atlas

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -4.6585, p-value = 2e-04
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3809169
sample estimates:
Sorensen dissimilarity 
             0.3452381 
attr(,"se")
standard error 
    0.02129595 


$Vogelstein$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$Vogelstein$cis

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = 4.4076, p-value = 1
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.6489965
sample estimates:
Sorensen dissimilarity 
             0.5940299 
attr(,"se")
standard error 
    0.03393844 


$Vogelstein$miscellaneous

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -4.2339, p-value = 9.999e-05
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3796934
sample estimates:
Sorensen dissimilarity 
             0.3375796 
attr(,"se")
standard error 
    0.02524032 


$Vogelstein$sanger

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -23.128, p-value = 9.999e-05
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.1312585
sample estimates:
Sorensen dissimilarity 
             0.1051546 
attr(,"se")
standard error 
    0.01467036 



$waldman
$waldman$atlas

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -9.3848, p-value = 9.999e-05
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3006583
sample estimates:
Sorensen dissimilarity 
             0.2697095 
attr(,"se")
standard error 
    0.01861884 


$waldman$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$waldman$cis

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = 4.9573, p-value = 1
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.6525683
sample estimates:
Sorensen dissimilarity 
             0.6010363 
attr(,"se")
standard error 
    0.03158842 


$waldman$miscellaneous

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -11.029, p-value = 9.999e-05
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.2577849
sample estimates:
Sorensen dissimilarity 
             0.2222222 
attr(,"se")
standard error 
    0.02014852 


$waldman$sanger

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -5.1402, p-value = 9.999e-05
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3639666
sample estimates:
Sorensen dissimilarity 
             0.3246269 
attr(,"se")
standard error 
    0.02330993 


$waldman$Vogelstein

	Bootstrap test for 2x2 contingency tables based on the Sorensen-Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -6.0739, p-value = 9.999e-05
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3470915
sample estimates:
Sorensen dissimilarity 
             0.3081081 
attr(,"se")
standard error 
    0.02244631 



attr(,"class")
[1] "equivSDhtestList" "list"            
> class(allBootTests)
[1] "equivSDhtestList" "list"            
> getPvalue(allBootTests, simplify = FALSE)
                   atlas cangenes       cis miscellaneous     sanger Vogelstein
atlas         0.00000000      NaN 1.0000000    0.00399960 0.00029997 0.00019998
cangenes             NaN        0       NaN           NaN        NaN        NaN
cis           1.00000000      NaN 0.0000000    0.99330067 0.99960004 1.00000000
miscellaneous 0.00399960      NaN 0.9933007    0.00000000 0.00019998 0.00009999
sanger        0.00029997      NaN 0.9996000    0.00019998 0.00000000 0.00009999
Vogelstein    0.00019998      NaN 1.0000000    0.00009999 0.00009999 0.00000000
waldman       0.00009999      NaN 1.0000000    0.00009999 0.00009999 0.00009999
                waldman
atlas         9.999e-05
cangenes            NaN
cis           1.000e+00
miscellaneous 9.999e-05
sanger        9.999e-05
Vogelstein    9.999e-05
waldman       0.000e+00
> getEffNboot(allBootTests)
          cangenes.atlas                cis.atlas             cis.cangenes 
                     NaN                    10000                      NaN 
     miscellaneous.atlas   miscellaneous.cangenes        miscellaneous.cis 
                   10000                      NaN                    10000 
            sanger.atlas          sanger.cangenes               sanger.cis 
                   10000                      NaN                    10000 
    sanger.miscellaneous         Vogelstein.atlas      Vogelstein.cangenes 
                   10000                    10000                      NaN 
          Vogelstein.cis Vogelstein.miscellaneous        Vogelstein.sanger 
                   10000                    10000                    10000 
           waldman.atlas         waldman.cangenes              waldman.cis 
                   10000                      NaN                    10000 
   waldman.miscellaneous           waldman.sanger       waldman.Vogelstein 
                   10000                    10000                    10000 
> 
> # To adjust for testing multiplicity:
> p.adjust(getPvalue(allBootTests), method = "holm")
          cangenes.atlas.p-value                cis.atlas.p-value 
                             NaN                       1.00000000 
            cis.cangenes.p-value      miscellaneous.atlas.p-value 
                             NaN                       0.02399760 
  miscellaneous.cangenes.p-value        miscellaneous.cis.p-value 
                             NaN                       1.00000000 
            sanger.atlas.p-value          sanger.cangenes.p-value 
                      0.00209979                              NaN 
              sanger.cis.p-value     sanger.miscellaneous.p-value 
                      1.00000000                       0.00179982 
        Vogelstein.atlas.p-value      Vogelstein.cangenes.p-value 
                      0.00179982                              NaN 
          Vogelstein.cis.p-value Vogelstein.miscellaneous.p-value 
                      1.00000000                       0.00149985 
       Vogelstein.sanger.p-value            waldman.atlas.p-value 
                      0.00149985                       0.00149985 
        waldman.cangenes.p-value              waldman.cis.p-value 
                             NaN                       1.00000000 
   waldman.miscellaneous.p-value           waldman.sanger.p-value 
                      0.00149985                       0.00149985 
      waldman.Vogelstein.p-value 
                      0.00149985 
> 
> # If only partial statistics are desired:
> dSorensen(allTabsBP.4)
                  atlas cangenes       cis miscellaneous    sanger Vogelstein
atlas         0.0000000        1 0.6819085     0.3802817 0.3598775  0.3452381
cangenes      1.0000000        0 1.0000000     1.0000000 1.0000000  1.0000000
cis           0.6819085        1 0.0000000     0.5364238 0.5696203  0.5940299
miscellaneous 0.3802817        1 0.5364238     0.0000000 0.3318584  0.3375796
sanger        0.3598775        1 0.5696203     0.3318584 0.0000000  0.1051546
Vogelstein    0.3452381        1 0.5940299     0.3375796 0.1051546  0.0000000
waldman       0.2697095        1 0.6010363     0.2222222 0.3246269  0.3081081
                waldman
atlas         0.2697095
cangenes      1.0000000
cis           0.6010363
miscellaneous 0.2222222
sanger        0.3246269
Vogelstein    0.3081081
waldman       0.0000000
> duppSorensen(allTabsBP.4)
                  atlas cangenes       cis miscellaneous    sanger Vogelstein
atlas         0.0000000      NaN 0.7262589     0.4174355 0.3959452  0.3802668
cangenes            NaN        0       NaN           NaN       NaN        NaN
cis           0.7262589      NaN 0.0000000     0.5950555 0.6271347  0.6498536
miscellaneous 0.4174355      NaN 0.5950555     0.0000000 0.3739718  0.3790962
sanger        0.3959452      NaN 0.6271347     0.3739718 0.0000000  0.1292852
Vogelstein    0.3802668      NaN 0.6498536     0.3790962 0.1292852  0.0000000
waldman       0.3003348      NaN 0.6529946     0.2553636 0.3629683  0.3450290
                waldman
atlas         0.3003348
cangenes            NaN
cis           0.6529946
miscellaneous 0.2553636
sanger        0.3629683
Vogelstein    0.3450290
waldman       0.0000000
> seSorensen(allTabsBP.4)
                   atlas cangenes        cis miscellaneous     sanger
atlas         0.00000000        0 0.02696312    0.02258792 0.02192764
cangenes      0.00000000        0 0.00000000    0.00000000 0.00000000
cis           0.02696312        0 0.00000000    0.03564549 0.03496631
miscellaneous 0.02258792        0 0.03564549    0.00000000 0.02560313
sanger        0.02192764        0 0.03496631    0.02560313 0.00000000
Vogelstein    0.02129595        0 0.03393844    0.02524032 0.01467036
waldman       0.01861884        0 0.03158842    0.02014852 0.02330993
              Vogelstein    waldman
atlas         0.02129595 0.01861884
cangenes      0.00000000 0.00000000
cis           0.03393844 0.03158842
miscellaneous 0.02524032 0.02014852
sanger        0.01467036 0.02330993
Vogelstein    0.00000000 0.02244631
waldman       0.02244631 0.00000000
> 
> 
> # Tipically, in a real study it would be interesting to scan tests
> # along some ontologies and levels inside these ontologies:
> # (which obviously will be a quite slow process)
> # gc()
> # set.seed(123)
> # allBootTests_BP_MF_lev4to8 <- allEquivTestSorensen(allOncoGeneLists,
> #                                                    boot = TRUE,
> #                                                    geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db",
> #                                                    ontos = c("BP", "MF"), GOLevels = 4:8)
> # getPvalue(allBootTests_BP_MF_lev4to8)
> # getEffNboot(allBootTests_BP_MF_lev4to8)
> 
> proc.time()
   user  system elapsed 
 137.10    8.12  146.64 

R version 4.3.3 (2024-02-29 ucrt) -- "Angel Food Cake"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(goSorensen)



Attaching package: 'goSorensen'

The following object is masked from 'package:utils':

    upgrade

> 
> testError <- function(e) {return(e)}
> 
> tryCatch(dSorensen("Sec1", onto = "BP"), error = testError)
<simpleError in buildEnrichTable.character(x, y, check.table = check.table, ...): Argument 'y' is missing, 'x' and 'y' must be 'character' vectors of valid gene identifiers>
> 
> data(allOncoGeneLists)
> ?allOncoGeneLists
> data(humanEntrezIDs)
> 
> # Non-sense random gene lists. Generating Entrez-like gene identifiers, but random:
> set.seed(1234567)
> genList1 <- unique(as.character(sample.int(99999, size = 100)))
> genList2 <- unique(as.character(sample.int(99999, size = 100)))
> # Gene identifiers are numbers like Entrez identifiers at 'humanEntrezIDs', but random.
> dSorensen(genList1, genList2,
+           listNames = c("genList1", "genList2"),
+           onto = "BP", GOLevel = 5,
+           geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
[1] NaN
> duppSorensen(genList1, genList2,
+              listNames = c("genList1", "genList2"),
+              onto = "BP", GOLevel = 5,
+              geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
[1] NaN
> seSorensen(genList1, genList2,
+            listNames = c("genList1", "genList2"),
+            onto = "BP", GOLevel = 5,
+            geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
[1] NaN
> nonSenseTst <- equivTestSorensen(genList1, genList2,
+                                  listNames = c("genList1", "genList2"),
+                                  onto = "BP", GOLevel = 5,
+                                  geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> nonSenseTst

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = NaN, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                   NaN 
attr(,"se")
standard error 
           NaN 

> tab <- getTable(nonSenseTst)
> tab
                    Enriched in genList2
Enriched in genList1 TRUE FALSE
               TRUE     0     0
               FALSE    0 10148
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 5
> # Or, alternatively:
> tab <- buildEnrichTable(genList1, genList2,
+                         listNames = c("genList1", "genList2"),
+                         onto = "BP", GOLevel = 5,
+                         geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> tab
                    Enriched in genList2
Enriched in genList1 TRUE FALSE
               TRUE     0     0
               FALSE    0 10148
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 5
> dSorensen(tab)
[1] NaN
> duppSorensen(tab)
[1] NaN
> seSorensen(tab)
[1] NaN
> equivTestSorensen(tab)

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = NaN, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                   NaN 
attr(,"se")
standard error 
           NaN 

> 
> # Even more non-sense, letters non numeric-style like those at 'humanEntrezIDs':
> set.seed(1234567)
> genList1 <- unique(vapply(seq_len(100), function(i) {
+   paste0(sample(c(letters, LETTERS), 6, replace = TRUE), collapse = "")
+ }, FUN.VALUE = character(1)))
> genList2 <- unique(vapply(seq_len(100), function(i) {
+   paste0(sample(c(letters, LETTERS), 6, replace = TRUE), collapse = "")
+ }, FUN.VALUE = character(1)))
> 
> # Gene identifiers incompatible with those at 'humanEntrezIDs':
> dSorensen(genList1, genList2,
+           listNames = c("genList1", "genList2"),
+           onto = "BP", GOLevel = 5,
+           geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
--> No gene can be mapped....
--> Expected input gene ID: 3480,8626,200162,2707,253943,56339
--> return NULL...
--> No gene can be mapped....
--> Expected input gene ID: 54937,124783,3485,285588,84221,9232
--> return NULL...
[1] NaN
> duppSorensen(genList1, genList2,
+              listNames = c("genList1", "genList2"),
+              onto = "BP", GOLevel = 5,
+              geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
--> No gene can be mapped....
--> Expected input gene ID: 100506013,11144,171484,836,51207,338773
--> return NULL...
--> No gene can be mapped....
--> Expected input gene ID: 84132,9918,51804,5892,85376,83700
--> return NULL...
[1] NaN
> seSorensen(genList1, genList2,
+            listNames = c("genList1", "genList2"),
+            onto = "BP", GOLevel = 5,
+            geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
--> No gene can be mapped....
--> Expected input gene ID: 642636,146845,23626,5932,5016,6152
--> return NULL...
--> No gene can be mapped....
--> Expected input gene ID: 120935,1364,378807,50487,57082,91746
--> return NULL...
[1] NaN
> nonSenseTst <- equivTestSorensen(genList1, genList2,
+                                  listNames = c("genList1", "genList2"),
+                                  onto = "BP", GOLevel = 5,
+                                  geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
--> No gene can be mapped....
--> Expected input gene ID: 6790,2295,4487,25,2182,8743
--> return NULL...
--> No gene can be mapped....
--> Expected input gene ID: 226,51087,5887,23542,30009,140894
--> return NULL...
> nonSenseTst

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = NaN, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                   NaN 
attr(,"se")
standard error 
           NaN 

> tab <- getTable(nonSenseTst)
> tab
                    Enriched in genList2
Enriched in genList1 TRUE FALSE
               TRUE     0     0
               FALSE    0 10148
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 5
> # Or, alternatively:
> tab <- buildEnrichTable(genList1, genList2,
+                         listNames = c("genList1", "genList2"),
+                         onto = "BP", GOLevel = 5,
+                         geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
--> No gene can be mapped....
--> Expected input gene ID: 7356,596,84168,23236,23627,91
--> return NULL...
--> No gene can be mapped....
--> Expected input gene ID: 480,285643,49,124912,375341,92667
--> return NULL...
> tab
                    Enriched in genList2
Enriched in genList1 TRUE FALSE
               TRUE     0     0
               FALSE    0 10148
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 5
> dSorensen(tab)
[1] NaN
> duppSorensen(tab)
[1] NaN
> seSorensen(tab)
[1] NaN
> equivTestSorensen(tab)

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = NaN, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                   NaN 
attr(,"se")
standard error 
           NaN 

> 
> proc.time()
   user  system elapsed 
 249.00    7.96  257.84