Bioconductor for Everyone: Exploring, Analyzing, and Visualizing Large Data Sets with R

Martin Morgan1

1Roswell Park Comprehensive Cancer Center

11 April 2019

Abstract

We’ll take a fast-paced tour through R and the software project I work on, Bioconductor (https://bioconductor.org), learning how to explore and visualize large cancer-related data sets. We’ll work through two particular analyses. In the process, we’ll learn some pretty significant new R skills. For instance, we will learn about formal classes for representing complex data, strategies for iteration and parallel processing, and accessing ‘remote’ resources accessible through web-based interfaces. This workshop should be interesting to people who know a bit of R, and want to learn more!

1 Biology
2 Large data
- 2.1 What and how
- 2.2 Strategies
3 Real-world: objects
- 3.1 Objects
4 Tidy data and the tidyverse
5 Data base
6 Other on-disk or remote representations
7 End matter
- 7.1 Acknowledgements
- Session info

1 Biology

Gene expression

RNA-seq

Single-cell technologies

2 Large data

2.1 What and how

What is it?

Doesn’t fit in memory
Computationally expensive to process

Common use cases

Query – in this haystack, where is my needle
Processing – e.g., reduction to summary statistics

2.2 Strategies

Queries

e.g., of data bases

Processing

Iteration & chunk-wise
Distributed / parallel

3 Real-world: objects

3.1 Objects

We’ll set up some data to use (pay no attention to this!)

dir <- tempdir()
if (!dir.exists(dir))
    dir.create(dir)
library(airway)
data(airway)
write.table(colData(airway), file.path(dir, "samples.csv"))
write.table(assay(airway), file.path(dir, "expression.csv"))

Related data elements

samples <- read.table(file.path(dir, "samples.csv"))
samples

##            SampleName    cell   dex albut        Run avgLength Experiment
## SRR1039508 GSM1275862  N61311 untrt untrt SRR1039508       126  SRX384345
## SRR1039509 GSM1275863  N61311   trt untrt SRR1039509       126  SRX384346
## SRR1039512 GSM1275866 N052611 untrt untrt SRR1039512       126  SRX384349
## SRR1039513 GSM1275867 N052611   trt untrt SRR1039513        87  SRX384350
## SRR1039516 GSM1275870 N080611 untrt untrt SRR1039516       120  SRX384353
## SRR1039517 GSM1275871 N080611   trt untrt SRR1039517       126  SRX384354
## SRR1039520 GSM1275874 N061011 untrt untrt SRR1039520       101  SRX384357
## SRR1039521 GSM1275875 N061011   trt untrt SRR1039521        98  SRX384358
##               Sample    BioSample
## SRR1039508 SRS508568 SAMN02422669
## SRR1039509 SRS508567 SAMN02422675
## SRR1039512 SRS508571 SAMN02422678
## SRR1039513 SRS508572 SAMN02422670
## SRR1039516 SRS508575 SAMN02422682
## SRR1039517 SRS508576 SAMN02422673
## SRR1039520 SRS508579 SAMN02422683
## SRR1039521 SRS508580 SAMN02422677

counts <- read.table(file.path(dir, "expression.csv"))
head(counts)

##                 SRR1039508 SRR1039509 SRR1039512 SRR1039513 SRR1039516
## ENSG00000000003        679        448        873        408       1138
## ENSG00000000005          0          0          0          0          0
## ENSG00000000419        467        515        621        365        587
## ENSG00000000457        260        211        263        164        245
## ENSG00000000460         60         55         40         35         78
## ENSG00000000938          0          0          2          0          1
##                 SRR1039517 SRR1039520 SRR1039521
## ENSG00000000003       1047        770        572
## ENSG00000000005          0          0          0
## ENSG00000000419        799        417        508
## ENSG00000000457        331        233        229
## ENSG00000000460         63         76         60
## ENSG00000000938          0          0          0

counts <- as.matrix(counts)

Coordinate data management

separately ‘managing’ samples and counts is error-prone, e.g. subset samples but not counts, and the association of sample rows and count columns is distrupted.

library(SummarizedExperiment)
SummarizedExperiment()

## class: SummarizedExperiment 
## dim: 0 0 
## metadata(0):
## assays(0):
## rownames: NULL
## rowData names(0):
## colnames: NULL
## colData names(0):

An ‘S4’ object for coordinating ‘assay’ matrices with row and column annotations

se <- SummarizedExperiment(
    assays = list(counts = counts),
    colData = samples
)
se

## class: SummarizedExperiment 
## dim: 64102 8 
## metadata(0):
## assays(1): counts
## rownames(64102): ENSG00000000003 ENSG00000000005 ... LRG_98 LRG_99
## rowData names(0):
## colnames(8): SRR1039508 SRR1039509 ... SRR1039520 SRR1039521
## colData names(9): SampleName cell ... Sample BioSample

Classes allow the developer to make data access easy

e.g., matrix-like ‘interface’
Accessors, so internal representation can be chosen for efficiency while user interface remains easy to use

se$dex

## [1] untrt trt   untrt trt   untrt trt   untrt trt  
## Levels: trt untrt

se[, se$dex == "trt"]

## class: SummarizedExperiment 
## dim: 64102 4 
## metadata(0):
## assays(1): counts
## rownames(64102): ENSG00000000003 ENSG00000000005 ... LRG_98 LRG_99
## rowData names(0):
## colnames(4): SRR1039509 SRR1039513 SRR1039517 SRR1039521
## colData names(9): SampleName cell ... Sample BioSample

Data manipulation, e.g., non-zero rows

idx <- rowSums(assay(se)) > 0
se[idx,]

## class: SummarizedExperiment 
## dim: 33469 8 
## metadata(0):
## assays(1): counts
## rownames(33469): ENSG00000000003 ENSG00000000419 ... ENSG00000273492
##   ENSG00000273493
## rowData names(0):
## colnames(8): SRR1039508 SRR1039509 ... SRR1039520 SRR1039521
## colData names(9): SampleName cell ... Sample BioSample

Simple visualize

dotchart(colSums(assay(se)), xlab = "Library size")

expr <- rowSums(assay(se))
plot(density(log(expr[expr > 0])), ylab = "log expression")

4 Tidy data and the tidyverse

Not ‘better’, but different

Challenges in base R and formal objects

Many different types – vector, data.frame, matrix, SummarizedExperiment – all with different operations
Each function is somehow different, e.g., [ applied to a matrix usually (!) returns a matrix, whereas rowSums() returns a vector

‘tidy’ analysis

Consistent representation of data – data.frame
Consistent methods
- First argument is always ‘the data’,
- Tidy functions are always ‘endomorphisms’ – the class of the input data is the same as the class of the result
- Only a few standard ‘verbs’ – filter(), select(), group_by(), count(), summarize(), mutate(), …

library(dplyr)
library(tibble)

The pipe, %>%

Base R: reasoning ‘inside out’

x <- runif(10, 1, 5)
log(ceiling(x))

##  [1] 1.386294 1.386294 1.386294 1.609438 1.098612 1.098612 1.609438 1.098612
##  [9] 1.386294 1.098612

Procedural R
```
x1 <- ceiling(x)
log(x1)
```
Tidy R – use %>% so that operations read left-to-right
```
x %>% ceiling() %>% log()
```

Data representation

A tibble is like a user-friendly data.frame

as_tibble(mtcars)

## # A tibble: 32 x 11
##      mpg   cyl  disp    hp  drat    wt  qsec    vs    am  gear  carb
##    <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
##  1  21       6  160    110  3.9   2.62  16.5     0     1     4     4
##  2  21       6  160    110  3.9   2.88  17.0     0     1     4     4
##  3  22.8     4  108     93  3.85  2.32  18.6     1     1     4     1
##  4  21.4     6  258    110  3.08  3.22  19.4     1     0     3     1
##  5  18.7     8  360    175  3.15  3.44  17.0     0     0     3     2
##  6  18.1     6  225    105  2.76  3.46  20.2     1     0     3     1
##  7  14.3     8  360    245  3.21  3.57  15.8     0     0     3     4
##  8  24.4     4  147.    62  3.69  3.19  20       1     0     4     2
##  9  22.8     4  141.    95  3.92  3.15  22.9     1     0     4     2
## 10  19.2     6  168.   123  3.92  3.44  18.3     1     0     4     4
## # … with 22 more rows

Row names are just another column

sample <- tibble::rownames_to_column(
    as.data.frame(colData(se)),
    var="Accession"
) %>% as_tibble()

Data is represented in ‘long-form’

count <- reshape::melt(assay(se), c("Feature", "Accession")) %>%
    as_tibble()
colnames(count)[3] = "Count"
count

## # A tibble: 512,816 x 3
##    Feature         Accession  Count
##    <fct>           <fct>      <int>
##  1 ENSG00000000003 SRR1039508   679
##  2 ENSG00000000005 SRR1039508     0
##  3 ENSG00000000419 SRR1039508   467
##  4 ENSG00000000457 SRR1039508   260
##  5 ENSG00000000460 SRR1039508    60
##  6 ENSG00000000938 SRR1039508     0
##  7 ENSG00000000971 SRR1039508  3251
##  8 ENSG00000001036 SRR1039508  1433
##  9 ENSG00000001084 SRR1039508   519
## 10 ENSG00000001167 SRR1039508   394
## # … with 512,806 more rows

Endomorphism: tibble in, tibble out

cars <- rownames_to_column(mtcars, "make") %>%
    as_tibble()
cars %>% filter(cyl >= 6)

## # A tibble: 21 x 12
##    make       mpg   cyl  disp    hp  drat    wt  qsec    vs    am  gear  carb
##    <chr>    <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
##  1 Mazda R…  21       6  160    110  3.9   2.62  16.5     0     1     4     4
##  2 Mazda R…  21       6  160    110  3.9   2.88  17.0     0     1     4     4
##  3 Hornet …  21.4     6  258    110  3.08  3.22  19.4     1     0     3     1
##  4 Hornet …  18.7     8  360    175  3.15  3.44  17.0     0     0     3     2
##  5 Valiant   18.1     6  225    105  2.76  3.46  20.2     1     0     3     1
##  6 Duster …  14.3     8  360    245  3.21  3.57  15.8     0     0     3     4
##  7 Merc 280  19.2     6  168.   123  3.92  3.44  18.3     1     0     4     4
##  8 Merc 28…  17.8     6  168.   123  3.92  3.44  18.9     1     0     4     4
##  9 Merc 45…  16.4     8  276.   180  3.07  4.07  17.4     0     0     3     3
## 10 Merc 45…  17.3     8  276.   180  3.07  3.73  17.6     0     0     3     3
## # … with 11 more rows

cars %>% select(make, mpg, cyl, disp)

## # A tibble: 32 x 4
##    make                mpg   cyl  disp
##    <chr>             <dbl> <dbl> <dbl>
##  1 Mazda RX4          21       6  160 
##  2 Mazda RX4 Wag      21       6  160 
##  3 Datsun 710         22.8     4  108 
##  4 Hornet 4 Drive     21.4     6  258 
##  5 Hornet Sportabout  18.7     8  360 
##  6 Valiant            18.1     6  225 
##  7 Duster 360         14.3     8  360 
##  8 Merc 240D          24.4     4  147.
##  9 Merc 230           22.8     4  141.
## 10 Merc 280           19.2     6  168.
## # … with 22 more rows

Generally, playing well with other packages in the ‘tidyverse’

library(ggplot2)

cars %>% ggplot(aes(x = factor(cyl), y = mpg)) + geom_boxplot()

5 Data base

5.1 Data base representation

Create a sqlite data base

library(RSQLite)
airway_db <- file.path(dir, "airway.sqlite")
con <- dbConnect(SQLite(), airway_db)

Add a sample table

sample <- tibble::rownames_to_column(
    as.data.frame(colData(se)),
    var="Accession"
)
dbWriteTable(con, "sample", sample)

Add a count table, in ‘tidy’ form

count <- reshape::melt(assay(se), c("Feature", "Accession"))
colnames(count)[3] = "Count"
dbWriteTable(con, "count", count)

Extract data using SQL statements

dbListTables(con)

## [1] "count"  "sample"

dbGetQuery(con, "SELECT * FROM Sample;")

##    Accession SampleName    cell   dex albut        Run avgLength Experiment
## 1 SRR1039508 GSM1275862  N61311 untrt untrt SRR1039508       126  SRX384345
## 2 SRR1039509 GSM1275863  N61311   trt untrt SRR1039509       126  SRX384346
## 3 SRR1039512 GSM1275866 N052611 untrt untrt SRR1039512       126  SRX384349
## 4 SRR1039513 GSM1275867 N052611   trt untrt SRR1039513        87  SRX384350
## 5 SRR1039516 GSM1275870 N080611 untrt untrt SRR1039516       120  SRX384353
## 6 SRR1039517 GSM1275871 N080611   trt untrt SRR1039517       126  SRX384354
## 7 SRR1039520 GSM1275874 N061011 untrt untrt SRR1039520       101  SRX384357
## 8 SRR1039521 GSM1275875 N061011   trt untrt SRR1039521        98  SRX384358
##      Sample    BioSample
## 1 SRS508568 SAMN02422669
## 2 SRS508567 SAMN02422675
## 3 SRS508571 SAMN02422678
## 4 SRS508572 SAMN02422670
## 5 SRS508575 SAMN02422682
## 6 SRS508576 SAMN02422673
## 7 SRS508579 SAMN02422683
## 8 SRS508580 SAMN02422677

dbGetQuery(con, "SELECT Accession, cell, dex FROM Sample;")

##    Accession    cell   dex
## 1 SRR1039508  N61311 untrt
## 2 SRR1039509  N61311   trt
## 3 SRR1039512 N052611 untrt
## 4 SRR1039513 N052611   trt
## 5 SRR1039516 N080611 untrt
## 6 SRR1039517 N080611   trt
## 7 SRR1039520 N061011 untrt
## 8 SRR1039521 N061011   trt

dbGetQuery(con, "SELECT * FROM Count LIMIT 3;")

##           Feature  Accession Count
## 1 ENSG00000000003 SRR1039508   679
## 2 ENSG00000000005 SRR1039508     0
## 3 ENSG00000000419 SRR1039508   467

dbDisconnect(con)

5.2 dbplyr

Open data base

library(dplyr)
library(dbplyr)
src <- src_sqlite(airway_db)
src

## src:  sqlite 3.22.0 [/private/var/folders/yn/gmsh_22s2c55v816r6d51fx1tnyl61/T/RtmpLt7lbb/airway.sqlite]
## tbls: count, sample

Manipulations on “Sample” table – standard verbs, plus collect()

tbl(src, "Sample")

## # Source:   table<Sample> [?? x 10]
## # Database: sqlite 3.22.0
## #   [/private/var/folders/yn/gmsh_22s2c55v816r6d51fx1tnyl61/T/RtmpLt7lbb/airway.sqlite]
##   Accession SampleName cell  dex   albut Run   avgLength Experiment Sample
##   <chr>     <chr>      <chr> <chr> <chr> <chr>     <int> <chr>      <chr> 
## 1 SRR10395… GSM1275862 N613… untrt untrt SRR1…       126 SRX384345  SRS50…
## 2 SRR10395… GSM1275863 N613… trt   untrt SRR1…       126 SRX384346  SRS50…
## 3 SRR10395… GSM1275866 N052… untrt untrt SRR1…       126 SRX384349  SRS50…
## 4 SRR10395… GSM1275867 N052… trt   untrt SRR1…        87 SRX384350  SRS50…
## 5 SRR10395… GSM1275870 N080… untrt untrt SRR1…       120 SRX384353  SRS50…
## 6 SRR10395… GSM1275871 N080… trt   untrt SRR1…       126 SRX384354  SRS50…
## 7 SRR10395… GSM1275874 N061… untrt untrt SRR1…       101 SRX384357  SRS50…
## 8 SRR10395… GSM1275875 N061… trt   untrt SRR1…        98 SRX384358  SRS50…
## # … with 1 more variable: BioSample <chr>

tbl(src, "Sample") %>% select(Accession, cell, dex)

## # Source:   lazy query [?? x 3]
## # Database: sqlite 3.22.0
## #   [/private/var/folders/yn/gmsh_22s2c55v816r6d51fx1tnyl61/T/RtmpLt7lbb/airway.sqlite]
##   Accession  cell    dex  
##   <chr>      <chr>   <chr>
## 1 SRR1039508 N61311  untrt
## 2 SRR1039509 N61311  trt  
## 3 SRR1039512 N052611 untrt
## 4 SRR1039513 N052611 trt  
## 5 SRR1039516 N080611 untrt
## 6 SRR1039517 N080611 trt  
## 7 SRR1039520 N061011 untrt
## 8 SRR1039521 N061011 trt

tbl(src, "Sample") %>% filter(dex == "trt") %>% collect()

## # A tibble: 4 x 10
##   Accession SampleName cell  dex   albut Run   avgLength Experiment Sample
##   <chr>     <chr>      <chr> <chr> <chr> <chr>     <int> <chr>      <chr> 
## 1 SRR10395… GSM1275863 N613… trt   untrt SRR1…       126 SRX384346  SRS50…
## 2 SRR10395… GSM1275867 N052… trt   untrt SRR1…        87 SRX384350  SRS50…
## 3 SRR10395… GSM1275871 N080… trt   untrt SRR1…       126 SRX384354  SRS50…
## 4 SRR10395… GSM1275875 N061… trt   untrt SRR1…        98 SRX384358  SRS50…
## # … with 1 more variable: BioSample <chr>

Manipulations on “Count” table

tbl(src, "Count")

## # Source:   table<Count> [?? x 3]
## # Database: sqlite 3.22.0
## #   [/private/var/folders/yn/gmsh_22s2c55v816r6d51fx1tnyl61/T/RtmpLt7lbb/airway.sqlite]
##    Feature         Accession  Count
##    <chr>           <chr>      <int>
##  1 ENSG00000000003 SRR1039508   679
##  2 ENSG00000000005 SRR1039508     0
##  3 ENSG00000000419 SRR1039508   467
##  4 ENSG00000000457 SRR1039508   260
##  5 ENSG00000000460 SRR1039508    60
##  6 ENSG00000000938 SRR1039508     0
##  7 ENSG00000000971 SRR1039508  3251
##  8 ENSG00000001036 SRR1039508  1433
##  9 ENSG00000001084 SRR1039508   519
## 10 ENSG00000001167 SRR1039508   394
## # … with more rows

tbl(src, "Count") %>%
    group_by(Accession) %>%
    summarize(library_size = SUM(Count)) %>%
    collect()

## # A tibble: 8 x 2
##   Accession  library_size
##   <chr>             <int>
## 1 SRR1039508     20637971
## 2 SRR1039509     18809481
## 3 SRR1039512     25348649
## 4 SRR1039513     15163415
## 5 SRR1039516     24448408
## 6 SRR1039517     30818215
## 7 SRR1039520     19126151
## 8 SRR1039521     21164133

Relations between tables

left_join(tbl(src, "Count"), tbl(src, "Sample"))

## Joining, by = "Accession"

## # Source:   lazy query [?? x 12]
## # Database: sqlite 3.22.0
## #   [/private/var/folders/yn/gmsh_22s2c55v816r6d51fx1tnyl61/T/RtmpLt7lbb/airway.sqlite]
##    Feature Accession Count SampleName cell  dex   albut Run   avgLength
##    <chr>   <chr>     <int> <chr>      <chr> <chr> <chr> <chr>     <int>
##  1 ENSG00… SRR10395…   679 GSM1275862 N613… untrt untrt SRR1…       126
##  2 ENSG00… SRR10395…     0 GSM1275862 N613… untrt untrt SRR1…       126
##  3 ENSG00… SRR10395…   467 GSM1275862 N613… untrt untrt SRR1…       126
##  4 ENSG00… SRR10395…   260 GSM1275862 N613… untrt untrt SRR1…       126
##  5 ENSG00… SRR10395…    60 GSM1275862 N613… untrt untrt SRR1…       126
##  6 ENSG00… SRR10395…     0 GSM1275862 N613… untrt untrt SRR1…       126
##  7 ENSG00… SRR10395…  3251 GSM1275862 N613… untrt untrt SRR1…       126
##  8 ENSG00… SRR10395…  1433 GSM1275862 N613… untrt untrt SRR1…       126
##  9 ENSG00… SRR10395…   519 GSM1275862 N613… untrt untrt SRR1…       126
## 10 ENSG00… SRR10395…   394 GSM1275862 N613… untrt untrt SRR1…       126
## # … with more rows, and 3 more variables: Experiment <chr>, Sample <chr>,
## #   BioSample <chr>

left_join(
    tbl(src, "Count"),
    tbl(src, "Sample") %>% select(Accession, cell, dex)
)

## Joining, by = "Accession"

## # Source:   lazy query [?? x 5]
## # Database: sqlite 3.22.0
## #   [/private/var/folders/yn/gmsh_22s2c55v816r6d51fx1tnyl61/T/RtmpLt7lbb/airway.sqlite]
##    Feature         Accession  Count cell   dex  
##    <chr>           <chr>      <int> <chr>  <chr>
##  1 ENSG00000000003 SRR1039508   679 N61311 untrt
##  2 ENSG00000000005 SRR1039508     0 N61311 untrt
##  3 ENSG00000000419 SRR1039508   467 N61311 untrt
##  4 ENSG00000000457 SRR1039508   260 N61311 untrt
##  5 ENSG00000000460 SRR1039508    60 N61311 untrt
##  6 ENSG00000000938 SRR1039508     0 N61311 untrt
##  7 ENSG00000000971 SRR1039508  3251 N61311 untrt
##  8 ENSG00000001036 SRR1039508  1433 N61311 untrt
##  9 ENSG00000001084 SRR1039508   519 N61311 untrt
## 10 ENSG00000001167 SRR1039508   394 N61311 untrt
## # … with more rows

Library size

Find column (Accession) counts

tbl(src, "Count") %>%
    group_by(Accession) %>%
    summarize(library_size = SUM(Count))

## # Source:   lazy query [?? x 2]
## # Database: sqlite 3.22.0
## #   [/private/var/folders/yn/gmsh_22s2c55v816r6d51fx1tnyl61/T/RtmpLt7lbb/airway.sqlite]
##   Accession  library_size
##   <chr>             <int>
## 1 SRR1039508     20637971
## 2 SRR1039509     18809481
## 3 SRR1039512     25348649
## 4 SRR1039513     15163415
## 5 SRR1039516     24448408
## 6 SRR1039517     30818215
## 7 SRR1039520     19126151
## 8 SRR1039521     21164133

Filter rows with non-zero counts

Rows with non-zero counts

keep <- tbl(src, "Count") %>%
    group_by(Feature) %>%
    summarize(row_sum = SUM(Count)) %>%
    filter(row_sum > 0) %>%
    select(Feature)

left_join() to keep only these rows

left_join(keep, tbl(src, "Count"))

## Joining, by = "Feature"

## # Source:   lazy query [?? x 3]
## # Database: sqlite 3.22.0
## #   [/private/var/folders/yn/gmsh_22s2c55v816r6d51fx1tnyl61/T/RtmpLt7lbb/airway.sqlite]
##    Feature         Accession  Count
##    <chr>           <chr>      <int>
##  1 ENSG00000000003 SRR1039508   679
##  2 ENSG00000000003 SRR1039509   448
##  3 ENSG00000000003 SRR1039512   873
##  4 ENSG00000000003 SRR1039513   408
##  5 ENSG00000000003 SRR1039516  1138
##  6 ENSG00000000003 SRR1039517  1047
##  7 ENSG00000000003 SRR1039520   770
##  8 ENSG00000000003 SRR1039521   572
##  9 ENSG00000000419 SRR1039508   467
## 10 ENSG00000000419 SRR1039509   515
## # … with more rows

5.3 Aside: SRAdb

library(BiocFileCache)
if (nrow(bfcquery(query="SRAdb", field = "rname")) == 0L) {
    fl <- SRAdb::getSRAdbFile(tempdir())
    bfcadd(rname = "SRAdb", fpath = fl, action = "move")
}

fl <- BiocFileCache::bfcrpath(rnames = "SRAdb")
src <- src_sqlite(fl)
tbl(src, "study")
tbl(src, "study") %>%
    filter(study_title %like% "%ovarian%")

6 Other on-disk or remote representations

Data bases are appropriate for ‘relational’ data.

The ‘big’ part of scientific data is often not relational

E.g., an expression matrix

Access patterns for databases adn scientific data often differ.

database: query
scientific data: process all data

Strategy for processing data: iterate through the file

In python, other languages: iterate one record (e.g., sample) at a time.
In R: iterate in chunks to allow vector processing.

6.1 hdf5

library(rhdf5)

Fast ‘block-wise’ access

6.2 TENxBrainData

library(TENxBrainData)
tenx <- TENxBrainData()

Illusions…

log(1 + assay(tenx))

## <27998 x 1306127> DelayedMatrix object of type "double":
##            AAACCTGAGATAGGAG-1 ... TTTGTCATCTGAAAGA-133
##     [1,]                    0   .                    0
##     [2,]                    0   .                    0
##     [3,]                    0   .                    0
##     [4,]                    0   .                    0
##     [5,]                    0   .                    0
##      ...                    .   .                    .
## [27994,]            0.0000000   .                    0
## [27995,]            0.6931472   .                    0
## [27996,]            0.0000000   .                    0
## [27997,]            0.0000000   .                    0
## [27998,]            0.0000000   .                    0

Subset

tenx_subset <- tenx[, sample(ncol(tenx), 200)]
count <- as.matrix(assay(tenx_subset))
dotchart(
    unname(colSums(count)),
    xlab = "Library size"
)

hist(log(1 + rowSums(count)))

Actually, though, chunk-wise data processing is transparent

dotchart(
    unname(colSums(assay(tenx_subset))),
    xlab = "Library size"
)

6.3 restfulSE

7 End matter

7.1 Acknowledgements

A portion of this work is supported by the Chan Zuckerberg Initiative DAF, an advised fund of Silicon Valley Community Foundation.

Research reported in this presentation was supported by the NHGRI and NCI of the National Institutes of Health under award numbers U41HG004059, U24CA180996, and U24CA232979. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

This work was performed on behalf of the SOUND Consortium and funded under the EU H2020 Personalizing Health and Care Program, Action contract number 633974.

Session info

## R version 3.6.0 alpha (2019-04-03 r76311)
## Platform: x86_64-apple-darwin17.7.0 (64-bit)
## Running under: macOS High Sierra 10.13.6
## 
## Matrix products: default
## BLAS:   /Users/ma38727/bin/R-3-6-branch/lib/libRblas.dylib
## LAPACK: /Users/ma38727/bin/R-3-6-branch/lib/libRlapack.dylib
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## attached base packages:
## [1] parallel  stats4    stats     graphics  grDevices utils     datasets 
## [8] methods   base     
## 
## other attached packages:
##  [1] TENxBrainData_1.3.0         HDF5Array_1.11.11          
##  [3] SingleCellExperiment_1.5.2  rhdf5_2.27.15              
##  [5] dbplyr_1.3.0                RSQLite_2.1.1              
##  [7] ggplot2_3.1.1               tibble_2.1.1               
##  [9] dplyr_0.8.0.1               airway_1.3.0               
## [11] SummarizedExperiment_1.13.0 DelayedArray_0.9.9         
## [13] BiocParallel_1.17.18        matrixStats_0.54.0         
## [15] Biobase_2.43.1              GenomicRanges_1.35.1       
## [17] GenomeInfoDb_1.19.3         IRanges_2.17.4             
## [19] S4Vectors_0.21.22           BiocGenerics_0.29.2        
## [21] BiocStyle_2.11.0           
## 
## loaded via a namespace (and not attached):
##  [1] httr_1.4.0                    bit64_0.9-7                  
##  [3] AnnotationHub_2.15.12         shiny_1.3.0                  
##  [5] assertthat_0.2.1              interactiveDisplayBase_1.21.0
##  [7] BiocManager_1.30.4.9006       BiocFileCache_1.7.7          
##  [9] blob_1.1.1                    GenomeInfoDbData_1.2.1       
## [11] yaml_2.2.0                    pillar_1.3.1                 
## [13] lattice_0.20-38               glue_1.3.1                   
## [15] digest_0.6.18                 promises_1.0.1               
## [17] XVector_0.23.2                colorspace_1.4-1             
## [19] httpuv_1.5.1                  htmltools_0.3.6              
## [21] Matrix_1.2-17                 plyr_1.8.4                   
## [23] pkgconfig_2.0.2               bookdown_0.9                 
## [25] zlibbioc_1.29.0               xtable_1.8-3                 
## [27] purrr_0.3.2                   scales_1.0.0                 
## [29] later_0.8.0                   withr_2.1.2                  
## [31] lazyeval_0.2.2                cli_1.1.0                    
## [33] mime_0.6                      magrittr_1.5                 
## [35] crayon_1.3.4                  memoise_1.1.0                
## [37] evaluate_0.13                 fansi_0.4.0                  
## [39] tools_3.6.0                   stringr_1.4.0                
## [41] Rhdf5lib_1.5.4                munsell_0.5.0                
## [43] AnnotationDbi_1.45.1          compiler_3.6.0               
## [45] rlang_0.3.4                   grid_3.6.0                   
## [47] RCurl_1.95-4.12               rappdirs_0.3.1               
## [49] bitops_1.0-6                  labeling_0.3                 
## [51] rmarkdown_1.12                ExperimentHub_1.9.3          
## [53] gtable_0.3.0                  codetools_0.2-16             
## [55] DBI_1.0.0                     reshape_0.8.8                
## [57] curl_3.3                      R6_2.4.0                     
## [59] knitr_1.22                    bit_1.1-14                   
## [61] utf8_1.1.4                    stringi_1.4.3                
## [63] Rcpp_1.0.1                    tidyselect_0.2.5             
## [65] xfun_0.6