STAT 302, Lecture Slides 3

class: center, top, title-slide

# STAT 302, Lecture Slides 3
## Data Visualization and Manipulation
### Bryan Martin

---

# Outline

1. Introduction to tidyverse
2. Data Visualization
3. Data Manipulation

.middler[**Goal:** Learn how to go from raw data to exploratory data analysis. (Project 1!)]

---
class: inverse

.sectionhead[Part 1: Introduction to tidyverse]
---
layout: true

# tidyverse
---

The `tidyverse` is a collection of powerful R packages for data science. 
They include:

* Reading and saving data: `readr`
* Data manipulation: `tidyr`, `dplyr`
* Data visualization: `ggplot2`
* Working with different data structures: `tibble`, `purrr`, `stringr`, `forcats`

You can install them all using

```r
install.packages("tidyverse")
```

(Remember, you only need to do this once!)

---

Load them all together in your workflow documents by calling

```r
library(tidyverse)
```

```
## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.0 ──
```

```
## ✓ ggplot2 3.3.2     ✓ purrr   0.3.4
## ✓ tibble  3.0.4     ✓ dplyr   1.0.2
## ✓ tidyr   1.1.2     ✓ stringr 1.4.0
## ✓ readr   1.4.0     ✓ forcats 0.5.0
```

```
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## x dplyr::filter()     masks stats::filter()
## x dplyr::group_rows() masks kableExtra::group_rows()
## x dplyr::lag()        masks stats::lag()
```

---

## Conflicts?

Recall that packages are essentially ways for you to install and use functions 
written by others.
Occasionally, some of these functions have the same name and there is a conflict.
Whichever package you load more recently using `library` will mask the old function, meaning that R will default to that version.

In general, this is fine, especially with `tidyverse`. These package authors know when 
they have masked common functions in R, and typically we will prefer `tidyverse` version.

The conflict message is to make sure you know about conflicts. 
You can (and should) hide this in your R Markdown files by adding the parameter
`message=FALSE` or `include=FALSE` to your code chunk when you load packages.

---
layout: false
layout: true

# Pipes
---

.sectionhead[![pipe](pipe.png)]

---

**pipes** use the `%>%` operator to take the output from a previous function call and "pipe" it through to the next function.
This can really help with readability of code when we use multiple nested functions!

We will use the `gapminder` data from Jenny Bryan to demonstrate throughout this section.
Load the package `gapminder` (after installing) and the corresponding data.

```r
library(gapminder)
data(gapminder)
```

Let's preview the data.

```r
gapminder
```

```
## # A tibble: 1,704 x 6
##    country     continent  year lifeExp      pop gdpPercap
##    <fct>       <fct>     <int>   <dbl>    <int>     <dbl>
##  1 Afghanistan Asia       1952    28.8  8425333      779.
##  2 Afghanistan Asia       1957    30.3  9240934      821.
##  3 Afghanistan Asia       1962    32.0 10267083      853.
##  4 Afghanistan Asia       1967    34.0 11537966      836.
##  5 Afghanistan Asia       1972    36.1 13079460      740.
##  6 Afghanistan Asia       1977    38.4 14880372      786.
##  7 Afghanistan Asia       1982    39.9 12881816      978.
##  8 Afghanistan Asia       1987    40.8 13867957      852.
##  9 Afghanistan Asia       1992    41.7 16317921      649.
## 10 Afghanistan Asia       1997    41.8 22227415      635.
## # … with 1,694 more rows
```

---

## A brief note on factors

```r
str(gapminder)
```

```
## tibble [1,704 × 6] (S3: tbl_df/tbl/data.frame)
##  $ country  : Factor w/ 142 levels "Afghanistan",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ continent: Factor w/ 5 levels "Africa","Americas",..: 3 3 3 3 3 3 3 3 3 3 ...
##  $ year     : int [1:1704] 1952 1957 1962 1967 1972 1977 1982 1987 1992 1997 ...
##  $ lifeExp  : num [1:1704] 28.8 30.3 32 34 36.1 ...
##  $ pop      : int [1:1704] 8425333 9240934 10267083 11537966 13079460 14880372 12881816 13867957 16317921 22227415 ...
##  $ gdpPercap: num [1:1704] 779 821 853 836 740 ...
```

**factors** are categorical data that use integer representation.

This can be efficient to store character vectors, because each string is only entered once. 
Because of this, creating data frames (but not tibbles!) in R often default to set strings as factors. 
However, it can also get annoying! 
Factors limit the categories, or `levels`, of the vector, which is often not appropriate.
I usually turn this off by setting the parameter `stringsAsFactors = FALSE` when I use `data.frame()`, unless I specifically want my data encoded as categorical.

---

## Functions in Functions

What if we want the mean log population within our data set? 
Using base R, we might type

```r
mean(log(gapminder$pop))
```

```
## [1] 15.76611
```

With pipes, we can use

```r
gapminder$pop %>%
  log() %>%
  mean()
```

```
## [1] 15.76611
```

Both of these are acceptable. However, when we start adding more and more functions,
pipes can make your code much easier and more natural to read.

---
layout: false

# filtering data

Very commonly, we will want to filter our data to restrict what we plot or analyze.
For example, what if we are only interested in data from China since 1970?

```r
china_1980 <- gapminder %>% 
  filter(country == "China" &
           year >= 1970)
china_1980
```

```
## # A tibble: 8 x 6
##   country continent  year lifeExp        pop gdpPercap
##   <fct>   <fct>     <int>   <dbl>      <int>     <dbl>
## 1 China   Asia       1972    63.1  862030000      677.
## 2 China   Asia       1977    64.0  943455000      741.
## 3 China   Asia       1982    65.5 1000281000      962.
## 4 China   Asia       1987    67.3 1084035000     1379.
## 5 China   Asia       1992    68.7 1164970000     1656.
## 6 China   Asia       1997    70.4 1230075000     2289.
## 7 China   Asia       2002    72.0 1280400000     3119.
## 8 China   Asia       2007    73.0 1318683096     4959.
```

---
class: inverse

.sectionhead[Part 2: Data Visualization]
---
layout: true

# Data Visualization
---

`ggplot2` is a fantastic way to make complex publication quality<sup>1</sup> images
without much pre-processing of your data.
Plots are built sequentially using layers, so it's easy to edit and fine-tune the plots you generate.

When using `ggplot2`, it is *essential* that your data are tidy!
If they are not, the functions probably will not look like you expect.

Let's work through how to build a plot layer by layer.

.footnote[[1] or STAT302 Project quality]

---

First, let's initialize a plot. Use the `data` parameter to tell `ggplot` what data frame to use.

* It should be tidy data, in either a `data.frame` or `tibble`!

.pull-left[

```r
*ggplot(data = gapminder)
```
]

.pull-right[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-10-1.png)
]

---

Add an aesthetic using the function `aes()` within the initial `ggplot()` call.

* Controls our axes variables as well as graphical parameters such as color, size, shape

.pull-left[

```r
ggplot(data = gapminder,
*      aes(x = year, y = lifeExp))
```
]

.pull-right[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-12-1.png)
]

---

Now `ggplot` knows what to plot, but it doesn't know how to plot it yet. Let's add some points with `geom_point()`.

* This is a new layer! We add layers using the `+` operator.

.pull-left[

```r
ggplot(data = gapminder,
       aes(x = year, y = lifeExp)) +
* geom_point()
```
]

.pull-right[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-14-1.png)
]

---

Let's make our points smaller and red.

.pull-left[

```r
ggplot(data = gapminder,
       aes(x = year, y = lifeExp)) +
* geom_point(color = "red", size = 0.75)
```
]

.pull-right[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-16-1.png)
]

---

Let's try switching them to lines.

.pull-left[

```r
ggplot(data = gapminder,
       aes(x = year, y = lifeExp)) +
* geom_line(color = "red", size = 0.75)
```
]

.pull-right[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-18-1.png)
]

---

We want lines connected by country, not just in the order they appear in the data.

.pull-left[

```r
ggplot(data = gapminder,
       aes(x = year, y = lifeExp,
*          group = country)) +
  geom_line(color = "red", size = 0.5) 
```
]

.pull-right[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-20-1.png)
]

---

We want to explore differences across continents, so let's color by continent

* We use `aes()` because we want to color by something in our data
* Putting a color within `aes()` will automatically add a label
* We have to remove the color within `geom_line()`, or it will override the `aes()`

.pull-left[

```r
ggplot(data = gapminder,
       aes(x = year, y = lifeExp,
           group = country,
*          color = continent)) +
* geom_line(size = 0.5)
```
]

.pull-right[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-22-1.png)
]

---

Let's add another layer. Trendlines by continent!

* We want them grouped differently than our lines (by continent), so we use a new `aes()`
* We will make them stick out by having them be thicker and black
* We don't want error bars, so we will remove `se`
* We will use the default option `loess`. (See `?geom_smooth()`)

.pull-left[

```r
ggplot(data = gapminder,
       aes(x = year, y = lifeExp,
           group = country,
           color = continent)) + 
  geom_line(size = 0.5) +
* geom_smooth(aes(group = continent),
*             se = FALSE, size = 1.5,
*             color = "black",
*             method = "loess")
```
]

.pull-right[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-24-1.png)
]

---

This is cluttered and hard to read. Let's try separating by continents using **facets**!

* We use `facet_wrap` 
* Takes in a **formula** object. Use a tilde `~` and then the variable name you want

.pull-left[

.pull-right[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-26-1.png)
]

---

Looking better! Now let's fine tune. First we'll formalize the labels on our plot using `labs()`.

* Can also edit labels one at a time using `xlab()`, `ylab()`, `ggmain()`, ...
* You should probably do this in every graph you present! Very rarely do you want the text styling of your data frame to match your output. Emphasis here on human readability!

```r
ggplot(data = gapminder,
       aes(x = year, y = lifeExp,
           group = country,
           color = continent)) + 
  geom_line(size = 0.5) +
  geom_smooth(aes(group = continent), 
              se = FALSE, size = 1.5, 
              color = "black", 
              method = "loess") +
  facet_wrap(~ continent) +
* labs(title = "Life expectancy over time by continent",
*      x = "Year", y = "Life Expectancy",
*      legend = "Continent")
```

---

.middler[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-28-1.png)
]

---

Let's center our title by adjusting `theme()`.

* `element_text()` tells ggplot how to display the text. Used for fine-tuning
* `hjust` is our horizontal alignment, we set it to one half

```r
ggplot(data = gapminder,
       aes(x = year, y = lifeExp,
           group = country,
           color = continent)) + 
  geom_line(size = 0.5) +
  geom_smooth(aes(group = continent), 
              se = FALSE, size = 1.5, color = "black", method = "loess") +
  facet_wrap(~ continent) +
  labs(title = "Life expectancy over time by continent", 
       x = "Year", y = "Life Expectancy", legend = "Continent") +
* theme(plot.title = element_text(hjust = 0.5,
*                                 face = "bold",
*                                 size = 14))
```

---

.middler[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-30-1.png)
]

---

Actually, that legend is redundant. Let's remove it.

```r
ggplot(data = gapminder,
       aes(x = year, y = lifeExp,
           group = country,
           color = continent)) + 
  geom_line(size = 0.5) +
  geom_smooth(aes(group = continent), 
              se = FALSE, size = 1.5, color = "black", method = "loess") +
  facet_wrap(~ continent) +
  labs(title = "Life expectancy over time by continent", 
       x = "Year", y = "Life Expectancy") +
  theme(plot.title = element_text(hjust = 0.5, face = "bold", size = 14),
*       legend.position = "none")
```

---

.middler[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-32-1.png)
]

---

I don't like the default gray background. I almost always set the `theme_bw()`.

* There are several other theme options! Too many to list, so look them up if you want more options.

```r
ggplot(data = gapminder,
       aes(x = year, y = lifeExp,
           group = country,
           color = continent)) + 
  geom_line(size = 0.5) +
  geom_smooth(aes(group = continent), 
              se = FALSE, size = 1.5, color = "black", method = "loess") +
  facet_wrap(~ continent) +
  labs(title = "Life expectancy over time by continent", 
       x = "Year", y = "Life Expectancy") +
  theme(plot.title = element_text(hjust = 0.5, face = "bold", size = 14),
        legend.position = "none") +
* theme_bw()
```

---

.middler[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-34-1.png)
]

---

This overwrote our custome `theme()` settings! Let's reorder the layers.

```r
ggplot(data = gapminder,
       aes(x = year, y = lifeExp,
           group = country,
           color = continent)) + 
  geom_line(size = 0.5) +
  geom_smooth(aes(group = continent), 
              se = FALSE, size = 1.5, color = "black", method = "loess") +
  facet_wrap(~ continent) +
  labs(title = "Life expectancy over time by continent", 
       x = "Year", y = "Life Expectancy") +
* theme_bw() +
* theme(plot.title = element_text(hjust = 0.5, face = "bold", size = 14),
*       legend.position = "none")
```

---

.middler[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-36-1.png)
]

---

We can make this more readable. Be careful of text that is too small! 
Let's increase all of our text proportionally using `base_size` within `theme_bw()`.

* We could also do this by adjusting `text` within `theme()`
* Note that we no longer need to manually adjust our title size. This will scale everything automatically.

```r
ggplot(data = gapminder,
       aes(x = year, y = lifeExp,
           group = country,
           color = continent)) + 
  geom_line(size = 0.5) +
  geom_smooth(aes(group = continent), 
              se = FALSE, size = 1.5, color = "black", method = "loess") +
  facet_wrap(~ continent) +
  labs(title = "Life expectancy over time by continent", 
       x = "Year", y = "Life Expectancy") +
* theme_bw(base_size = 16) +
* theme(plot.title = element_text(hjust = 0.5, face = "bold"),
        legend.position = "none") 
```

---

.middler[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-38-1.png)
]

---

Almost there! Now our text is a good size, but it overlaps. There are a few ways we could fix this. We'll rotate our text.

```r
ggplot(data = gapminder,
       aes(x = year, y = lifeExp,
           group = country,
           color = continent)) + 
  geom_line(size = 0.5) +
  geom_smooth(aes(group = continent), 
              se = FALSE, size = 1.5, color = "black", method = "loess") +
  facet_wrap(~ continent) +
  labs(title = "Life expectancy over time by continent", 
       x = "Year", y = "Life Expectancy") +
  theme_bw(base_size = 16) + 
  theme(plot.title = element_text(hjust = 0.5, face = "bold"), 
        legend.position = "none",
*       axis.text.x = element_text(angle = 45, hjust = 1, vjust = 1))
```

---

.middler[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-40-1.png)
]

---

Lastly, let's space out our panels by adjusting `panel.spacing.x`.

```r
ggplot(data = gapminder,
       aes(x = year, y = lifeExp,
           group = country,
           color = continent)) + 
  geom_line(size = 0.5) +
  geom_smooth(aes(group = continent), 
              se = FALSE, size = 1.5, color = "black", method = "loess") +
  facet_wrap(~ continent) +
  labs(title = "Life expectancy over time by continent", 
       x = "Year", y = "Life Expectancy") +
  theme_bw(base_size = 16) + 
  theme(plot.title = element_text(hjust = 0.5, face = "bold"), 
        legend.position = "none",
        axis.text.x = element_text(angle = 45, hjust = 1, vjust = 1),
*       panel.spacing.x = unit(0.75, "cm"))
```

---

.middler[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-42-1.png)
]

---

Great, that looks good! Now we can store it as an object.

```r
*lifeExp_plot <- ggplot(data = gapminder,
                       aes(x = year, y = lifeExp,
                           group = country,
                           color = continent)) + 
  geom_line(size = 0.5) +
  geom_smooth(aes(group = continent), 
              se = FALSE, size = 1.5, color = "black", method = "loess") +
  facet_wrap(~ continent) +
  labs(title = "Life expectancy over time by continent", 
       x = "Year", y = "Life Expectancy") +
  theme_bw(base_size = 16) + 
  theme(plot.title = element_text(hjust = 0.5, face = "bold"), 
        legend.position = "none",
        axis.text.x = element_text(angle = 45, hjust = 1, vjust = 1),
        panel.spacing.x = unit(0.75, "cm")) 
```

---

Then we can plot it by just calling our object.

```r
lifeExp_plot
```

.center[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-45-1.png)
]

---

We can also save it in our `Figures` subfolder using `ggsave()`.

* Set `height` and `width` parameters to automatically resize the image

```r
ggsave(filename = "Figures/lifeExp_plot.pdf", plot = lifeExp_plot,
       height = 5, width = 7)
```

.center[**Never** save a figure from your analysis using screenshots or point-and-click!

They will be lower quality and non-reproducible!]

---

## Some comments on `ggplot`:

* What we just made was a *very* complicated and fine-tuned plot!
* I have to Google how to adjust certain things all the time
* So does the creator of `ggplot2`

.center[![hadley](hadley.png)]

---

## A simpler example: Histogram

.pull-left[

```r
*ggplot(data = gapminder,
*      aes(x = lifeExp))
```
]

.pull-right[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-48-1.png)
]

---

## A simpler example: Histogram

.pull-left[

```r
ggplot(data = gapminder,
       aes(x = lifeExp)) + 
* geom_histogram()
```
]

.pull-right[

```
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
```

![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-50-1.png)
]

---

## A simpler example: Histogram

.pull-left[

```r
ggplot(data = gapminder,
       aes(x = lifeExp)) + 
* geom_histogram(binwidth = 1)
```
]

.pull-right[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-52-1.png)
]

---

## A simpler example: Histogram

.pull-left[

```r
ggplot(data = gapminder,
       aes(x = lifeExp)) + 
  geom_histogram(binwidth = 1, 
*                color = "black",
*                fill = "lightblue")
```
]

.pull-right[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-54-1.png)
]

---

## A simpler example: Histogram

.pull-left[

```r
ggplot(data = gapminder,
       aes(x = lifeExp)) + 
  geom_histogram(binwidth = 1, 
                 color = "black", 
                 fill = "lightblue") +
* theme_bw(base_size = 20)
```
]

.pull-right[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-56-1.png)
]

---

## A simpler example: Histogram

.pull-left[

```r
ggplot(data = gapminder,
       aes(x = lifeExp)) + 
  geom_histogram(binwidth = 1, 
                 color = "black", 
                 fill = "lightblue") +
  theme_bw(base_size = 20) +
* labs(x = "Life Expectancy",
*      y = "Count")
```
]

.pull-right[
![](lectureslides_datavizmanip_files/figure-html/unnamed-chunk-58-1.png)
]

---