R for Data Analysis: A Short Tutorial

class: top, left, inverse, title-slide

.title[
# R for Data Analysis: A Short Tutorial
]
.subtitle[
## Session 2: Data Wrangling
]
.author[
### Dimiter Toshkov
]
.institute[
### Institute of Public Administration, Leiden University
]
.date[
### last updated: 2025-03-31
]

---

.title-slide {
  background-image: url(https://cran.r-project.org/Rlogo.svg);
  background-position: 50% 0%; ## just start changing this
  background-size: 150px;
  background-color: #fff;
  padding-left: 100px;  /* delete this for 4:3 aspect ratio */
}

.remark-slide-content {
    font-size: 28px;
    padding: 1em 1em 1em 1em;
}

.remark-slide-content > h1 {
  font-size: 32px;
  margin-top: -85px;
}

</style>

# Last session...

---
class: center, inverse, top

background-image: url("data:image/png;base64,#figs/bigstock-197264191-1.jpg")
background-size: contain

---
# Importing data
<svg viewBox="0 0 448 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M448 73.143v45.714C448 159.143 347.667 192 224 192S0 159.143 0 118.857V73.143C0 32.857 100.333 0 224 0s224 32.857 224 73.143zM448 176v102.857C448 319.143 347.667 352 224 352S0 319.143 0 278.857V176c48.125 33.143 136.208 48.572 224 48.572S399.874 209.143 448 176zm0 160v102.857C448 479.143 347.667 512 224 512S0 479.143 0 438.857V336c48.125 33.143 136.208 48.572 224 48.572S399.874 369.143 448 336z"></path></svg> Let's get some data  from the European Social Survey! You will have to register and download the data file first.

``` r
library(haven)
# adapt the path to the file if needed 
df <- read_sav("./data/ESS11.sav") # this is a tibble, a fancy type of data frame

dt <- as.data.frame(df) # we can make it a plain data frame
```
---
# There are different ways of getting data into R (1)

---
# There are different ways of getting data into R (2)

For a longer overview of options for different file types [see here](https://cran.r-project.org/web/packages/rio/vignettes/rio.html) and [read here](https://www.datacamp.com/community/tutorials/r-tutorial-read-excel-into-r) for more details on data import.

---
# Once we have the data imported

---
# Check out the variable labels
When we import an SPSS file (`sav`), we can inspect labels and value codes.

``` r
library(labelled)
attributes(dt$trstun)
## $label
## [1] "Trust in the United Nations"
## 
## $format.spss
## [1] "F2.0"
## 
## $class
## [1] "haven_labelled" "vctrs_vctr"     "double"        
## 
## $labels
## No trust at all               1               2               3               4 
##               0               1               2               3               4 
##               5               6               7               8               9 
##               5               6               7               8               9 
##  Complete trust         Refusal      Don't know       No answer 
##              10              77              88              99
```

---
# Working with the labels
We can get the wording of the variables (questions) with *label* and we can get the labels of the answer categories with *labels*.
They are both **attributes** of the variable.

``` r
lapply(dt, function(x) attributes(x)$label)[1:10]
lapply(dt, function(x) attributes(x)$labels)[1:5]
```

---
# There is alwasy another way
There is an alternative way to achieve the same thing with the `labelled` package. Note that this will only work on data that is imported with `haven` (and has labels already coded).

``` r
var_label(dt) # variable names
var_label(dt$gndr) 
var_label(dt$cntry) <- "country" # can be reassigned
look_for(dt, 'internet', details=TRUE) # search in variable and value names
val_labels(dt$stfdem) # inspect the value labels
```

---
# Zoom in on some variables

``` r
table(dt$vote) # this is good for factor-like variables
## 
##     1     2     3 
## 29551  7539  2706
prop.table(table(dt$vote)) # turn frequencies into proportions
## 
##          1          2          3 
## 0.74256207 0.18944115 0.06799678
summary(dt$polintr) # for continuous variables, use summary
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##   1.000   2.000   3.000   2.626   3.000   4.000      56
```

---
# Transforming variables (1)

``` r
summary(dt$new.vote <- factor(dt$vote))
##     1     2     3  NA's 
## 29551  7539  2706   360
levels(dt$new.vote) <- c('voted', 'no vote', 'dont know') # recode levels
summary(dt$new.vote)
##     voted   no vote dont know      NA's 
##     29551      7539      2706       360
```

---
# Transforming variables (2)
You can use `cut()` to split a continuous variable into categories

``` r
dt$satdem.cat <- cut(dt$stfdem, # which variable
                     breaks = c(-Inf, 3, 7, Inf), # interval break points, incl. start and end
                     labels=c("low","medium","high")) # labels of the new categories
table(dt$satdem.cat)
## 
##    low medium   high 
##  10109  20589   8214
```
---
# Subsetting (1)

``` r
dt.subset.1 <- dt [1:10, c('cntry','vote','polintr')]
dt.subset.2 <- dt [seq(1, 101, by=10), 1:5]
dt.subset.3 <- dt [dt$vote==1, -c(1:5)]
dt.subset.nonas <- dt [is.na(dt$vote)==FALSE &
                         is.na(dt$polintr)==FALSE, 1:5]
```
---
# Subsetting (2)

``` r
dt.subset.4 <- subset (dt, vote==2 | 
                         polintr > median(polintr, na.rm=TRUE))
```
---
# Getting rid of missing values

``` r
sum(is.na(dt$vote)) # count NAs of individual columns
## [1] 360
dt.subset.complete <- dt [complete.cases(dt)==TRUE,]
dim(dt.subset.complete) # oops
## [1]   0 642
```

---
class: center, inverse, top

background-image: url("data:image/png;base64,#figs/wrangle.jpg")
background-size: contain

---
# Now let's do thnigs the *tidy* way

``` r
library (dplyr) # most of the tidy data wrangling functions are in this package
library (tidyverse) # which is part of this collection

```

Also, our working environment is getting quite messy. Let's remove objects we don't need to free up memory:

``` r
rm (dt.subset.1, dt.subset.2, dt.subset.3, dt.subset.4, 
   dt.subset.complete, dt.subset.nonas)
# rm (list = ls()) # to remove all objects
```

---
# Subsetting (the tidy way)

To start with, let's see the tidy ways of subsetting

``` r
# choose columns
dt.subset.1 <- select (dt, cntry, vote, polintr, stfdem) 
# choose rows
dt.subset.1 <- filter (dt.subset.1, vote == 1, 
                       polintr > 2, stfdem != 0)
# choose rows by ordinal position
dt.subset.1 <- slice (dt.subset.1, 1:10) 
```
---
# The need for a pipe (1)

.pull-left[
1. Coerce a vector into a character vector
2. Then coerce it into a numeric vector
3. Then take the mean
4. Then take the square root of the mean
5. Then round the number to 2 digits 
]

.pull-right[

``` r
v = dt[1:100, "stfdem"]
round(sqrt(
  mean(
    as.numeric(
      as.character(v)
      ), 
    na.rm=TRUE)
  ), 
digits=2)
## [1] 2.4
```
]

---
# The need for a pipe (2)

Not very easy to read, is it? And it usually looks even worse

``` r
round(sqrt(mean(as.numeric(as.character(v)), na.rm=TRUE)), digits=2)
## [1] 2.4
```

Of course, we can do the operations one by one, but then we will have to reasign the object every step of the way, which is tedious.

``` r
v = dt[1:100, "stfdem"]  
v = as.numeric(as.character(v))
m = sqrt(mean (v, na.rm=T))
round(m, digits = 2)
## [1] 2.4
```

---
# Meet the pipe
<svg viewBox="0 0 496 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M248 8C111 8 0 119 0 256s111 248 248 248 248-111 248-248S385 8 248 8zm0 448c-110.3 0-200-89.7-200-200S137.7 56 248 56s200 89.7 200 200-89.7 200-200 200zm84-143.4c-20.8 25-51.5 39.4-84 39.4s-63.2-14.3-84-39.4c-8.5-10.2-23.6-11.5-33.8-3.1-10.2 8.5-11.5 23.6-3.1 33.8 30 36 74.1 56.6 120.9 56.6s90.9-20.6 120.9-56.6c8.5-10.2 7.1-25.3-3.1-33.8-10.2-8.4-25.3-7.1-33.8 3.1zM136.5 211c7.7-13.7 19.2-21.6 31.5-21.6s23.8 7.9 31.5 21.6l9.5 17c2.1 3.7 6.2 4.7 9.3 3.7 3.6-1.1 6-4.5 5.7-8.3-3.3-42.1-32.2-71.4-56-71.4s-52.7 29.3-56 71.4c-.3 3.7 2.1 7.2 5.7 8.3 3.4 1.1 7.4-.5 9.3-3.7l9.5-17zM328 152c-23.8 0-52.7 29.3-56 71.4-.3 3.7 2.1 7.2 5.7 8.3 3.5 1.1 7.4-.5 9.3-3.7l9.5-17c7.7-13.7 19.2-21.6 31.5-21.6s23.8 7.9 31.5 21.6l9.5 17c2.1 3.7 6.2 4.7 9.3 3.7 3.6-1.1 6-4.5 5.7-8.3-3.3-42.1-32.2-71.4-56-71.4z"></path></svg> Wouldn't it be nice if we could write the operations in the order that they have to be executed, without having to reassign objects all the time?

---
# Using the pipe
Let's see the pipe in action:

``` r
v = dt[1:100, "stfdem"]

v %>% 
  as.character() %>% 
  as.numeric() %>% 
  mean(na.rm=TRUE) %>% 
  sqrt() %>% 
  round(digits=2)
## [1] 2.4
```

Tidy, indeed! For a short tutorial on using the pipe, [read here](https://www.datacamp.com/community/tutorials/pipe-r-tutorial).

---
# More on pipes
Essentially, the tidy pipe (orginally, from the package `margrittr`) takes the object before the pipe and inserts it as the first argument in the function after the pipe.

Not so long ago, there was no simple forward pipe in the base R installation. Since, *R version 4.1.0* introduced the operator `|>`, which functions as a 'native' pipe. But there are some differences with the tidy `magrittr` pipe. If you want to know more about the differences, read [this post](https://ivelasq.rbind.io/blog/understanding-the-r-pipe/). Personally, I have had no reason to use the native pipe so far.

---
# Let's go back to data wrangling (with pipes)

---
class: center, inverse, top
background-image: url("data:image/png;base64,#figs/pipe.jpg")
background-size: contain

---
# Let's go back to data wrangling (with pipes)
Select and filter

``` r
dt.subset<- dt %>% 
  select (cntry, vote, polintr, stfdem) %>% # selecting variables
  filter (vote==1, polintr>2, stfdem!=0) %>% # filtering observations
  slice (1:3)
```
If we want to keep the output from the pipe, we have to remember to assign it to a (new) object.

---
# Creating new variables is easy

``` r
dt %>% 
  mutate (weight.rounded = round(pspwght, digits=0),
          country.ess = paste(cntry, essround,  sep="."),
          vote.factor = factor (vote),
          vote.dummy.w = ifelse (vote==1, 1, 0)) %>%
  select (weight.rounded, country.ess, vote.factor, vote.dummy.w) %>%
  slice(sample (5)) %>%
  arrange(weight.rounded)
##   weight.rounded country.ess vote.factor vote.dummy.w
## 1              0       AT.11           2            0
## 2              0       AT.11           1            1
## 3              0       AT.11           1            1
## 4              1       AT.11           1            1
## 5              4       AT.11           1            1
```

---
# Recoding variables
<svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M184.561 261.903c3.232 13.997-12.123 24.635-24.068 17.168l-40.736-25.455-50.867 81.402C55.606 356.273 70.96 384 96.012 384H148c6.627 0 12 5.373 12 12v40c0 6.627-5.373 12-12 12H96.115c-75.334 0-121.302-83.048-81.408-146.88l50.822-81.388-40.725-25.448c-12.081-7.547-8.966-25.961 4.879-29.158l110.237-25.45c8.611-1.988 17.201 3.381 19.189 11.99l25.452 110.237zm98.561-182.915l41.289 66.076-40.74 25.457c-12.051 7.528-9 25.953 4.879 29.158l110.237 25.45c8.672 1.999 17.215-3.438 19.189-11.99l25.45-110.237c3.197-13.844-11.99-24.719-24.068-17.168l-40.687 25.424-41.263-66.082c-37.521-60.033-125.209-60.171-162.816 0l-17.963 28.766c-3.51 5.62-1.8 13.021 3.82 16.533l33.919 21.195c5.62 3.512 13.024 1.803 16.536-3.817l17.961-28.743c12.712-20.341 41.973-19.676 54.257-.022zM497.288 301.12l-27.515-44.065c-3.511-5.623-10.916-7.334-16.538-3.821l-33.861 21.159c-5.62 3.512-7.33 10.915-3.818 16.536l27.564 44.112c13.257 21.211-2.057 48.96-27.136 48.96H320V336.02c0-14.213-17.242-21.383-27.313-11.313l-80 79.981c-6.249 6.248-6.249 16.379 0 22.627l80 79.989C302.689 517.308 320 510.3 320 495.989V448h95.88c75.274 0 121.335-82.997 81.408-146.88z"></path></svg> To recode variables, use `recode()` inside `mutate()`:

``` r
dt.s <- dt %>% 
  mutate (vote.new = dplyr::recode (as.numeric(vote), '1' = 'voted', '0' = 'no vote', 
                                    .default = NA_character_), # special tidy NAs
          stfdem.na = na_if(stfdem, -88),
          stfdem.newna = replace_na(stfdem, 0)) %>%
  rename (satdem = stfdem) %>% 
  select (vote, vote.new, stfdem.na, stfdem.newna, satdem) %>%
  slice(1:5) 
dt.s
##   vote vote.new stfdem.na stfdem.newna satdem
## 1    1    voted         6            6      6
## 2    1    voted         7            7      7
## 3    1    voted         6            6      6
## 4    2     <NA>         6            6      6
## 5    1    voted         8            8      8
```

---
# Summarizing variables
We can also summarize variables by groups and add the result to the data:

``` r
dt.s2 <- dt %>% 
  group_by(cntry) %>% 
  mutate(stfdem.mean = mean(stfdem, na.rm=TRUE),
         count=n()) %>% 
  select(cntry, vote, stfdem, stfdem.mean, count) %>%
  ungroup()

dt.s2
## # A tibble: 40,156 × 5
##    cntry        vote      stfdem    stfdem.mean count
##    <chr+lbl>    <dbl+lbl> <dbl+lbl>       <dbl> <int>
##  1 AT [Austria] 1 [Yes]   6 [6]            5.75  2354
##  2 AT [Austria] 1 [Yes]   7 [7]            5.75  2354
##  3 AT [Austria] 1 [Yes]   6 [6]            5.75  2354
##  4 AT [Austria] 2 [No]    6 [6]            5.75  2354
##  5 AT [Austria] 1 [Yes]   8 [8]            5.75  2354
##  6 AT [Austria] 1 [Yes]   3 [3]            5.75  2354
##  7 AT [Austria] 1 [Yes]   6 [6]            5.75  2354
##  8 AT [Austria] 1 [Yes]   8 [8]            5.75  2354
##  9 AT [Austria] 1 [Yes]   8 [8]            5.75  2354
## 10 AT [Austria] 1 [Yes]   7 [7]            5.75  2354
## # ℹ 40,146 more rows
```

---
# Summarizing variables
There is another way to produce tables of aggregate values:

``` r
dt.aggregate <- dt %>% 
  group_by(cntry) %>% 
  summarize(stfdem.mean = mean(stfdem, na.rm=TRUE),
         count=n()) %>%
  arrange(desc(stfdem.mean))  # order by the values of a variable in descending order

dt.aggregate
## # A tibble: 24 × 3
##    cntry            stfdem.mean count
##    <chr+lbl>              <dbl> <int>
##  1 CH [Switzerland]        7.53  1384
##  2 NO [Norway]             7.19  1337
##  3 FI [Finland]            6.78  1563
##  4 SE [Sweden]             6.59  1230
##  5 NL [Netherlands]        5.97  1695
##  6 IS [Iceland]            5.92   842
##  7 IE [Ireland]            5.85  2017
##  8 AT [Austria]            5.75  2354
##  9 DE [Germany]            5.70  2420
## 10 BE [Belgium]            5.27  1594
## # ℹ 14 more rows
```

---
# Merging data (1)

.panelset[
  .panel[.panel-name[Inner join]

``` r
d1 <- data.frame(cbind ("ID" = c("A","C"), "values" = c(1:2)))
d2 <- data.frame(cbind ("ID" = c("A","D"), "values" = c(3:4)))

inner_join(d1, d2, by='ID') # only observations that are in both are kept
```

```
##   ID values.x values.y
## 1  A        1        3
```
]

.panel[.panel-name[Left join]

``` r
d1 <- data.frame(cbind ("ID" = c("A","C"), "values" = c(1:2)))
d2 <- data.frame(cbind ("ID" = c("A","D"), "values" = c(3:4)))

left_join(d1, d2, by='ID') # all observations from d1 are kept
```

```
##   ID values.x values.y
## 1  A        1        3
## 2  C        2     <NA>
```
]

.panel[.panel-name[Right join]

``` r
d1 <- data.frame(cbind ("ID" = c("A","C"), "values" = c(1:2)))
d2 <- data.frame(cbind ("ID" = c("A","D"), "values" = c(3:4)))

right_join(d1, d2, by='ID') # all observations from d2 are kept
```

```
##   ID values.x values.y
## 1  A        1        3
## 2  D     <NA>        4
```
]

.panel[.panel-name[Full join]

``` r
d1 <- data.frame(cbind ("ID" = c("A","C"), "values" = c(1:2)))
d2 <- data.frame(cbind ("ID" = c("A","D"), "values" = c(3:4)))

full_join(d1, d2, by='ID') # all observations are kept
```

```
##   ID values.x values.y
## 1  A        1        3
## 2  C        2     <NA>
## 3  D     <NA>        4
```
]
]

---
# Merging data (2)
<svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M504 256c0 136.997-111.043 248-248 248S8 392.997 8 256C8 119.083 119.043 8 256 8s248 111.083 248 248zm-248 50c-25.405 0-46 20.595-46 46s20.595 46 46 46 46-20.595 46-46-20.595-46-46-46zm-43.673-165.346l7.418 136c.347 6.364 5.609 11.346 11.982 11.346h48.546c6.373 0 11.635-4.982 11.982-11.346l7.418-136c.375-6.874-5.098-12.654-11.982-12.654h-63.383c-6.884 0-12.356 5.78-11.981 12.654z"></path></svg> Beware situations where the merging variable has non-unique values in one of the datasets. New rows can be silently added to the left-hand side dataset, for example:

``` r
d1 <- data.frame(cbind ("ID" = c("A","C"), "values" = c(1:2)))
d2 <- data.frame(cbind ("ID" = c("A","A"), "values" = c(3:4)))

left_join(d1, d2, by='ID') # now the result has three rows!
```

```
##   ID values.x values.y
## 1  A        1        3
## 2  A        1        4
## 3  C        2     <NA>
```

---
# Combining datasets 
<svg viewBox="0 0 576 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M564 224c6.627 0 12-5.373 12-12v-72c0-6.627-5.373-12-12-12h-72c-6.627 0-12 5.373-12 12v12h-88v-24h12c6.627 0 12-5.373 12-12V44c0-6.627-5.373-12-12-12h-72c-6.627 0-12 5.373-12 12v12H96V44c0-6.627-5.373-12-12-12H12C5.373 32 0 37.373 0 44v72c0 6.627 5.373 12 12 12h12v160H12c-6.627 0-12 5.373-12 12v72c0 6.627 5.373 12 12 12h72c6.627 0 12-5.373 12-12v-12h88v24h-12c-6.627 0-12 5.373-12 12v72c0 6.627 5.373 12 12 12h72c6.627 0 12-5.373 12-12v-12h224v12c0 6.627 5.373 12 12 12h72c6.627 0 12-5.373 12-12v-72c0-6.627-5.373-12-12-12h-12V224h12zM352 64h32v32h-32V64zm0 256h32v32h-32v-32zM64 352H32v-32h32v32zm0-256H32V64h32v32zm32 216v-12c0-6.627-5.373-12-12-12H72V128h12c6.627 0 12-5.373 12-12v-12h224v12c0 6.627 5.373 12 12 12h12v160h-12c-6.627 0-12 5.373-12 12v12H96zm128 136h-32v-32h32v32zm280-64h-12c-6.627 0-12 5.373-12 12v12H256v-12c0-6.627-5.373-12-12-12h-12v-24h88v12c0 6.627 5.373 12 12 12h72c6.627 0 12-5.373 12-12v-72c0-6.627-5.373-12-12-12h-12v-88h88v12c0 6.627 5.373 12 12 12h12v160zm40 64h-32v-32h32v32zm0-256h-32v-32h32v32z"></path></svg> We can also combine combine datasets by row (adding observations from two or more datasets):

``` r
d1 <- dt [1:100, ]
d2 <- dt [101:200, ]
d.combined <- bind_rows(d1, d2, .id = "id.dataset")
d.combined [c(1:2, 101:102), 1:3]
##     id.dataset     name essround
## 1            1 ESS11e02       11
## 2            1 ESS11e02       11
## 101          2 ESS11e02       11
## 102          2 ESS11e02       11
```
The last argument creates an *id* variable for the dataset from which the observation came from.

---
# Batch processing
Often we want to do operations on more than one variable. Then we can use `mutate_all()`, `mutate_at`, `mutate_if` and their equivalents.

``` r
dt [1:2, 19:20]
##   actrolga psppipla
## 1        5        4
## 2        2        3
dt.s <- dt %>% 
  mutate_at (19:20, mean, na.rm=TRUE) %>%
  mutate_at (19:20, round, digits=1) 
dt.s [1:2,19:20]
##   actrolga psppipla
## 1      2.1      2.2
## 2      2.1      2.2
```

---
# Batch renaming
We can also rename batches of column names. Note that we can select names that we want with `starts_with()`, `ends_with`, `contains()` and `matches()`.

``` r
dt <- dt %>% rename_at(vars(starts_with("trst")), ~ paste0("newname", .))
```

---
# More data
<svg viewBox="0 0 448 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M448 73.143v45.714C448 159.143 347.667 192 224 192S0 159.143 0 118.857V73.143C0 32.857 100.333 0 224 0s224 32.857 224 73.143zM448 176v102.857C448 319.143 347.667 352 224 352S0 319.143 0 278.857V176c48.125 33.143 136.208 48.572 224 48.572S399.874 209.143 448 176zm0 160v102.857C448 479.143 347.667 512 224 512S0 479.143 0 438.857V336c48.125 33.143 136.208 48.572 224 48.572S399.874 369.143 448 336z"></path></svg> For the next steps, let's get some data straight from the internet. The Dutch Statistics Office (CBS) has its own package that allows direct access to its data:

``` r
library(cbsodataR)
im <- cbs_get_data('60032') %>% # Migratie; land van herkomst / vestiging, geboorteland, geslacht; 1995-2022
  cbs_add_label_columns() %>% 
  cbs_add_date_column() 
```

---
# Some reorganization
These operations should all be familiar:

``` r
ims2 <- im %>%
  filter(Geslacht_label != 'Totaal mannen en vrouwen',
         LandVanHerkomstVestiging_label == 'Totaal landen',
         Geboorteland_label == 'Totaal',
         Perioden_freq == 'Y') %>%
  rename (year = Perioden_label,
          immigration = Immigratie_1,
          sex = Geslacht_label) %>% 
  select (year, immigration, sex) 
```

---
# Long to wide and back to long

---
# Pivoting data: long to wide
<svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M504.971 359.029c9.373 9.373 9.373 24.569 0 33.941l-80 79.984c-15.01 15.01-40.971 4.49-40.971-16.971V416h-58.785a12.004 12.004 0 0 1-8.773-3.812l-70.556-75.596 53.333-57.143L352 336h32v-39.981c0-21.438 25.943-31.998 40.971-16.971l80 79.981zM12 176h84l52.781 56.551 53.333-57.143-70.556-75.596A11.999 11.999 0 0 0 122.785 96H12c-6.627 0-12 5.373-12 12v56c0 6.627 5.373 12 12 12zm372 0v39.984c0 21.46 25.961 31.98 40.971 16.971l80-79.984c9.373-9.373 9.373-24.569 0-33.941l-80-79.981C409.943 24.021 384 34.582 384 56.019V96h-58.785a12.004 12.004 0 0 0-8.773 3.812L96 336H12c-6.627 0-12 5.373-12 12v56c0 6.627 5.373 12 12 12h110.785c3.326 0 6.503-1.381 8.773-3.812L352 176h32z"></path></svg> Often data is in a *long* format, when we need it *wide*.

``` r
ims2[c(1,dim(ims2)[1]), ]
## # A tibble: 2 × 3
##   year  immigration sex    
##   <fct>       <int> <fct>  
## 1 1995        51481 Mannen 
## 2 2022       208252 Vrouwen
```

``` r
ims2.w <- pivot_wider(ims2, # the dataset
     names_from = sex, # which variable (with categories) to unpack
     values_from = immigration, # where to get the values from
     names_prefix = 'im_') # optional, to change labels
head(ims2.w, 2)
## # A tibble: 2 × 3
##   year  im_Mannen im_Vrouwen
##   <fct>     <int>      <int>
## 1 1995      51481      44618
## 2 1996      56556      52193
```
---
# Pivoting data: wide to long 
<svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M504.971 359.029c9.373 9.373 9.373 24.569 0 33.941l-80 79.984c-15.01 15.01-40.971 4.49-40.971-16.971V416h-58.785a12.004 12.004 0 0 1-8.773-3.812l-70.556-75.596 53.333-57.143L352 336h32v-39.981c0-21.438 25.943-31.998 40.971-16.971l80 79.981zM12 176h84l52.781 56.551 53.333-57.143-70.556-75.596A11.999 11.999 0 0 0 122.785 96H12c-6.627 0-12 5.373-12 12v56c0 6.627 5.373 12 12 12zm372 0v39.984c0 21.46 25.961 31.98 40.971 16.971l80-79.984c9.373-9.373 9.373-24.569 0-33.941l-80-79.981C409.943 24.021 384 34.582 384 56.019V96h-58.785a12.004 12.004 0 0 0-8.773 3.812L96 336H12c-6.627 0-12 5.373-12 12v56c0 6.627 5.373 12 12 12h110.785c3.326 0 6.503-1.381 8.773-3.812L352 176h32z"></path></svg> But then other times data is in a *wide* format, and we need it *long*.

``` r
ims2.l <- pivot_longer(ims2.w, # the dataset
     cols= c(im_Mannen, im_Vrouwen), # which variables to collapse
     names_to = 'sex', # name of the new variable with the categories
     values_to = 'immigration') # name of the new variable with the data
head(ims2.l, 2)
## # A tibble: 2 × 3
##   year  sex        immigration
##   <fct> <chr>            <int>
## 1 1995  im_Mannen        51481
## 2 1995  im_Vrouwen       44618
```

---
# More on data wrangling 
<svg viewBox="0 0 384 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M336 0H48C21.49 0 0 21.49 0 48v464l192-112 192 112V48c0-26.51-21.49-48-48-48zm0 428.43l-144-84-144 84V54a6 6 0 0 1 6-6h276c3.314 0 6 2.683 6 5.996V428.43z"></path></svg> A good short introduction to data wrangling with the `tidyverse` is available [here](https://cran.r-project.org/web/packages/dplyr/vignettes/dplyr.html).

---
# Exporting data

R has its own format `.RData`, which is great, but not for people who don`t use R.

``` r
save(dt, file='./data_out/mydata.RData')
```
Note that we import `.RData` files with `load(file = "./pathtoyour/mydata.RData")`.

Saving as a `csv` is usually a good idea, especially if you work with others. We can also save `sav` and `dta` files with the `haven` package.

``` r
write.csv (dt, './data_out/mydata.csv')
write_sav(df, "./data_out/mydata.sav")
```

---
# How to get in touch?