The previous chapter introduced R and gave you enough background to do some simple computations on data sets. This chapter focuses on the foundational knowledge and skills you’ll need in order to use R effectively in the long term. Specifically, it’s a deep dive into R’s various data structures and data types.
13.1 Vectors
A vector is a collection of values. Vectors are the fundamental unit of data in R, and you’ve already used them in the previous sections.
For instance, each column in a data frame is a vector. So the site_name column in the California least terns data set (Section 12.8) is a vector. Take a look at it now. You can use head to avoid printing too much. Set the second argument to 10 so that exactly 10 values are printed:
head(terns$site_name, 10)
[1] "PITTSBURG POWER PLANT"
[2] "ALBANY CENTRAL AVE"
[3] "ALAMEDA POINT"
[4] "KETTLEMAN CITY"
[5] "OCEANO DUNES STATE VEHICULAR RECREATION AREA"
[6] "RANCHO GUADALUPE DUNES PRESERVE"
[7] "VANDENBERG SFB"
[8] "SANTA CLARA RIVER MCGRATH STATE BEACH"
[9] "ORMOND BEACH"
[10] "NBVC POINT MUGU"
Like all vectors, this vector is ordered, which just means the values, or elements, have specific positions. The value of the 1st element is PITTSBURG POWER PLANT, the 2nd is ALBANY CENTRAL AVE, the 5th is OCEANO DUNES STATE VEHICULAR RECREATION AREA, and so on.
Notice that the elements of this vector are all strings. In R, the elements of a vector must all be the same type of data (we say the elements are homogeneous). A vector can contain integers, decimal numbers, strings, or any of several other types of data, but not a mix these all at once.
The other columns in the least terns data frame are also vectors. For instance, the year column is a vector of integers:
head(terns$year)
[1] 2000 2000 2000 2000 2000 2000
Vectors can contain any number of elements, including 0 or 1 element. Unlike mathematics, R does not distinguish between vectors and scalars (solitary values). As far as R is concerned, a solitary value, like 3, is a vector with 1 element.
You can check the length of a vector (and other objects) with the length function:
length(3)
[1] 1
length("hello")
[1] 1
length(terns$year)
[1] 791
Since the last of these is a column from the data frame terns, the length is the same as the number of rows in terns.
13.1.1 Creating Vectors
Sometimes you’ll want to create your own vectors. You can do this by concatenating several vectors together with the c function. It accepts any number of vector arguments, and combines them into a single vector:
c(1, 2, 19, -3)
[1] 1 2 19 -3
c("hi", "hello")
[1] "hi" "hello"
c(1, 2, c(3, 4))
[1] 1 2 3 4
If the arguments you pass to the c function have different data types, R will attempt to convert them to a common data type that preserves the information:
c(1, "cool", 2.3)
[1] "1" "cool" "2.3"
Section 13.2.2 explains the rules for this conversion in more detail.
The colon operator : creates vectors that contain sequences of integers. This is useful for creating “toy” data to test things on, and later we’ll see that it’s also important in several other contexts. Here are a few different sequences:
1:3
[1] 1 2 3
-3:5
[1] -3 -2 -1 0 1 2 3 4 5
10:1
[1] 10 9 8 7 6 5 4 3 2 1
Beware that both endpoints are included in the sequence, even in sequences like 1:0, and that the difference between elements is always -1 or 1. If you want more control over the generated sequence, use the seq function instead.
13.1.2 Indexing Vectors
You can access individual elements of a vector with the indexing operator[ (also called the square bracket operator). The syntax is:
VECTOR[INDEXES]
Here INDEXES is a vector of positions of elements you want to get or set.
For example, let’s make a vector with 5 elements and get the 2nd element:
x =c(4, 8, 3, 2, 1)x[2]
[1] 8
Now let’s get the 3rd and 1st element:
x[c(3, 1)]
[1] 3 4
You can use the indexing operator together with the assignment operator to assign elements of a vector:
x[3] =0x
[1] 4 8 0 2 1
Indexing is among the most frequently used operations in R, so take some time to try it out with few different vectors and indexes. We’ll revisit indexing in Section 14.1 to learn a lot more about it.
13.1.3 Vectorization
Let’s look at what happens if we call a mathematical function, like sin, on a vector:
Of course, the first version is much easier to type.
Functions that take a vector argument and get applied element-by-element like this are said to be vectorized. Most functions in R are vectorized, especially math functions. Some examples include sin, cos, tan, log, exp, and sqrt.
Functions that are not vectorized tend to be ones that combine or aggregate values in some way. For instance, the sum, mean, median, length, and class functions are not vectorized.
A function can be vectorized across multiple arguments. This is easiest to understand in terms of the arithmetic operators. Let’s see what happens if we add two vectors together:
x =c(1, 2, 3, 4)y =c(-1, 7, 10, -10)x + y
[1] 0 9 13 -6
The elements are paired up and added according to their positions. The other arithmetic operators are also vectorized:
x - y
[1] 2 -5 -7 14
x * y
[1] -1 14 30 -40
x / y
[1] -1.0000000 0.2857143 0.3000000 -0.4000000
13.1.4 Recycling
When a function is vectorized across multiple arguments, what happens if the vectors have different lengths? Whenever you think of a question like this as you’re learning R, the best way to find out is to create some toy data and test it yourself. Let’s try that now:
x =c(1, 2, 3, 4)y =c(-1, 1)x + y
[1] 0 3 2 5
The elements of the shorter vector are recycled to match the length of the longer vector. That is, after the second element, the elements of y are repeated to make a vector with the same length as x (because x is longer), and then vectorized addition is carried out as usual.
Here’s what that looks like written down:
1 2 3 4
+ -1 1 -1 1
-----------
0 3 2 5
If the length of the longer vector is not a multiple of the length of the shorter vector, R issues a warning, but still returns the result. The warning as meant as a reminder, because unintended recycling is a common source of bugs:
x =c(1, 2, 3, 4, 5)y =c(-1, 1)x + y
Warning in x + y: longer object length is not a multiple of shorter object
length
[1] 0 3 2 5 4
Recycling might seem strange at first, but it’s convenient if you want to use a specific value (or pattern of values) with a vector. For instance, suppose you want to multiply all the elements of a vector by 2. Recycling makes this easy:
2*c(1, 2, 3)
[1] 2 4 6
When you use recycling, most of the time one of the arguments will be a scalar like this.
13.2 Data Types & Classes
Data can be categorized into different types based on sets of shared characteristics. For instance, statisticians tend to think about whether data are numeric or categorical:
numeric
continuous (real or complex numbers)
discrete (integers)
categorical
nominal (categories with no ordering)
ordinal (categories with some ordering)
Of course, other types of data, like graphs (networks) and natural language (books, speech, and so on), are also possible. Categorizing data this way is useful for reasoning about which methods to apply to which data.
In R, data objects are categorized in two different ways:
The class of an R object describes what the object does, or the role that it plays. Sometimes objects can do more than one thing, so objects can have more than one class. The class function, which debuted in Section 12.9, returns the classes of its argument.
The type of an R object describes what the object is. Technically, the type corresponds to how the object is stored in your computer’s memory. Each object has exactly one type. The typeof function returns the type of its argument.
Of the two, classes tend to be more important than types. If you aren’t sure what an object is, checking its classes should be the first thing you do.
The built-in classes you’ll use all the time correspond to vectors and lists (which we’ll learn more about in Section 13.2.1):
Class
Example
Description
logical
TRUE, FALSE
Logical (or Boolean) values
integer
-1L, 1L, 2L
Integer numbers
numeric
-2.1, 7, 34.2
Real numbers
complex
3-2i, -8+0i
Complex numbers
character
"hi", "YAY"
Text strings
list
list(TRUE, 1, "hi")
Ordered collection of heterogeneous elements
R doesn’t distinguish between scalars and vectors, so the class of a vector is the same as the class of its elements:
class("hi")
[1] "character"
class(c("hello", "hi"))
[1] "character"
In addition, for most vectors, the class and the type are the same:
x =c(TRUE, FALSE)class(x)
[1] "logical"
typeof(x)
[1] "logical"
The exception to this rule is numeric vectors, which have type double for historical reasons:
By default, R assumes any numbers you enter in code are numeric, even if they’re integer-valued.
The class integer also represents integer numbers, but it’s not used as often as numeric. A few functions, such as the sequence operator : and the length function, return integers. You can also force R to create an integer by adding the suffix L to a number, but there are no major drawbacks to using the double default:
class(1:3)
[1] "integer"
class(3)
[1] "numeric"
class(3L)
[1] "integer"
Besides the classes for vectors and lists, there are several built-in classes that represent more sophisticated data structures:
Class
Description
function
Functions
factor
Categorical values
matrix
Two-dimensional ordered collection of homogeneous elements
array
Multi-dimensional ordered collection of homogeneous elements
data.frame
Data frames
For these, the class is usually different from the type. We’ll learn more about most of these later on.
13.2.1 Lists
A list is an ordered data structure where the elements can have different types (they are heterogeneous). This differs from a vector, where the elements all have to have the same type, as we saw in Section 13.1. The tradeoff is that most vectorized functions do not work with lists.
You can make an ordinary list with the list function:
x =list(1, c("hi", "bye"))class(x)
[1] "list"
typeof(x)
[1] "list"
For ordinary lists, the type and the class are both list. In Section 14.1, we’ll learn how to get and set list elements, and in later sections we’ll learn more about when and why to use lists.
You’ve already seen one list, the terns data frame:
class(terns)
[1] "data.frame"
typeof(terns)
[1] "list"
Under the hood, data frames are lists, and each column is a list element. Because the class is data.frame rather than list, R treats data frames differently from ordinary lists. This difference is apparent in how data frames are printed compared to ordinary lists.
13.2.2 Implicit Coercion
R’s types fall into a natural hierarchy of expressiveness:
R’s hierarchy of types.
Each type on the right is more expressive than the ones to its left. That is, with the convention that FALSE is 0 and TRUE is 1, we can represent any logical value as an integer. In turn, we can represent any integer as a double, and any double as a complex number. By writing the number out, we can also represent any complex number as a string.
The point is that no information is lost as we follow the arrows from left to right along the types in the hierarchy. In fact, R will automatically and silently convert from types on the left to types on the right as needed. This is called implicit coercion.
As an example, consider what happens if we add a logical value to a number:
TRUE+2
[1] 3
R automatically converts the TRUE to the numeric value 1, and then carries out the arithmetic as usual.
We’ve already seen implicit coercion at work once before, when we learned the c function. Since the elements of a vector all have to have the same type, if you pass several different types to c, then R tries to use implicit coercion to make them the same:
x =c(TRUE, "hi", 1, 1+3i)class(x)
[1] "character"
x
[1] "TRUE" "hi" "1" "1+3i"
Implicit coercion is strictly one-way; it never occurs in the other direction. If you want to coerce a type on the right to one on the left, you can do it explicitly with one of the as.TYPE functions. For instance, the as.numeric (or as.double) function coerces to numeric:
as.numeric("3.1")
[1] 3.1
There are a few types that fall outside of the hierarchy entirely, like functions. Implicit coercion doesn’t apply to these. If you try to use these types where it doesn’t make sense to, R generally returns an error:
sin +3
Error in sin + 3: non-numeric argument to binary operator
If you try to use these types as elements of a vector, you get back a list instead:
x =c(1, 2, sum)class(x)
[1] "list"
Understanding how implicit coercion works will help you avoid bugs, and can also be a time-saver. For example, we can use implicit coercion to succinctly count how many elements of a vector satisfy a some condition:
x =c(1, 3, -1, 10, -2, 3, 8, 2)condition = x <4sum(condition) # or sum(x < 4)
[1] 6
If you still don’t quite understand how the code above works, try inspecting each variable. In general, inspecting each step or variable is a good strategy for understanding why a piece of code works (or doesn’t work!). Here the implicit coercion happens in the third line.
13.2.3 Matrices & Arrays
A matrix is the two-dimensional analogue of a vector. The elements, which are arranged into rows and columns, are ordered and homogeneous.
You can create a matrix from a vector with the matrix function. By default, the columns are filled first:
# A matrix with 2 rows and 3 columns:matrix(1:6, 2, 3)
[,1] [,2] [,3]
[1,] 1 3 5
[2,] 2 4 6
The class of a matrix is always matrix, and the type matches the type of the elements:
x =matrix(c("a", "b", NA, "c"), 2, 2)x
[,1] [,2]
[1,] "a" NA
[2,] "b" "c"
class(x)
[1] "matrix" "array"
typeof(x)
[1] "character"
You can use the matrix multiplication operator %*% to multiply two matrices with compatible dimensions.
An array is a further generalization of matrices to higher dimensions. You can create an array from a vector with the array function. The characteristics of arrays are almost identical to matrices, but the class of an array is always array.
13.2.4 Factors
A feature is categorical if it measures a qualitative category. For example, the genres rock, blues, alternative, folk, pop are categories.
R uses the class factor to represent categorical data. Visualizations and statistical models sometimes treat factors differently than other data types, so it’s important to make sure you have the right data type. If you’re ever unsure, remember that you can check the class of an object with the class function.
When it reads a data set, R usually can’t tell which features are categorical. That means identifying and converting the categorical features is up to you. For beginners, it can be difficult to understand whether a feature is categorical or not. The key is to think about whether you want to use the feature to divide the data into groups.
For example, if you want to know how many songs are in the rock genre, you first need to divide the songs by genre, and then count the number of songs in each group (or at least the rock group).
As a second example, months recorded as numbers can be categorical or not, depending on how you want to use them. You might want to treat them as categorical (for example, to compute max rainfall in each month) or you might want to treat them as numbers (for example, to compute the number of months time between two events).
The bottom line is that you have to think about what you’ll be doing in the analysis. In some cases, you might treat a feature as categorical only for part of the analysis.
Let’s think about which features are categorical in least terns data set. To refresh your memory of what’s in the data set, take a look at the structural summary:
str(terns)
'data.frame': 791 obs. of 43 variables:
$ year : int 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 ...
$ site_name : chr "PITTSBURG POWER PLANT" "ALBANY CENTRAL AVE" "ALAMEDA POINT" "KETTLEMAN CITY" ...
$ site_name_2013_2018: chr "Pittsburg Power Plant" "NA_NO POLYGON" "Alameda Point" "Kettleman" ...
$ site_name_1988_2001: chr "NA_2013_2018 POLYGON" "Albany Central Avenue" "NA_2013_2018 POLYGON" "NA_2013_2018 POLYGON" ...
$ site_abbr : chr "PITT_POWER" "AL_CENTAVE" "ALAM_PT" "KET_CTY" ...
$ region_3 : chr "S.F._BAY" "S.F._BAY" "S.F._BAY" "KINGS" ...
$ region_4 : chr "S.F._BAY" "S.F._BAY" "S.F._BAY" "KINGS" ...
$ event : chr "LA_NINA" "LA_NINA" "LA_NINA" "LA_NINA" ...
$ bp_min : num 15 6 282 2 4 9 30 21 73 166 ...
$ bp_max : num 15 12 301 3 5 9 32 21 73 167 ...
$ fl_min : int 16 1 200 1 4 17 11 9 60 64 ...
$ fl_max : int 18 1 230 2 4 17 11 9 65 64 ...
$ total_nests : int 15 20 312 3 5 9 32 22 73 252 ...
$ nonpred_eggs : int 3 NA 124 NA 2 0 NA 4 2 NA ...
$ nonpred_chicks : int 0 NA 81 3 0 1 27 3 0 NA ...
$ nonpred_fl : int 0 NA 2 1 0 0 0 NA 0 NA ...
$ nonpred_ad : int 0 NA 1 6 0 0 0 NA 0 NA ...
$ pred_control : chr "" "" "" "" ...
$ pred_eggs : int 4 NA 17 NA 0 NA 0 NA NA NA ...
$ pred_chicks : int 2 NA 0 NA 4 NA 3 NA NA NA ...
$ pred_fl : int 0 NA 0 NA 0 NA 0 NA NA NA ...
$ pred_ad : int 0 NA 0 NA 0 NA 0 NA NA NA ...
$ pred_pefa : chr "N" "" "N" "" ...
$ pred_coy_fox : chr "N" "" "N" "" ...
$ pred_meso : chr "N" "" "N" "" ...
$ pred_owlspp : chr "N" "" "N" "" ...
$ pred_corvid : chr "Y" "" "N" "" ...
$ pred_other_raptor : chr "Y" "" "Y" "" ...
$ pred_other_avian : chr "N" "" "Y" "" ...
$ pred_misc : chr "N" "" "N" "" ...
$ total_pefa : int 0 NA 0 NA 0 NA 0 NA NA NA ...
$ total_coy_fox : int 0 NA 0 NA 0 NA 0 NA NA NA ...
$ total_meso : int 0 NA 0 NA 0 NA 0 NA NA NA ...
$ total_owlspp : int 0 NA 0 NA 0 NA 0 NA NA NA ...
$ total_corvid : int 4 NA 0 NA 0 NA 0 NA NA NA ...
$ total_other_raptor : int 2 NA 6 NA 0 NA 3 NA NA NA ...
$ total_other_avian : int 0 NA 11 NA 4 NA 0 NA NA NA ...
$ total_misc : int 0 NA 0 NA 0 NA 0 NA NA NA ...
$ first_observed : chr "2000-05-11" "" "2000-05-01" "2000-06-10" ...
$ last_observed : chr "2000-08-05" "" "2000-08-19" "2000-09-24" ...
$ first_nest : chr "2000-05-26" "" "2000-05-16" "2000-06-17" ...
$ first_chick : chr "2000-06-18" "" "2000-06-07" "2000-07-22" ...
$ first_fledge : chr "2000-07-08" "" "2000-06-30" "2000-08-06" ...
The site_name, site_abbr, and event columns are all examples of categorical data. The region_ columns and some of the pred_ columns also contain categorical data.
One way to check whether a feature is useful for grouping (and thus effectively categorical) is to count the number of times each value appears. You can do this with the table function. For instance, to count the number of times each category of event appears:
table(terns$event)
EL_NINO LA_NINA NEUTRAL
120 258 413
Features with only a few unique values, repeated many times, are ideal for grouping. Numerical features, like total_nests, usually aren’t good for grouping, both because of what they measure and because they tend to have many unique values, which leads to very small groups.
The year column can be treated as categorical or quantitative data. It’s easy to imagine grouping observations by year, but years are also numerical: they have an order and we might want to do math on them. The most appropriate type for year depends on how we want to use it for analysis.
You can convert a column to the factor class with the factor function. Try this for the event column:
This is an important way factors are different from strings (or character vectors). It ensures that if you plot a factor, the missing levels will still be represented on the plot.
Tip
You can make a factor forget levels that aren’t present with the droplevels function:
droplevels(event[1:3])
[1] LA_NINA LA_NINA LA_NINA
Levels: LA_NINA
13.3 Special Values
R has four special values to represent missing or invalid data.
13.3.1 Missing Values
The value NA, called the missing value, represents missing entries in a data set. It’s implied that the entries are missing due to how the data was collected, although there are exceptions. As an example, imagine the data came from a survey, and respondents chose not to answer some questions. In the data set, their answers for those questions can be recorded as NA.
The missing value is a chameleon: it can be a logical, integer, numeric, complex, or character value. By default, the missing value is logical, and the other types occur through coercion (Section 13.2.2):
class(NA)
[1] "logical"
class(c(1, NA))
[1] "numeric"
class(c("hi", NA, NA))
[1] "character"
The missing value is also contagious: it represents an unknown quantity, so using it as an argument to a function usually produces another missing value. The idea is that if the inputs to a computation are unknown, generally so is the output:
NA-3
[1] NA
mean(c(1, 2, NA))
[1] NA
As a consequence, testing whether an object is equal to the missing value with == doesn’t return a meaningful result:
5==NA
[1] NA
NA==NA
[1] NA
You can use the is.na function instead:
is.na(5)
[1] FALSE
is.na(NA)
[1] TRUE
is.na(c(1, NA, 3))
[1] FALSE TRUE FALSE
Missing values are a feature that sets R apart from most other programming languages.
13.3.2 Infinity
The value Inf represents infinity, and can be numeric or complex. You’re most likely to encounter it as the result of certain computations:
13/0
[1] Inf
class(Inf)
[1] "numeric"
You can use the is.infinite function to test whether a value is infinite:
is.infinite(3)
[1] FALSE
is.infinite(c(-Inf, 0, Inf))
[1] TRUE FALSE TRUE
13.3.3 Not a Number
The value NaN (“not a number”) represents a quantity that’s undefined mathematically. For instance, dividing 0 by 0 is undefined:
0/0
[1] NaN
class(NaN)
[1] "numeric"
Like Inf, NaN can be numeric or complex.
You can use the is.nan function to test whether a value is NaN:
is.nan(c(10.1, log(-1), 3))
Warning in log(-1): NaNs produced
[1] FALSE TRUE FALSE
13.3.4 Null
The value NULL represents a quantity that’s undefined in R. Most of the time, NULL indicates the absence of a result. For instance, vectors don’t have dimensions, so the dim function returns NULL for vectors:
dim(c(1, 2))
NULL
class(NULL)
[1] "NULL"
typeof(NULL)
[1] "NULL"
Unlike the other special values, NULL has its own unique type and class.
You can use the is.null function to test whether a value is NULL:
