17  Aggregation & Grouping

NoteLearning Goals

After this lesson, you should be able to:

  • Explain what a map (or apply) function does
  • Call a function repeatedly with the purrr package’s map functions
  • Split data into groups and apply a function to each
  • Compute group statistics with the dplyr package’s group_by and summarize functions
ImportantRequired Packages

This chapter uses the following packages:

  • dplyr
  • purrr

Chapter 14 explains how to install and load packages.

17.1 The map Function

Section 13.1.3 introduced vectorization, a convenient and efficient way to compute multiple results. That section also mentioned that some of R’s functions—the ones that summarize or aggregate data—are not vectorized.

The class function is an example of a function that’s not vectorized. If we call the class function on the least terns data set, we get just one result for the data set as a whole:

class(terns)
[1] "data.frame"

What if we want to get the class of each column? We can get the class for a single column by selecting the column with $, the dollar sign operator:

class(terns$year)
[1] "integer"

But what if we want the classes for all the columns? We could write a call to class for each column, but that would be tedious. When you’re working with a programming language, you should try to avoid tedium; there’s usually a better, more automated way.

Section 13.2.1 pointed out that data frames are technically lists, where each column is one element. With that in mind, what we need here is a line of code that calls class on each element of the data frame. The idea is similar to vectorization, but since we have a list and a non-vectorized function, we have to do a bit more than just call class(terns).

The purrr package is a collection of functions to help you do things repeatedly or for each element of a data structure. Install and load the package in order to follow along:

# install.packages("purrr")
library("purrr")

The package’s map function calls a function on each element of a vector or list. We sometimes also say it maps or applies a function over the elements. The syntax is:

map(DATA, F, ...)

The map function calls the function F once for each element of DATA. It passes the element to F as the first argument. It also passes the ... arguments to F, which are constant across all of the calls.

Let’s try this out with the least terns data and the class function:

map(terns, class)
$year
[1] "integer"

$site_name
[1] "character"

$site_name_2013_2018
[1] "character"

$site_name_1988_2001
[1] "character"

$site_abbr
[1] "character"

$region_3
[1] "character"

$region_4
[1] "character"

$event
[1] "character"

$bp_min
[1] "numeric"

$bp_max
[1] "numeric"

$fl_min
[1] "integer"

$fl_max
[1] "integer"

$total_nests
[1] "integer"

$nonpred_eggs
[1] "integer"

$nonpred_chicks
[1] "integer"

$nonpred_fl
[1] "integer"

$nonpred_ad
[1] "integer"

$pred_control
[1] "character"

$pred_eggs
[1] "integer"

$pred_chicks
[1] "integer"

$pred_fl
[1] "integer"

$pred_ad
[1] "integer"

$pred_pefa
[1] "character"

$pred_coy_fox
[1] "character"

$pred_meso
[1] "character"

$pred_owlspp
[1] "character"

$pred_corvid
[1] "character"

$pred_other_raptor
[1] "character"

$pred_other_avian
[1] "character"

$pred_misc
[1] "character"

$total_pefa
[1] "integer"

$total_coy_fox
[1] "integer"

$total_meso
[1] "integer"

$total_owlspp
[1] "integer"

$total_corvid
[1] "integer"

$total_other_raptor
[1] "integer"

$total_other_avian
[1] "integer"

$total_misc
[1] "integer"

$first_observed
[1] "character"

$last_observed
[1] "character"

$first_nest
[1] "character"

$first_chick
[1] "character"

$first_fledge
[1] "character"

The result is similar to if the class function was vectorized. In fact, if we use a vector and a vectorized function with map, the result is nearly identical to the result from vectorization:

x = c(1, 2, pi)

sin(x)
[1] 8.414710e-01 9.092974e-01 1.224647e-16
map(x, sin)
[[1]]
[1] 0.841471

[[2]]
[1] 0.9092974

[[3]]
[1] 1.224647e-16

The only difference is that the result from map is a list. In fact, the map function always returns a list with one element for each element of the input data.

17.2 Other Map Functions

The purrr package provides many different map functions, all of which have names that start with map. All of them call another function on each element of a data structure. They also all have the same syntax. Where they differ is in how they return results. A few of these are shown in Table 17.1.

Table 17.1
Function Return Type
map_lgl logical
map_int integer
map_dbl numeric (double)
map_chr character
map list
NoteApply Functions

R’s apply functions are a built-in equivalent to map functions. The lapply, sapply, and tapply functions are the three most important functions in the family of apply functions, but there are many more. The lapply function is nearly identical to the map function.

We focus on and recommend the map functions rather than the apply functions because they are more consistent in their syntax and specific in their return types. You can learn more about R’s apply functions by reading this StackOverflow post.

When you have a choice between using vectorization or a map function, you should always choose vectorization. Vectorization is clearer—compare the two lines of code above—and it’s also significantly more efficient. In fact, vectorization is the most efficient way to call a function repeatedly in R.

As we saw with the class function, there are some situations where vectorization is not possible. That’s when you should think about using a map function.

Let’s look at some examples of the other map functions. If we use map_chr to find the classes of the columns in the least terns data, we get a character vector:

map_chr(terns, class)
               year           site_name site_name_2013_2018 site_name_1988_2001 
          "integer"         "character"         "character"         "character" 
          site_abbr            region_3            region_4               event 
        "character"         "character"         "character"         "character" 
             bp_min              bp_max              fl_min              fl_max 
          "numeric"           "numeric"           "integer"           "integer" 
        total_nests        nonpred_eggs      nonpred_chicks          nonpred_fl 
          "integer"           "integer"           "integer"           "integer" 
         nonpred_ad        pred_control           pred_eggs         pred_chicks 
          "integer"         "character"           "integer"           "integer" 
            pred_fl             pred_ad           pred_pefa        pred_coy_fox 
          "integer"           "integer"         "character"         "character" 
          pred_meso         pred_owlspp         pred_corvid   pred_other_raptor 
        "character"         "character"         "character"         "character" 
   pred_other_avian           pred_misc          total_pefa       total_coy_fox 
        "character"         "character"           "integer"           "integer" 
         total_meso        total_owlspp        total_corvid  total_other_raptor 
          "integer"           "integer"           "integer"           "integer" 
  total_other_avian          total_misc      first_observed       last_observed 
          "integer"           "integer"         "character"         "character" 
         first_nest         first_chick        first_fledge 
        "character"         "character"         "character" 

Likewise, if we use map_dbl to compute the sin values, we get a numeric vector, the same as from vectorization:

map_dbl(x, sin)
[1] 8.414710e-01 9.092974e-01 1.224647e-16

In spite of that, vectorization is still more efficient than sapply, so use vectorization instead when possible.

Map functions are incredibly useful for summarizing data. For example, suppose we want to compute the frequencies for all of the columns in the least terns data set that aren’t numeric.

First, we need to identify the columns. One way to do this is with the is.numeric function. Despite the name, this function actually tests whether its argument is a real number, not whether it its argument is a numeric vector. In other words, it also returns true for integer values. We can use map_lgl to apply this function to all of the columns in the least terns data set:

is_not_number = !map_lgl(terns, is.numeric)
is_not_number
               year           site_name site_name_2013_2018 site_name_1988_2001 
              FALSE                TRUE                TRUE                TRUE 
          site_abbr            region_3            region_4               event 
               TRUE                TRUE                TRUE                TRUE 
             bp_min              bp_max              fl_min              fl_max 
              FALSE               FALSE               FALSE               FALSE 
        total_nests        nonpred_eggs      nonpred_chicks          nonpred_fl 
              FALSE               FALSE               FALSE               FALSE 
         nonpred_ad        pred_control           pred_eggs         pred_chicks 
              FALSE                TRUE               FALSE               FALSE 
            pred_fl             pred_ad           pred_pefa        pred_coy_fox 
              FALSE               FALSE                TRUE                TRUE 
          pred_meso         pred_owlspp         pred_corvid   pred_other_raptor 
               TRUE                TRUE                TRUE                TRUE 
   pred_other_avian           pred_misc          total_pefa       total_coy_fox 
               TRUE                TRUE               FALSE               FALSE 
         total_meso        total_owlspp        total_corvid  total_other_raptor 
              FALSE               FALSE               FALSE               FALSE 
  total_other_avian          total_misc      first_observed       last_observed 
              FALSE               FALSE                TRUE                TRUE 
         first_nest         first_chick        first_fledge 
               TRUE                TRUE                TRUE 

Is it worth using R code to identify the non-numeric columns? Since there are only 43 columns in the least terns data set, maybe not. But if the data set was larger, with say 100 columns, it definitely would be.

In general, it’s a good habit to use R to do things rather than do them manually. You’ll get more practice programming, and your code will be more flexible if you want to adapt it to other data sets.

Now that we know which columns are non-numeric, we can use the table function to compute frequencies. We only want to compute frequencies for those columns, so we need to subset the data:

map(terns[, is_not_number], table)
$site_name

                               ALAMEDA POINT 
                                          21 
                          ALBANY CENTRAL AVE 
                                           2 
                                ANAHEIM LAKE 
                                           3 
                            ARIZONA GLENDALE 
                                           1 
        BATIQUITOS LAGOON ECOLOGICAL RESERVE 
                                          21 
              BOLSA CHICA ECOLOGICAL RESERVE 
                                          21 
                               BURRIS ISLAND 
                                          18 
                CHULA VISTA WILDLIFE RESERVE 
                                          21 
                      COAL OIL POINT RESERVE 
                                          14 
           DSTREET FILL SWEETWATER MARSH NWR 
                                          21 
             EDEN LANDING ECOLOGICAL RESERVE 
                                          17 
                             FAIRBANKS RANCH 
                                           9 
                  GUADALUPE NIPOMO DUNES NWR 
                                           1 
                  HAYWARD REGIONAL SHORELINE 
                                          19 
                             HOLLYWOOD BEACH 
                                          18 
                      HUNTINGTON STATE BEACH 
                                          21 
                              KETTLEMAN CITY 
                                          15 
                                   LA HARBOR 
                                          21 
                               MALIBU LAGOON 
                                           7 
                          MCB CAMP PENDLETON 
                                          21 
                      MISSION BAY FAA ISLAND 
                                          21 
                  MISSION BAY MARINERS POINT 
                                          21 
             MISSION BAY NORTH FIESTA ISLAND 
                                          21 
           MISSION BAY SAN DIEGO RIVER MOUTH 
                                          14 
                     MISSION BAY STONY POINT 
                                          18 
                          MONTEZUMA WETLANDS 
                                          18 
             NAPA SONOMA MARSH WILDLIFE AREA 
                                          17 
                            NAS NORTH ISLAND 
                                          21 
              NAVAL AMPHIBIOUS BASE CORONADO 
                                          21 
                             NBVC POINT MUGU 
                                          21 
OCEANO DUNES STATE VEHICULAR RECREATION AREA 
                                          21 
                                ORMOND BEACH 
                                          21 
                       PITTSBURG POWER PLANT 
                                          15 
             RANCHO GUADALUPE DUNES PRESERVE 
                                          20 
                      SACRAMENTO BUFFERLANDS 
                                          10 
                                  SALTON SEA 
                                           7 
      SAN DIEGUITO LAGOON ECOLOGICAL RESERVE 
                                          10 
         SAN ELIJO LAGOON ECOLOGICAL RESERVE 
                                          21 
                           SAN PABLO BAY NWR 
                                           3 
       SANTA CLARA RIVER MCGRATH STATE BEACH 
                                          21 
  SATICOY UNITED WATER CONSERVATION DISTRICT 
                                           9 
                        SDIA LINDBERGH FIELD 
                                          21 
                  SEAL BEACH NWR ANAHEIM BAY 
                                          21 
                   SILVER STRAND STATE BEACH 
                                           1 
    SOUTH SAN DIEGO BAY UNIT SDNWR SALTWORKS 
                                          21 
                        TIJUANA ESTUARY NERR 
                                          21 
        UPPER NEWPORT BAY ECOLOGICAL RESERVE 
                                          21 
                              VANDENBERG SFB 
                                          21 
                                VENICE BEACH 
                                          21 

$site_name_2013_2018

                                                          
                                                        3 
                                            Alameda Point 
                                                       21 
                                             Anaheim Lake 
                                                        3 
                                        Batiquitos Lagoon 
                                                       21 
                                              Bolsa Chica 
                                                       21 
                                              Bufferlands 
                                                       10 
                                             Burris Basin 
                                                       18 
                                           Camp Pendleton 
                                                       21 
                              Chula Vista Wildlife Refuge 
                                                       21 
                                   Coal Oil Point Reserve 
                                                       14 
                                            D Street Fill 
                                                       21 
                          Eden Landing Ecological Reserve 
                                                       17 
                                               FAA Island 
                                                       21 
                                          Fairbanks Ranch 
                                                        9 
                               Hayward Regional Shoreline 
                                                       19 
                                          Hollywood Beach 
                                                       18 
                              Huntington Beach State Park 
                                                       21 
                                                Kettleman 
                                                       15 
             Lindbergh Field/Former Naval Training Center 
                                                       21 
                                            Malibu Lagoon 
                                                        7 
                                          Mariner's Point 
                                                       21 
                                                Montezuma 
                                                       18 
                                            NA_NO POLYGON 
                                                        5 
Napa Sonoma Marsh Wildlife Area Huichica Unit (Pond 7/7A) 
                                                       17 
                                      Naval Base Coronado 
                                                       42 
                                          NBVC Point Mugu 
                                                       21 
                                      North Fiesta Island 
                                                       21 
             Oceano Dunes State Vehicular Recreation Area 
                                                       21 
                                             Ormond Beach 
                                                       21 
                                    Pittsburg Power Plant 
                                                       15 
                                               Port of LA 
                                                       21 
                          Rancho Guadalupe Dunes Preserve 
                                                       20 
                                               Salton Sea 
                                                        7 
                                                Saltworks 
                                                       21 
                                    San Diego River Mouth 
                                                       14 
                                      San Diequito Lagoon 
                                                       10 
                                         San Elijo Lagoon 
                                                       21 
                                        Santa Clara River 
                                                       21 
               Saticoy United Water Conservation District 
                                                        9 
                      Seal Beach National Wildlife Refuge 
                                                       21 
                                              Stony Point 
                                                       18 
                                          Tijuana Estuary 
                                                       21 
                                        Upper Newport Bay 
                                                       21 
                                           Vandenberg AFB 
                                                       21 
                                             Venice Beach 
                                                       21 

$site_name_1988_2001

                      Albany Central Avenue  NA_2013_2018 POLYGON 
                    3                     2                   783 
        NA_NO POLYGON 
                    3 

$site_abbr

  AL_CENTAVE      ALAM_PT       ANA_LK     ARZ_GLEN         BCER         BLER 
           2           21            3            1           21           21 
     BUR_ISL     COAL_OIL           CV         D_ST         ELER     FAIR_RAN 
          18           14           21           21           17            9 
     GND_NWR HAY_REG_SHOR           HB          HSB      KET_CTY      LA_HARB 
           1           19           18           21           15           21 
     MAL_LAG       MB_FAA    MB_MAR_PT       MB_NFI   MB_SDRIV_S     MB_STONY 
           7           21           21           21           14           18 
       MCBCP         MONT          NAB        NASNI        NSMWA OCEANO_DUNES 
          21           18           21           21           17           21 
      ORMOND   PITT_POWER      PT_MUGU         RGDP   S_CLAR_MCG      SAC_BUF 
          21           15           21           20           21           10 
        SALT     SALT_SEA    SAN_ELIJO SANDIEGU_LAG      SAT_WCD      SDIA_LF 
          21            7           21           10            9           21 
SEAL_BCH_NWR     SLVRSTRD       SPBNWR       TJ_RIV        UNBER      VAN_SFB 
          21            1            3           21           21           21 
     VEN_BCH 
          21 

$region_3

   ARIZONA    CENTRAL      KINGS   S.F._BAY SACRAMENTO   SOUTHERN 
         1         77         15        112         10        576 

$region_4

   ARIZONA    CENTRAL      KINGS   S.F._BAY SACRAMENTO   SOUTHERN    VENTURA 
         1         77         15        112         10        486         90 

$event

EL_NINO LA_NINA NEUTRAL 
    120     258     413 

$pred_control

      N   Y 
342 134 315 

$pred_pefa

      N   Y 
143 423 225 

$pred_coy_fox

      N   Y 
142 535 114 

$pred_meso

      N   Y 
142 552  97 

$pred_owlspp

      N   Y 
142 531 118 

$pred_corvid

      N   Y 
142 423 226 

$pred_other_raptor

      N  NN   Y 
145 437   1 208 

$pred_other_avian

      N   Y 
143 395 253 

$pred_misc

      N   Y 
159 415 217 

$first_observed

           2000-04-16 2000-04-19 2000-04-21 2000-04-22 2000-04-23 2000-04-26 
       141          1          1          1          1          1          1 
2000-04-28 2000-05-01 2000-05-04 2000-05-06 2000-05-07 2000-05-09 2000-05-11 
         5          1          3          1          2          1          1 
2000-05-21 2000-06-06 2000-06-10 2004-04-08 2004-04-11 2004-04-12 2004-04-21 
         1          1          1          1          1          2          1 
2004-04-22 2004-04-23 2004-04-27 2004-04-30 2004-05-01 2004-05-03 2004-05-07 
         2          1          1          1          1          2          1 
2004-05-09 2004-05-10 2004-05-14 2004-05-17 2004-05-31 2004-06-03 2005-04-14 
         1          1          1          1          1          1          1 
2005-04-15 2005-04-18 2005-04-19 2005-04-20 2005-04-21 2005-04-22 2005-04-23 
         1          2          1          2          2          1          2 
2005-04-24 2005-04-28 2005-04-29 2005-04-30 2005-05-01 2005-05-05 2005-05-07 
         1          2          1          1          1          1          1 
2005-05-08 2005-05-10 2005-05-13 2005-05-27 2005-05-28 2006-04-08 2006-04-10 
         1          1          1          1          1          2          1 
2006-04-13 2006-04-15 2006-04-16 2006-04-20 2006-04-22 2006-04-27 2006-04-28 
         1          1          1          1          3          1          1 
2006-05-01 2006-05-03 2006-05-08 2006-05-10 2006-05-14 2006-05-15 2006-05-20 
         1          2          1          1          2          1          1 
2006-06-23 2006-06-24 2007-04-16 2007-04-18 2007-04-22 2007-04-23 2007-04-24 
         1          1          1          1          1          1          2 
2007-04-25 2007-04-26 2007-04-28 2007-05-01 2007-05-02 2007-05-03 2007-05-11 
         4          2          1          3          3          1          1 
2007-05-13 2007-05-14 2007-05-18 2007-05-24 2007-06-02 2007-06-06 2007-06-08 
         2          1          1          1          1          1          1 
2008-04-11 2008-04-16 2008-04-23 2008-04-24 2008-04-25 2008-04-27 2008-04-28 
         1          2          1          4          1          3          3 
2008-05-01 2008-05-04 2008-05-05 2008-05-07 2008-05-08 2008-05-09 2008-05-12 
         1          1          1          1          1          2          1 
2008-05-14 2008-05-15 2008-05-21 2008-05-22 2008-05-24 2008-05-28 2008-05-31 
         1          1          1          1          1          1          1 
2009-04-16 2009-04-19 2009-04-22 2009-04-23 2009-04-24 2009-04-25 2009-04-26 
         2          1          1          4          2          1          3 
2009-04-29 2009-04-30 2009-05-02 2009-05-03 2009-05-04 2009-05-05 2009-05-06 
         1          1          1          2          1          1          1 
2009-05-08 2009-05-11 2009-05-13 2009-05-28 2010-04-14 2010-04-16 2010-04-17 
         3          1          2          1          1          2          1 
2010-04-18 2010-04-19 2010-04-21 2010-04-22 2010-04-25 2010-04-27 2010-04-28 
         2          1          1          2          2          1          1 
2010-04-29 2010-05-01 2010-05-02 2010-05-04 2010-05-06 2010-05-07 2010-05-13 
         1          1          1          1          1          1          2 
2010-05-14 2010-05-15 2010-05-16 2010-05-26 2010-05-29 2010-05-30 2010-06-09 
         2          4          1          1          1          1          1 
2011-04-09 2011-04-15 2011-04-18 2011-04-20 2011-04-21 2011-04-22 2011-04-24 
         1          1          1          1          5          2          1 
2011-04-25 2011-04-27 2011-04-28 2011-04-29 2011-05-01 2011-05-04 2011-05-05 
         1          1          1          1          1          1          2 
2011-05-07 2011-05-08 2011-05-11 2011-05-12 2011-05-14 2011-05-28 2011-06-11 
         1          1          1          1          3          1          1 
2011-06-14 2011-06-25 2012-04-14 2012-04-15 2012-04-17 2012-04-18 2012-04-19 
         1          1          2          1          2          3          5 
2012-04-20 2012-04-21 2012-04-22 2012-04-25 2012-04-29 2012-05-01 2012-05-03 
         1          2          2          1          2          1          1 
2012-05-06 2012-05-08 2012-05-10 2012-05-11 2012-05-13 2012-05-14 2012-05-23 
         1          1          3          1          1          1          1 
2012-05-30 2012-06-09 2013-04-14 2013-04-15 2013-04-18 2013-04-21 2013-04-24 
         1          1          1          1          1          1          2 
2013-04-25 2013-04-29 2013-04-30 2013-05-01 2013-05-02 2013-05-03 2013-05-07 
         1          1          1          2          2          3          1 
2013-05-10 2013-05-11 2013-05-12 2013-05-13 2013-05-15 2013-05-17 2013-05-19 
         1          1          1          1          1          1          1 
2013-05-27 2013-06-01 2013-06-09 2013-06-30 2013-07-19 2014-04-15 2014-04-17 
         1          2          1          1          1          2          5 
2014-04-19 2014-04-21 2014-04-24 2014-04-25 2014-04-26 2014-04-30 2014-05-01 
         1          1          1          2          1          1          1 
2014-05-02 2014-05-04 2014-05-08 2014-05-09 2014-05-10 2014-05-11 2014-05-16 
         1          2          1          2          2          1          1 
2014-05-22 2014-05-24 2014-05-25 2014-05-31 2015-04-06 2015-04-15 2015-04-17 
         1          2          1          1          1          2          2 
2015-04-18 2015-04-19 2015-04-22 2015-04-23 2015-04-27 2015-04-29 2015-04-30 
         1          3          3          1          1          2          3 
2015-05-01 2015-05-07 2015-05-08 2015-05-09 2015-05-14 2015-05-17 2015-05-20 
         3          1          2          1          2          1          1 
2015-05-30 2015-06-05 2015-07-05 2015-07-07 2015-07-22 2016-04-10 2016-04-11 
         1          1          1          1          1          2          1 
2016-04-12 2016-04-13 2016-04-14 2016-04-15 2016-04-18 2016-04-20 2016-04-21 
         1          1          2          1          2          1          1 
2016-04-23 2016-04-26 2016-04-30 2016-05-01 2016-05-02 2016-05-03 2016-05-04 
         1          1          2          1          1          2          1 
2016-05-06 2016-05-07 2016-05-11 2016-05-12 2016-05-14 2016-05-18 2016-05-25 
         2          2          2          1          1          2          1 
2016-05-31 2016-06-12 2016-06-16 2016-06-18 2016-07-15 2017-04-13 2017-04-14 
         1          1          1          2          1          2          1 
2017-04-15 2017-04-16 2017-04-17 2017-04-18 2017-04-19 2017-04-20 2017-04-21 
         2          1          2          2          1          1          1 
2017-04-22 2017-04-24 2017-04-26 2017-04-29 2017-05-04 2017-05-06 2017-05-07 
         1          1          1          1          2          3          2 
2017-05-10 2017-05-11 2017-05-12 2017-05-19 2017-05-25 2017-06-03 2017-07-02 
         3          2          1          1          2          1          1 
2017-07-04 2018-03-25 2018-03-30 2018-04-13 2018-04-18 2018-04-19 2018-04-20 
         1          1          1          2          1          2          3 
2018-04-21 2018-04-25 2018-04-26 2018-04-27 2018-04-28 2018-04-29 2018-05-04 
         2          2          1          1          4          2          1 
2018-05-05 2018-05-10 2018-05-11 2018-05-13 2018-05-16 2018-05-20 2018-05-21 
         1          2          1          1          1          1          1 
2019-04-06 2019-04-12 2019-04-14 2019-04-17 2019-04-18 2019-04-19 2019-04-20 
         1          3          1          1          1          3          2 
2019-04-22 2019-04-24 2019-04-26 2019-04-27 2019-05-01 2019-05-02 2019-05-03 
         1          1          1          4          1          1          2 
2019-05-06 2019-05-08 2019-05-09 2019-05-10 2019-05-11 2019-05-12 2020-04-06 
         2          1          2          1          3          1          1 
2020-04-10 2020-04-11 2020-04-17 2020-04-18 2020-04-20 2020-04-21 2020-04-23 
         1          1          2          1          2          1          1 
2020-04-24 2020-04-25 2020-04-26 2020-04-29 2020-04-30 2020-05-01 2020-05-02 
         2          2          2          2          2          1          1 
2020-05-06 2020-05-08 2020-05-09 2020-05-10 2020-05-14 2020-05-17 2020-06-24 
         2          3          3          1          1          1          1 
2021-04-10 2021-04-12 2021-04-13 2021-04-17 2021-04-18 2021-04-19 2021-04-21 
         1          1          1          1          1          2          4 
2021-04-22 2021-04-23 2021-04-24 2021-04-25 2021-04-27 2021-04-28 2021-04-29 
         3          1          2          1          1          2          2 
2021-05-01 2021-05-06 2021-05-11 2021-05-13 2021-05-16 2021-05-21 2021-05-26 
         1          2          1          2          1          1          1 
2021-05-27 2021-06-04 2021-06-16 2022-04-10 2022-04-13 2022-04-15 2022-04-17 
         1          1          1          1          1          1          2 
2022-04-18 2022-04-21 2022-04-22 2022-04-23 2022-04-24 2022-04-26 2022-04-28 
         2          2          2          1          1          2          2 
2022-04-30 2022-05-05 2022-05-06 2022-05-08 2022-05-11 2022-05-14 2022-05-15 
         3          1          2          2          1          1          1 
2022-05-17 2022-05-19 2022-05-25 2022-05-29 2023-04-15 2023-04-18 2023-04-20 
         2          2          1          1          1          1          2 
2023-04-22 2023-04-23 2023-04-24 2023-04-26 2023-04-27 2023-04-28 2023-04-29 
         2          4          1          1          1          2          1 
2023-04-30 2023-05-01 2023-05-03 2023-05-04 2023-05-06 2023-05-07 2023-05-10 
         2          1          1          1          1          2          1 
2023-05-11 2023-05-17 2023-05-19 2023-05-24 2023-05-25 2023-05-27 2023-05-28 
         2          1          1          1          1          1          2 
2023-07-23 2023-08-04 2028-04-19 
         1          1          1 

$last_observed

           2000-07-21 2000-08-05 2000-08-12 2000-08-13 2000-08-14 2000-08-17 
       149          1          2          1          2          1          1 
2000-08-18 2000-08-19 2000-08-20 2000-08-24 2000-08-26 2000-08-30 2000-08-31 
         1          1          3          1          1          1          1 
2000-09-01 2000-09-05 2000-09-06 2000-09-07 2000-09-15 2000-09-24 2004-06-14 
         1          1          1          1          1          1          1 
2004-07-01 2004-07-18 2004-07-21 2004-07-22 2004-07-23 2004-08-02 2004-08-06 
         2          1          1          1          1          1          1 
2004-08-09 2004-08-10 2004-08-13 2004-08-16 2004-08-19 2004-08-22 2004-08-23 
         1          1          1          1          1          1          1 
2004-08-24 2004-09-03 2004-09-09 2004-09-10 2005-06-10 2005-06-17 2005-06-24 
         1          1          1          1          1          1          1 
2005-06-29 2005-07-07 2005-07-23 2005-07-26 2005-07-28 2005-07-29 2005-07-31 
         1          1          1          1          1          1          1 
2005-08-03 2005-08-05 2005-08-13 2005-08-15 2005-08-17 2005-08-18 2005-08-24 
         1          1          1          1          1          1          1 
2005-08-26 2005-08-27 2005-08-29 2005-09-08 2005-09-12 2005-09-15 2006-06-18 
         2          1          1          2          1          1          1 
2006-06-27 2006-07-13 2006-07-23 2006-07-29 2006-08-05 2006-08-07 2006-08-09 
         1          1          1          1          1          2          1 
2006-08-11 2006-08-16 2006-08-17 2006-08-18 2006-08-31 2006-09-02 2006-09-05 
         1          2          1          1          1          1          1 
2006-09-10 2006-09-11 2006-09-13 2006-09-17 2006-09-18 2007-06-24 2007-07-18 
         1          1          1          1          1          1          1 
2007-08-01 2007-08-06 2007-08-08 2007-08-10 2007-08-11 2007-08-15 2007-08-17 
         1          1          1          1          1          2          1 
2007-08-19 2007-08-20 2007-08-22 2007-08-24 2007-08-25 2007-09-02 2007-09-06 
         3          3          1          2          3          1          1 
2007-09-08 2007-09-09 2007-09-12 2007-09-15 2008-06-09 2008-06-10 2008-06-14 
         1          1          1          1          1          1          1 
2008-06-27 2008-07-18 2008-07-31 2008-08-05 2008-08-07 2008-08-10 2008-08-13 
         1          2          1          1          1          1          1 
2008-08-15 2008-08-17 2008-08-19 2008-08-21 2008-08-24 2008-08-26 2008-08-28 
         1          3          1          1          1          1          1 
2008-08-29 2008-08-31 2008-09-01 2008-09-04 2008-09-19 2008-09-20 2009-06-10 
         3          2          1          1          1          1          1 
2009-07-13 2009-07-17 2009-07-18 2009-07-24 2009-07-26 2009-07-30 2009-08-01 
         1          2          1          1          1          1          1 
2009-08-05 2009-08-06 2009-08-07 2009-08-10 2009-08-11 2009-08-12 2009-08-13 
         1          1          1          1          1          1          2 
2009-08-14 2009-08-15 2009-08-19 2009-08-20 2009-08-21 2009-08-22 2009-08-24 
         1          1          1          1          1          1          1 
2009-08-25 2009-08-27 2009-09-05 2009-09-26 2009-09-27 2010-07-01 2010-07-17 
         1          1          1          1          1          2          1 
2010-07-21 2010-07-22 2010-07-27 2010-07-29 2010-07-30 2010-08-01 2010-08-04 
         1          1          1          3          2          1          1 
2010-08-05 2010-08-07 2010-08-09 2010-08-11 2010-08-12 2010-08-13 2010-08-14 
         1          1          1          2          4          1          1 
2010-08-15 2010-08-17 2010-08-18 2010-08-21 2010-08-23 2010-08-26 2010-09-01 
         1          1          1          1          1          1          1 
2010-09-03 2011-06-25 2011-07-06 2011-07-19 2011-07-21 2011-07-26 2011-07-27 
         1          1          1          1          1          1          1 
2011-07-29 2011-07-31 2011-08-01 2011-08-02 2011-08-03 2011-08-04 2011-08-05 
         3          1          1          1          1          1          1 
2011-08-06 2011-08-10 2011-08-11 2011-08-13 2011-08-15 2011-08-17 2011-08-18 
         2          1          1          1          1          1          1 
2011-08-20 2011-08-22 2011-08-23 2011-08-24 2011-08-26 2011-08-27 2012-06-13 
         1          1          1          1          1          3          1 
2012-06-16 2012-06-19 2012-06-22 2012-07-01 2012-07-10 2012-07-12 2012-07-13 
         1          1          1          1          1          1          2 
2012-07-17 2012-07-19 2012-07-22 2012-07-24 2012-07-25 2012-07-27 2012-08-01 
         1          1          1          1          1          1          1 
2012-08-05 2012-08-06 2012-08-08 2012-08-09 2012-08-10 2012-08-11 2012-08-12 
         1          1          1          1          2          1          1 
2012-08-16 2012-08-25 2012-08-26 2012-08-31 2012-09-01 2012-09-15 2013-05-09 
         1          1          2          1          1          2          1 
2013-06-07 2013-06-09 2013-06-10 2013-06-23 2013-06-26 2013-06-28 2013-07-01 
         1          1          1          1          1          1          1 
2013-07-10 2013-07-11 2013-07-21 2013-07-26 2013-07-28 2013-08-01 2013-08-02 
         1          1          1          1          1          1          1 
2013-08-03 2013-08-08 2013-08-11 2013-08-12 2013-08-16 2013-08-17 2013-08-21 
         1          1          2          1          1          1          3 
2013-08-26 2013-08-30 2013-09-02 2014-04-21 2014-05-03 2014-06-13 2014-06-29 
         2          1          1          1          1          1          1 
2014-07-15 2014-07-16 2014-07-24 2014-07-25 2014-07-26 2014-07-27 2014-08-02 
         1          3          1          1          1          1          1 
2014-08-03 2014-08-06 2014-08-07 2014-08-09 2014-08-10 2014-08-13 2014-08-14 
         2          2          1          1          1          1          2 
2014-08-15 2014-08-16 2014-08-24 2014-08-30 2014-09-06 2014-09-11 2014-09-16 
         1          1          1          1          1          1          1 
2015-05-12 2015-06-19 2015-06-22 2015-07-05 2015-07-09 2015-07-16 2015-07-17 
         1          1          1          1          1          2          1 
2015-07-23 2015-07-24 2015-07-27 2015-07-28 2015-07-29 2015-07-30 2015-07-31 
         1          2          1          1          3          1          1 
2015-08-03 2015-08-04 2015-08-06 2015-08-07 2015-08-09 2015-08-14 2015-08-15 
         1          1          1          1          1          1          1 
2015-08-16 2015-08-19 2015-08-20 2015-08-21 2015-08-23 2015-08-30 2015-09-02 
         1          1          2          1          1          1          1 
2015-09-03 2015-09-06 2016-06-02 2016-07-07 2016-07-10 2016-07-13 2016-07-14 
         1          1          1          1          1          2          2 
2016-07-15 2016-07-17 2016-07-20 2016-07-27 2016-07-28 2016-07-30 2016-07-31 
         3          1          2          1          2          1          1 
2016-08-02 2016-08-03 2016-08-04 2016-08-06 2016-08-11 2016-08-12 2016-08-14 
         1          2          1          3          1          1          1 
2016-08-15 2016-08-18 2016-08-24 2016-08-25 2016-08-26 2016-08-28 2016-09-02 
         1          3          2          1          1          1          1 
2016-09-03 2017-05-17 2017-05-21 2017-06-07 2017-06-24 2017-07-12 2017-07-17 
         1          1          1          1          1          2          1 
2017-07-21 2017-07-22 2017-07-23 2017-07-24 2017-07-28 2017-07-29 2017-07-30 
         1          1          1          1          1          1          1 
2017-07-31 2017-08-03 2017-08-04 2017-08-05 2017-08-10 2017-08-12 2017-08-13 
         1          2          1          1          2          1          3 
2017-08-18 2017-08-19 2017-08-21 2017-08-23 2017-08-24 2017-08-26 2017-08-30 
         1          1          1          1          1          2          1 
2017-09-11 2018-06-21 2018-06-23 2018-06-25 2018-07-05 2018-07-13 2018-07-16 
         2          1          1          1          1          1          1 
2018-07-21 2018-07-22 2018-07-23 2018-07-25 2018-07-26 2018-07-28 2018-07-29 
         1          1          1          2          1          1          1 
2018-07-30 2018-08-01 2018-08-02 2018-08-04 2018-08-08 2018-08-12 2018-08-15 
         1          1          2          2          1          1          1 
2018-08-23 2018-08-26 2018-08-29 2018-09-03 2018-09-06 2018-09-10 2019-05-20 
         1          1          3          1          1          1          1 
2019-06-12 2019-07-03 2019-07-18 2019-07-21 2019-07-24 2019-07-25 2019-07-26 
         2          1          1          1          1          1          1 
2019-07-30 2019-07-31 2019-08-01 2019-08-08 2019-08-09 2019-08-10 2019-08-12 
         1          1          1          1          2          1          1 
2019-08-14 2019-08-17 2019-08-18 2019-08-19 2019-08-21 2019-08-24 2019-08-26 
         2          1          1          2          2          2          1 
2019-08-27 2019-08-31 2019-09-06 2019-09-08 2019-09-16 2020-06-21 2020-07-02 
         1          1          1          1          1          1          1 
2020-07-15 2020-07-23 2020-07-26 2020-08-03 2020-08-05 2020-08-09 2020-08-11 
         1          3          1          1          1          1          2 
2020-08-12 2020-08-13 2020-08-14 2020-08-16 2020-08-19 2020-08-20 2020-08-22 
         2          3          1          1          2          1          1 
2020-08-26 2020-08-27 2020-08-28 2020-08-30 2020-09-04 2020-09-09 2020-09-12 
         2          2          1          1          1          1          2 
2020-09-17 2021-05-12 2021-06-03 2021-06-04 2021-06-06 2021-06-08 2021-07-04 
         1          1          1          1          1          1          1 
2021-07-11 2021-07-14 2021-07-24 2021-07-29 2021-08-01 2021-08-02 2021-08-03 
         1          1          1          1          1          1          1 
2021-08-04 2021-08-05 2021-08-11 2021-08-12 2021-08-13 2021-08-15 2021-08-18 
         1          1          2          1          2          1          2 
2021-08-19 2021-08-20 2021-08-21 2021-08-22 2021-08-26 2021-08-29 2021-09-01 
         1          2          1          1          1          1          1 
2021-09-10 2021-09-18 2021-09-23 2022-05-15 2022-05-27 2022-06-15 2022-06-29 
         1          1          2          1          1          1          1 
2022-07-01 2022-07-09 2022-07-18 2022-07-19 2022-07-21 2022-07-22 2022-07-25 
         1          1          1          1          1          1          1 
2022-07-26 2022-07-27 2022-07-31 2022-08-01 2022-08-02 2022-08-05 2022-08-06 
         1          1          2          1          1          1          2 
2022-08-09 2022-08-10 2022-08-11 2022-08-12 2022-08-17 2022-08-18 2022-08-19 
         1          1          1          1          1          1          1 
2022-08-24 2022-08-25 2022-08-28 2022-09-01 2022-09-02 2022-09-13 2023-05-28 
         1          1          2          1          1          1          1 
2023-06-14 2023-07-15 2023-07-26 2023-07-28 2023-07-29 2023-07-30 2023-08-02 
         1          1          2          1          1          1          1 
2023-08-03 2023-08-04 2023-08-09 2023-08-16 2023-08-17 2023-08-19 2023-08-20 
         1          2          2          2          1          4          1 
2023-08-24 2023-08-25 2023-08-26 2023-08-27 2023-08-28 2023-08-31 2023-09-02 
         1          1          1          1          1          2          2 
2023-09-09 2023-09-11 2023-09-22 
         2          1          2 

$first_nest

           2000-05-05 2000-05-09 2000-05-10 2000-05-11 2000-05-13 2000-05-16 
       189          1          2          3          1          3          1 
2000-05-18 2000-05-19 2000-05-26 2000-05-28 2000-05-29 2000-05-31 2000-06-01 
         2          1          1          2          1          1          1 
2000-06-06 2000-06-08 2000-06-17 2004-05-05 2004-05-08 2004-05-09 2004-05-11 
         1          1          1          1          1          1          1 
2004-05-13 2004-05-14 2004-05-15 2004-05-16 2004-05-19 2004-05-20 2004-05-21 
         2          1          1          2          2          1          1 
2004-05-22 2004-05-26 2004-05-27 2004-05-28 2004-06-03 2004-06-06 2004-06-14 
         1          1          1          1          1          1          1 
2004-06-17 2005-05-04 2005-05-06 2005-05-07 2005-05-08 2005-05-11 2005-05-12 
         1          1          3          1          1          1          3 
2005-05-13 2005-05-14 2005-05-17 2005-05-18 2005-05-19 2005-05-20 2005-05-21 
         1          2          1          1          1          1          1 
2005-05-26 2005-05-28 2005-06-03 2005-06-10 2005-06-15 2005-06-16 2006-05-10 
         2          1          1          1          1          1          1 
2006-05-12 2006-05-14 2006-05-15 2006-05-17 2006-05-18 2006-05-20 2006-05-22 
         3          1          1          2          1          3          3 
2006-05-23 2006-05-25 2006-05-27 2006-06-01 2006-06-09 2006-06-10 2006-06-21 
         1          1          2          1          1          1          1 
2007-05-12 2007-05-14 2007-05-16 2007-05-17 2007-05-18 2007-05-19 2007-05-22 
         1          1          3          4          1          2          1 
2007-05-23 2007-05-25 2007-05-26 2007-05-31 2007-06-02 2007-06-04 2007-06-05 
         1          1          2          1          1          1          1 
2007-06-06 2007-06-08 2007-06-17 2007-06-21 2007-06-27 2008-05-06 2008-05-07 
         1          2          1          1          1          1          1 
2008-05-09 2008-05-10 2008-05-12 2008-05-15 2008-05-16 2008-05-17 2008-05-18 
         1          1          2          2          3          2          3 
2008-05-21 2008-05-23 2008-05-26 2008-05-28 2008-05-31 2008-06-05 2008-06-11 
         2          1          1          1          1          3          1 
2008-06-19 2009-05-03 2009-05-04 2009-05-06 2009-05-07 2009-05-09 2009-05-10 
         1          1          1          1          1          2          1 
2009-05-12 2009-05-13 2009-05-14 2009-05-18 2009-05-20 2009-05-21 2009-05-24 
         2          2          4          1          2          1          2 
2009-06-03 2009-06-06 2009-06-08 2009-06-14 2009-07-29 2010-05-01 2010-05-05 
         1          2          1          1          1          1          2 
2010-05-06 2010-05-07 2010-05-08 2010-05-09 2010-05-13 2010-05-14 2010-05-16 
         3          1          1          1          3          4          2 
2010-05-21 2010-05-22 2010-05-25 2010-05-27 2010-05-28 2010-06-04 2010-06-05 
         1          2          1          2          1          2          1 
2010-06-06 2010-06-09 2010-06-12 2010-06-17 2010-06-18 2011-04-30 2011-05-02 
         1          1          1          1          1          1          1 
2011-05-04 2011-05-06 2011-05-08 2011-05-09 2011-05-12 2011-05-14 2011-05-15 
         1          2          1          2          2          2          3 
2011-05-17 2011-05-18 2011-05-20 2011-05-21 2011-05-28 2011-05-29 2011-06-02 
         1          2          1          1          2          3          1 
2011-06-07 2011-06-08 2011-06-11 2012-04-30 2012-05-02 2012-05-04 2012-05-07 
         1          1          1          1          1          3          1 
2012-05-08 2012-05-09 2012-05-10 2012-05-11 2012-05-12 2012-05-13 2012-05-14 
         1          1          4          4          1          1          2 
2012-05-15 2012-05-16 2012-05-20 2012-05-24 2012-05-26 2012-05-29 2012-05-30 
         1          1          2          1          1          1          1 
2012-06-01 2012-06-11 2013-05-02 2013-05-03 2013-05-06 2013-05-09 2013-05-10 
         1          1          1          2          1          1          2 
2013-05-11 2013-05-12 2013-05-13 2013-05-14 2013-05-15 2013-05-16 2013-05-17 
         2          1          1          2          1          3          1 
2013-05-26 2013-06-01 2013-06-02 2013-06-06 2013-06-07 2013-06-29 2013-08-15 
         3          2          1          1          2          1          1 
2014-04-29 2014-05-01 2014-05-03 2014-05-08 2014-05-09 2014-05-10 2014-05-11 
         1          1          1          3          4          2          1 
2014-05-12 2014-05-14 2014-05-16 2014-05-17 2014-05-21 2014-05-22 2014-05-24 
         2          2          2          1          1          2          2 
2014-05-28 2014-05-29 2014-05-31 2014-06-01 2014-07-05 2015-04-27 2015-05-03 
         1          2          1          1          1          1          1 
2015-05-06 2015-05-07 2015-05-08 2015-05-09 2015-05-11 2015-05-12 2015-05-14 
         3          2          1          3          4          1          2 
2015-05-15 2015-05-16 2015-05-19 2015-05-20 2015-05-21 2015-05-24 2015-05-28 
         2          1          1          2          1          1          1 
2015-05-30 2015-06-05 2015-06-13 2015-07-11 2016-04-27 2016-04-29 2016-05-01 
         1          1          1          1          1          1          1 
2016-05-04 2016-05-05 2016-05-06 2016-05-08 2016-05-11 2016-05-12 2016-05-13 
         1          2          1          1          2          3          2 
2016-05-14 2016-05-16 2016-05-18 2016-05-19 2016-05-22 2016-05-25 2016-05-26 
         2          1          1          1          1          3          1 
2016-05-28 2016-05-29 2016-06-02 2016-06-04 2016-06-16 2016-06-18 2017-04-28 
         1          1          1          1          1          2          1 
2017-04-30 2017-05-03 2017-05-04 2017-05-06 2017-05-07 2017-05-08 2017-05-10 
         2          1          1          1          2          1          2 
2017-05-11 2017-05-12 2017-05-13 2017-05-19 2017-05-21 2017-05-24 2017-05-26 
         2          3          1          3          1          1          1 
2017-05-28 2017-05-31 2017-06-02 2017-06-03 2017-06-04 2017-06-14 2018-05-03 
         1          1          2          1          1          1          1 
2018-05-04 2018-05-06 2018-05-10 2018-05-11 2018-05-12 2018-05-14 2018-05-16 
         1          2          2          2          2          2          3 
2018-05-17 2018-05-18 2018-05-19 2018-05-20 2018-05-23 2018-05-29 2018-05-30 
         2          1          1          2          1          1          2 
2018-05-31 2018-06-01 2018-06-02 2018-06-15 2019-05-03 2019-05-04 2019-05-05 
         3          1          1          1          1          1          1 
2019-05-09 2019-05-10 2019-05-11 2019-05-14 2019-05-16 2019-05-17 2019-05-22 
         4          3          5          1          1          1          1 
2019-05-23 2019-05-24 2019-05-25 2019-06-01 2019-06-02 2019-06-05 2019-06-06 
         1          1          2          1          1          1          2 
2019-06-12 2019-07-06 2020-05-07 2020-05-10 2020-05-11 2020-05-13 2020-05-14 
         1          1          1          1          1          2          2 
2020-05-15 2020-05-16 2020-05-17 2020-05-19 2020-05-20 2020-05-21 2020-05-22 
         3          1          1          1          1          2          3 
2020-05-23 2020-05-24 2020-05-28 2020-05-29 2020-05-30 2020-06-03 2020-06-11 
         1          1          1          2          2          1          1 
2020-06-12 2020-06-27 2020-07-03 2021-05-05 2021-05-06 2021-05-08 2021-05-09 
         1          1          1          2          1          1          2 
2021-05-10 2021-05-12 2021-05-13 2021-05-14 2021-05-17 2021-05-19 2021-05-20 
         1          2          1          3          1          4          2 
2021-05-21 2021-05-25 2021-05-27 2021-05-28 2021-05-30 2021-06-02 2021-06-04 
         1          1          1          1          1          2          2 
2021-06-17 2022-05-06 2022-05-07 2022-05-08 2022-05-09 2022-05-11 2022-05-14 
         1          1          1          3          1          1          1 
2022-05-15 2022-05-16 2022-05-17 2022-05-18 2022-05-19 2022-05-20 2022-05-21 
         1          1          1          1          2          2          3 
2022-05-25 2022-05-26 2022-05-27 2022-05-29 2022-06-03 2022-06-08 2022-06-11 
         1          1          1          2          2          1          1 
2022-06-25 2022-06-27 2022-07-01 2023-05-07 2023-05-10 2023-05-11 2023-05-12 
         1          1          1          2          1          1          2 
2023-05-13 2023-05-14 2023-05-16 2023-05-19 2023-05-21 2023-05-24 2023-05-26 
         1          3          2          1          1          2          1 
2023-05-28 2023-05-30 2023-06-01 2023-06-02 2023-06-03 2023-06-07 2023-06-09 
         1          1          2          2          1          2          1 
2023-06-10 2023-06-17 2023-07-23 
         1          1          1 

$first_chick

           2000-05-28 2000-05-31 2000-06-01 2000-06-02 2000-06-03 2000-06-04 
       262          1          1          1          2          3          1 
2000-06-06 2000-06-07 2000-06-08 2000-06-09 2000-06-18 2000-06-20 2000-06-22 
         1          1          1          1          1          2          1 
2000-06-24 2000-06-26 2000-06-28 2000-07-22 2004-05-31 2004-06-04 2004-06-05 
         1          1          1          1          2          2          1 
2004-06-06 2004-06-09 2004-06-11 2004-06-14 2004-06-16 2004-06-18 2004-06-20 
         1          1          2          1          1          1          1 
2004-06-21 2004-06-22 2004-06-27 2004-07-04 2004-07-18 2005-05-27 2005-05-28 
         1          2          1          1          1          1          1 
2005-06-01 2005-06-02 2005-06-03 2005-06-05 2005-06-06 2005-06-09 2005-06-10 
         2          3          1          1          1          1          4 
2005-06-11 2005-06-12 2005-06-15 2005-06-26 2005-07-01 2005-07-21 2006-06-02 
         1          1          1          2          1          1          1 
2006-06-03 2006-06-05 2006-06-07 2006-06-09 2006-06-10 2006-06-11 2006-06-13 
         1          1          1          1          1          1          1 
2006-06-14 2006-06-16 2006-06-17 2006-06-20 2006-06-24 2006-06-30 2006-07-03 
         2          2          2          1          2          1          1 
2006-07-14 2007-06-06 2007-06-07 2007-06-09 2007-06-10 2007-06-11 2007-06-12 
         1          2          2          1          3          1          2 
2007-06-13 2007-06-14 2007-06-15 2007-06-20 2007-06-27 2007-06-28 2007-06-29 
         1          2          3          3          1          1          1 
2007-07-02 2007-07-15 2007-07-19 2008-05-30 2008-06-02 2008-06-04 2008-06-05 
         1          1          1          1          1          2          4 
2008-06-07 2008-06-08 2008-06-09 2008-06-12 2008-06-17 2008-06-18 2008-06-25 
         1          2          1          2          1          1          1 
2008-06-26 2008-06-27 2008-06-29 2008-07-06 2008-07-10 2009-05-27 2009-05-28 
         2          1          1          1          1          1          2 
2009-05-29 2009-05-30 2009-05-31 2009-06-03 2009-06-04 2009-06-05 2009-06-06 
         1          1          2          1          4          1          3 
2009-06-10 2009-06-11 2009-06-17 2009-06-20 2009-06-24 2009-06-25 2009-06-26 
         1          1          1          1          1          1          1 
2009-06-29 2009-07-15 2010-05-25 2010-05-26 2010-05-27 2010-05-30 2010-06-01 
         1          1          1          2          3          3          1 
2010-06-03 2010-06-04 2010-06-09 2010-06-10 2010-06-11 2010-06-16 2010-06-17 
         3          1          1          2          1          2          1 
2010-06-18 2010-06-25 2010-06-30 2010-07-01 2010-07-02 2010-07-07 2010-07-09 
         1          1          2          1          1          1          1 
2011-05-25 2011-05-26 2011-05-27 2011-05-29 2011-05-31 2011-06-02 2011-06-03 
         1          1          1          1          1          2          1 
2011-06-04 2011-06-05 2011-06-06 2011-06-08 2011-06-09 2011-06-10 2011-06-11 
         1          1          1          2          1          1          1 
2011-06-12 2011-06-14 2011-06-16 2011-06-17 2011-06-18 2011-06-30 2011-07-01 
         1          1          1          1          1          1          1 
2011-07-10 2011-07-13 2011-07-21 2012-05-20 2012-05-23 2012-05-24 2012-05-26 
         1          1          1          1          1          1          1 
2012-05-27 2012-05-28 2012-05-31 2012-06-01 2012-06-02 2012-06-04 2012-06-06 
         1          2          1          2          1          1          3 
2012-06-08 2012-06-10 2012-06-15 2012-06-20 2012-06-23 2012-06-24 2012-07-01 
         1          1          2          1          3          1          1 
2013-05-11 2013-05-26 2013-05-27 2013-05-29 2013-05-31 2013-06-01 2013-06-03 
         1          3          1          1          1          1          1 
2013-06-06 2013-06-07 2013-06-08 2013-06-09 2013-06-12 2013-06-13 2013-06-19 
         2          3          1          2          1          1          1 
2013-06-21 2013-06-22 2013-06-23 2013-06-26 2013-06-29 2014-05-22 2014-05-24 
         1          1          1          1          1          1          1 
2014-05-25 2014-05-29 2014-05-31 2014-06-01 2014-06-02 2014-06-03 2014-06-05 
         1          2          2          2          1          1          3 
2014-06-06 2014-06-09 2014-06-11 2014-06-13 2014-06-14 2014-06-15 2014-06-19 
         2          1          1          1          2          1          1 
2014-06-20 2014-06-21 2014-06-29 2014-07-02 2014-07-26 2014-08-01 2015-05-22 
         1          1          1          1          1          1          1 
2015-05-25 2015-05-26 2015-05-28 2015-05-29 2015-05-30 2015-05-31 2015-06-04 
         1          1          2          1          1          2          6 
2015-06-05 2015-06-06 2015-06-08 2015-06-10 2015-06-11 2015-06-12 2015-06-19 
         1          1          1          1          1          1          1 
2015-06-26 2015-06-27 2015-08-02 2016-05-22 2016-05-25 2016-05-26 2016-05-27 
         1          1          1          1          1          2          1 
2016-05-28 2016-05-29 2016-06-02 2016-06-04 2016-06-05 2016-06-06 2016-06-08 
         2          3          1          1          1          1          5 
2016-06-09 2016-06-10 2016-06-12 2016-06-16 2016-06-17 2016-06-18 2016-07-13 
         1          1          2          2          1          1          1 
2016-07-14 2016-07-15 2016-07-16 2017-05-22 2017-05-25 2017-05-26 2017-05-28 
         1          1          1          1          3          1          3 
2017-05-30 2017-06-01 2017-06-02 2017-06-03 2017-06-08 2017-06-11 2017-06-12 
         1          3          3          1          1          1          1 
2017-06-14 2017-06-16 2017-06-17 2017-06-21 2017-06-22 2017-06-24 2018-05-27 
         2          1          1          2          1          1          1 
2018-05-30 2018-05-31 2018-06-02 2018-06-03 2018-06-04 2018-06-05 2018-06-07 
         1          3          1          1          1          1          4 
2018-06-08 2018-06-10 2018-06-13 2018-06-14 2018-06-18 2018-06-20 2018-06-23 
         2          2          1          1          1          1          1 
2018-06-25 2018-06-28 2018-07-13 2019-05-30 2019-05-31 2019-06-02 2019-06-03 
         1          1          1          2          2          1          2 
2019-06-04 2019-06-05 2019-06-06 2019-06-07 2019-06-09 2019-06-16 2019-06-18 
         1          1          1          1          1          3          1 
2019-06-20 2019-06-23 2019-06-26 2019-06-27 2019-06-30 2019-07-02 2019-07-03 
         1          1          2          2          1          1          2 
2019-07-04 2020-05-31 2020-06-01 2020-06-03 2020-06-04 2020-06-05 2020-06-07 
         1          1          1          1          3          1          1 
2020-06-12 2020-06-13 2020-06-14 2020-06-19 2020-06-20 2020-06-21 2020-06-23 
         2          1          2          2          1          2          1 
2020-06-27 2020-07-02 2020-07-03 2020-07-05 2020-07-09 2020-07-16 2020-07-20 
         1          1          2          2          1          1          1 
2020-07-30 2021-05-28 2021-05-30 2021-05-31 2021-06-03 2021-06-04 2021-06-05 
         1          1          1          2          3          1          1 
2021-06-06 2021-06-07 2021-06-11 2021-06-12 2021-06-14 2021-06-16 2021-06-18 
         1          1          2          2          1          1          1 
2021-06-27 2021-06-30 2021-07-01 2021-07-02 2021-07-15 2022-05-27 2022-06-02 
         2          1          2          1          1          1          5 
2022-06-03 2022-06-04 2022-06-06 2022-06-08 2022-06-11 2022-06-12 2022-06-13 
         1          1          1          1          1          2          1 
2022-06-15 2022-06-16 2022-06-17 2022-06-19 2022-06-20 2022-06-22 2022-06-23 
         1          1          3          1          1          1          2 
2022-06-25 2023-05-28 2023-05-31 2023-06-01 2023-06-04 2023-06-05 2023-06-09 
         1          1          1          1          1          4          1 
2023-06-10 2023-06-14 2023-06-17 2023-06-18 2023-06-19 2023-06-23 2023-06-25 
         2          2          1          1          1          1          1 
2023-06-29 2023-07-05 2023-07-09 2023-07-27 2023-08-02 2023-08-04 3000-06-02 
         1          1          1          1          1          1          1 

$first_fledge

           2000-06-20 2000-06-21 2000-06-22 2000-06-24 2000-06-25 2000-06-26 
       387          1          1          2          3          2          2 
2000-06-30 2000-07-04 2000-07-08 2000-07-13 2000-07-15 2000-07-16 2000-07-17 
         2          1          1          1          1          1          1 
2000-07-20 2000-07-24 2000-08-06 2004-06-23 2004-06-25 2004-06-26 2004-06-30 
         1          1          1          1          1          1          1 
2004-07-02 2004-07-04 2004-07-10 2004-07-13 2004-07-14 2004-07-17 2004-07-18 
         1          2          1          1          2          1          1 
2004-07-25 2005-06-22 2005-06-23 2005-06-24 2005-06-25 2005-06-30 2005-07-01 
         1          3          1          2          3          1          1 
2005-07-02 2005-07-07 2005-07-08 2005-07-13 2005-07-18 2005-07-20 2005-07-22 
         1          1          1          1          1          1          1 
2005-08-27 2006-06-23 2006-06-25 2006-06-29 2006-06-30 2006-07-01 2006-07-04 
         1          1          1          1          1          1          2 
2006-07-07 2006-07-12 2006-07-13 2006-07-15 2006-07-17 2006-07-24 2006-07-28 
         3          1          1          2          1          1          1 
2006-08-02 2006-08-11 2007-06-26 2007-06-27 2007-06-28 2007-06-29 2007-06-30 
         1          1          1          1          2          1          1 
2007-07-02 2007-07-04 2007-07-05 2007-07-06 2007-07-10 2007-07-11 2007-07-12 
         2          3          1          1          1          2          2 
2007-07-13 2007-07-18 2007-07-23 2007-08-08 2008-06-20 2008-06-22 2008-06-25 
         2          2          1          2          1          1          2 
2008-06-26 2008-06-27 2008-06-28 2008-06-29 2008-07-02 2008-07-15 2008-07-17 
         3          1          2          1          1          1          1 
2008-07-18 2008-07-20 2008-07-24 2008-07-30 2008-07-31 2008-08-17 2009-06-16 
         1          2          1          1          1          1          1 
2009-06-20 2009-06-21 2009-06-24 2009-06-25 2009-06-26 2009-06-29 2009-06-30 
         2          1          3          3          1          1          1 
2009-07-01 2009-07-03 2009-07-05 2009-07-10 2009-07-15 2009-07-16 2009-07-20 
         1          1          1          1          2          1          1 
2009-08-07 2010-06-09 2010-06-16 2010-06-17 2010-06-18 2010-06-20 2010-06-22 
         1          1          1          2          3          2          1 
2010-06-24 2010-06-25 2010-06-27 2010-06-30 2010-07-01 2010-07-09 2010-07-14 
         1          2          1          1          1          2          2 
2010-07-21 2010-07-23 2010-07-27 2010-07-29 2010-07-30 2010-08-05 2011-06-14 
         1          1          1          1          1          1          1 
2011-06-17 2011-06-19 2011-06-22 2011-06-23 2011-06-25 2011-06-27 2011-06-28 
         1          1          1          4          2          1          1 
2011-07-01 2011-07-03 2011-07-06 2011-07-09 2011-07-11 2011-07-21 2011-07-22 
         3          1          1          1          1          1          1 
2011-07-24 2012-06-15 2012-06-17 2012-06-19 2012-06-21 2012-06-22 2012-06-23 
         1          1          2          1          1          1          1 
2012-06-24 2012-06-26 2012-06-27 2012-06-28 2012-06-29 2012-07-01 2012-07-03 
         1          1          1          1          1          1          1 
2012-07-07 2012-07-14 2012-07-15 2012-07-19 2012-07-21 2012-08-17 2013-06-20 
         1          1          1          1          1          1          1 
2013-06-21 2013-06-22 2013-06-23 2013-06-26 2013-06-27 2013-06-29 2013-06-30 
         1          2          1          1          2          2          1 
2013-07-02 2013-07-04 2013-07-05 2013-07-10 2013-07-11 2013-07-17 2013-07-20 
         1          1          1          2          1          1          1 
2013-07-31 2013-08-02 2014-06-15 2014-06-16 2014-06-18 2014-06-19 2014-06-21 
         1          1          1          2          1          3          1 
2014-06-22 2014-06-23 2014-06-27 2014-06-28 2014-07-03 2014-07-04 2014-07-05 
         2          1          3          1          1          2          2 
2014-07-09 2014-07-12 2014-07-16 2014-08-09 2014-08-15 2015-06-13 2015-06-14 
         1          1          1          1          1          1          3 
2015-06-17 2015-06-18 2015-06-20 2015-06-22 2015-06-25 2015-06-26 2015-06-27 
         1          1          1          1          2          3          1 
2015-06-30 2015-07-02 2015-07-03 2015-07-04 2015-07-17 2015-08-19 2016-06-09 
         1          1          2          1          1          1          1 
2016-06-14 2016-06-15 2016-06-17 2016-06-19 2016-06-23 2016-06-24 2016-06-27 
         1          1          1          3          4          1          1 
2016-06-29 2016-07-02 2016-07-03 2016-07-06 2016-07-07 2016-07-08 2016-07-09 
         2          3          3          2          2          2          1 
2016-08-05 2017-06-11 2017-06-15 2017-06-17 2017-06-18 2017-06-20 2017-06-21 
         1          2          1          1          1          1          1 
2017-06-22 2017-06-23 2017-06-24 2017-06-27 2017-06-29 2017-07-02 2017-07-05 
         3          1          1          2          1          1          1 
2017-07-06 2017-07-07 2017-07-11 2017-07-12 2017-07-14 2017-07-25 2017-08-05 
         1          1          1          1          1          1          1 
2018-06-21 2018-06-22 2018-06-23 2018-06-24 2018-06-26 2018-06-27 2018-06-28 
         2          1          2          2          1          2          3 
2018-06-29 2018-07-04 2018-07-05 2018-07-06 2018-07-12 2018-07-14 2018-07-19 
         1          2          1          1          1          1          1 
2018-08-03 2019-06-19 2019-06-20 2019-06-21 2019-06-23 2019-06-26 2019-06-27 
         1          1          1          1          1          1          2 
2019-06-28 2019-06-29 2019-06-30 2019-07-01 2019-07-03 2019-07-04 2019-07-07 
         1          1          1          1          1          1          1 
2019-07-08 2019-07-10 2019-07-12 2019-07-13 2019-07-18 2019-07-19 2019-07-20 
         1          1          2          1          1          1          1 
2019-07-21 2019-08-01 2020-06-20 2020-06-24 2020-06-25 2020-07-02 2020-07-03 
         3          1          1          2          3          1          1 
2020-07-04 2020-07-08 2020-07-09 2020-07-13 2020-07-14 2020-07-15 2020-07-17 
         1          1          3          1          1          2          2 
2020-07-24 2020-07-26 2020-08-06 2020-08-13 2020-08-23 2022-06-25 
         1          2          1          1          1          1 

We use map rather than one of the other map functions for this step because the table for each column will have a different length.

The purrr documentation provides more details about how to use the many functions in the package.

17.3 The Split-Apply Pattern

In a data set with categorical features, it’s often useful to compute something for each category. The map functions can compute something for each element of a data structure, but categories are not necessarily elements.

For example, the least terns data set has 6 different categories in the region_3 column. If we want all of the rows for one region, one way to get them is by filtering:

library("dplyr")

Attaching package: 'dplyr'
The following objects are masked from 'package:stats':

    filter, lag
The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union
southern = filter(terns, region_3 == "SOUTHERN")
head(southern)
  year                             site_name
1 2000 SANTA CLARA RIVER MCGRATH STATE BEACH
2 2000                          ORMOND BEACH
3 2000                       NBVC POINT MUGU
4 2000                          VENICE BEACH
5 2000                             LA HARBOR
6 2000            SEAL BEACH NWR ANAHEIM BAY
                  site_name_2013_2018  site_name_1988_2001    site_abbr
1                   Santa Clara River NA_2013_2018 POLYGON   S_CLAR_MCG
2                        Ormond Beach NA_2013_2018 POLYGON       ORMOND
3                     NBVC Point Mugu NA_2013_2018 POLYGON      PT_MUGU
4                        Venice Beach NA_2013_2018 POLYGON      VEN_BCH
5                          Port of LA NA_2013_2018 POLYGON      LA_HARB
6 Seal Beach National Wildlife Refuge NA_2013_2018 POLYGON SEAL_BCH_NWR
  region_3 region_4   event bp_min bp_max fl_min fl_max total_nests
1 SOUTHERN  VENTURA LA_NINA     21     21      9      9          22
2 SOUTHERN  VENTURA LA_NINA     73     73     60     65          73
3 SOUTHERN  VENTURA LA_NINA    166    167     64     64         252
4 SOUTHERN SOUTHERN LA_NINA    274    294    150    200         308
5 SOUTHERN SOUTHERN LA_NINA    437    437    570    570         565
6 SOUTHERN SOUTHERN LA_NINA    107    107    180    180         107
  nonpred_eggs nonpred_chicks nonpred_fl nonpred_ad pred_control pred_eggs
1            4              3         NA         NA                     NA
2            2              0          0          0                     NA
3           NA             NA         NA         NA                     NA
4           26             NA         NA         NA                     32
5           77             NA         NA         NA                     24
6           10              3         NA         NA                     NA
  pred_chicks pred_fl pred_ad pred_pefa pred_coy_fox pred_meso pred_owlspp
1          NA      NA      NA                                             
2          NA      NA      NA         N            N         Y           N
3          NA      NA      NA                                             
4          20      20       3         Y            N         Y           N
5          NA      15      NA         Y            N         N           N
6          NA      NA      NA                                             
  pred_corvid pred_other_raptor pred_other_avian pred_misc total_pefa
1                                                                  NA
2           N                 Y                N         N         NA
3                                                                  NA
4           Y                 N                N         N          2
5           N                 Y                Y         N         17
6                                                                  NA
  total_coy_fox total_meso total_owlspp total_corvid total_other_raptor
1            NA         NA           NA           NA                 NA
2            NA         NA           NA           NA                 NA
3            NA         NA           NA           NA                 NA
4             0         42            0           31                  0
5             0          0            0            0                  4
6            NA         NA           NA           NA                 NA
  total_other_avian total_misc first_observed last_observed first_nest
1                NA         NA     2000-06-06    2000-09-05 2000-06-06
2                NA         NA                              2000-06-08
3                NA         NA     2000-05-21    2000-08-12 2000-06-01
4                 0          0     2000-04-19    2000-08-20 2000-05-29
5                24          0     2000-04-28    2000-08-20 2000-05-10
6                NA         NA                                        
  first_chick first_fledge
1  2000-06-28   2000-07-24
2  2000-06-26   2000-07-17
3  2000-06-24   2000-07-16
4                         
5  3000-06-02   2000-06-22
6                         

To get all 6 regions separately, we’d have to do this 6 times. If we want to compute something for each region, say the median of the total_nests column, we also have to repeat that computation 6 times. Here’s what it would look like for just the SOUTHERN region:

median(southern$total_nests, na.rm = TRUE)
[1] 64

If the categories were elements, we could avoid writing code to index each category, and just use a map function to apply the median function to each.

The split function splits a vector or data frame into groups based on a vector of categories. The first argument to split is the data, and the second argument is a congruent vector of categories.

We can use split to elegantly compute medians of total_nests broken down by country. First, we split the data by region Since we only want to compute on the total_nests column, we only split that column:

by_region = split(terns$total_nests, terns$region_3)
class(by_region)
[1] "list"
names(by_region)
[1] "ARIZONA"    "CENTRAL"    "KINGS"      "S.F._BAY"   "SACRAMENTO"
[6] "SOUTHERN"  

The result from split is a list with one element for each category. The individual elements contain pieces of the original total_nests column:

head(by_region$SOUTHERN)
[1]  22  73 252 308 565 107

Since the categories are elements in the split data, now we can use map_dbl the same way we did in previous examples:

map_dbl(by_region, median, na.rm = TRUE)
   ARIZONA    CENTRAL      KINGS   S.F._BAY SACRAMENTO   SOUTHERN 
         3         17          1         35          1         64 

This two-step process is an R idiom called the split-apply pattern. First you use split to convert categories into list elements, then you use a map (or apply) function to compute something on each category. Any time you want to compute results by category, you should think of this pattern.

17.4 Grouping

The dplyr package’s group_by and summarize functions are analogous to split and purrr’s map functions, but return the results in a data frame with one row for each category. In many cases, this is more convenient than a list or vector result.

As an example, here’s how to use group_by and summarize to compute the total_nests medians:

by_region = group_by(terns, region_3)
summarize(by_region, median(total_nests, na.rm = TRUE))
# A tibble: 6 × 2
  region_3   `median(total_nests, na.rm = TRUE)`
  <chr>                                    <dbl>
1 ARIZONA                                      3
2 CENTRAL                                     17
3 KINGS                                        1
4 S.F._BAY                                    35
5 SACRAMENTO                                   1
6 SOUTHERN                                    64

The dplyr documentation provides more details about how to use the group_by and summarize functions.