library(tidyverse)
library(jsonlite)
dnd_base <- "https://www.dnd5eapi.co/api/monsters/"I’ve been playing Dungeons and Dragons 5th edition (D&D 5e) for a few years now and really enjoy it, although COVID has really hindered my opportunity to play. That said, I recently discovered a D&D 5e API, so I figured I’d do a series of blog posts analyzing D&D data from this API. In this first post, I wanted to do a quick walkthrough of how to get data from this API using R and wrangling it into a structure that’s more or less conducive to later analysis. In later posts, I’ll explore the data and then get into some modeling.
As something of an aside – the API has data for character classes, spells, races, monsters, etc. I’m mostly going to focus on the monsters data, but might use some of the other data later on.
Setup
First, I’ll load the packages I need to get and wrangle the data, which is really just {tidyverse}, {jsonlite} and good old base R. I’m also adding in the base URL of the API.
Fetching Data
So, the first step here is to actually get the data from the API. Let’s walk through the process here, illustrating this with a single monster (the aboleth) and then applying the process to all of the monsters.
We’ll use the fromJSON() function to get JSON data from the API. We’ll see that this gives us a pretty gnarly nested list.
example <- fromJSON(paste0(dnd_base, "aboleth"))
glimpse(example)List of 28
$ index : chr "aboleth"
$ name : chr "Aboleth"
$ size : chr "Large"
$ type : chr "aberration"
$ alignment : chr "lawful evil"
$ armor_class : int 17
$ hit_points : int 135
$ hit_dice : chr "18d10"
$ speed :List of 2
..$ walk: chr "10 ft."
..$ swim: chr "40 ft."
$ strength : int 21
$ dexterity : int 9
$ constitution : int 15
$ intelligence : int 18
$ wisdom : int 15
$ charisma : int 18
$ proficiencies :'data.frame': 5 obs. of 2 variables:
..$ value : int [1:5] 6 8 6 12 10
..$ proficiency:'data.frame': 5 obs. of 3 variables:
.. ..$ index: chr [1:5] "saving-throw-con" "saving-throw-int" "saving-throw-wis" "skill-history" ...
.. ..$ name : chr [1:5] "Saving Throw: CON" "Saving Throw: INT" "Saving Throw: WIS" "Skill: History" ...
.. ..$ url : chr [1:5] "/api/proficiencies/saving-throw-con" "/api/proficiencies/saving-throw-int" "/api/proficiencies/saving-throw-wis" "/api/proficiencies/skill-history" ...
$ damage_vulnerabilities: list()
$ damage_resistances : list()
$ damage_immunities : list()
$ condition_immunities : list()
$ senses :List of 2
..$ darkvision : chr "120 ft."
..$ passive_perception: int 20
$ languages : chr "Deep Speech, telepathy 120 ft."
$ challenge_rating : int 10
$ xp : int 5900
$ special_abilities :'data.frame': 3 obs. of 3 variables:
..$ name: chr [1:3] "Amphibious" "Mucous Cloud" "Probing Telepathy"
..$ desc: chr [1:3] "The aboleth can breathe air and water." "While underwater, the aboleth is surrounded by transformative mucus. A creature that touches the aboleth or tha"| __truncated__ "If a creature communicates telepathically with the aboleth, the aboleth learns the creature's greatest desires "| __truncated__
..$ dc :'data.frame': 3 obs. of 3 variables:
.. ..$ dc_type :'data.frame': 3 obs. of 3 variables:
.. ..$ dc_value : int [1:3] NA 14 NA
.. ..$ success_type: chr [1:3] NA "none" NA
$ actions :'data.frame': 4 obs. of 7 variables:
..$ name : chr [1:4] "Multiattack" "Tentacle" "Tail" "Enslave"
..$ desc : chr [1:4] "The aboleth makes three tentacle attacks." "Melee Weapon Attack: +9 to hit, reach 10 ft., one target. Hit: 12 (2d6 + 5) bludgeoning damage. If the target i"| __truncated__ "Melee Weapon Attack: +9 to hit, reach 10 ft. one target. Hit: 15 (3d6 + 5) bludgeoning damage." "The aboleth targets one creature it can see within 30 ft. of it. The target must succeed on a DC 14 Wisdom savi"| __truncated__
..$ options :'data.frame': 4 obs. of 2 variables:
.. ..$ choose: int [1:4] 1 NA NA NA
.. ..$ from :List of 4
..$ attack_bonus: int [1:4] NA 9 9 NA
..$ dc :'data.frame': 4 obs. of 3 variables:
.. ..$ dc_type :'data.frame': 4 obs. of 3 variables:
.. ..$ dc_value : int [1:4] NA 14 NA 14
.. ..$ success_type: chr [1:4] NA "none" NA "none"
..$ damage :List of 4
.. ..$ : NULL
.. ..$ :'data.frame': 2 obs. of 2 variables:
.. ..$ :'data.frame': 1 obs. of 2 variables:
.. ..$ : NULL
..$ usage :'data.frame': 4 obs. of 2 variables:
.. ..$ type : chr [1:4] NA NA NA "per day"
.. ..$ times: int [1:4] NA NA NA 3
$ legendary_actions :'data.frame': 3 obs. of 4 variables:
..$ name : chr [1:3] "Detect" "Tail Swipe" "Psychic Drain (Costs 2 Actions)"
..$ desc : chr [1:3] "The aboleth makes a Wisdom (Perception) check." "The aboleth makes one tail attack." "One creature charmed by the aboleth takes 10 (3d6) psychic damage, and the aboleth regains hit points equal to "| __truncated__
..$ attack_bonus: int [1:3] NA NA 0
..$ damage :List of 3
.. ..$ : NULL
.. ..$ : NULL
.. ..$ :'data.frame': 1 obs. of 2 variables:
$ url : chr "/api/monsters/aboleth"To clean this list up a bit, we’ll use the enframe() function (from {tibble}) to convert the lists into a dataframe and then the pivot_wider() function to reshape this into a single-row tibble.
example %>%
enframe() %>%
pivot_wider(names_from = name,
values_from = value) %>%
glimpse()Rows: 1
Columns: 28
$ index <list> "aboleth"
$ name <list> "Aboleth"
$ size <list> "Large"
$ type <list> "aberration"
$ alignment <list> "lawful evil"
$ armor_class <list> 17
$ hit_points <list> 135
$ hit_dice <list> "18d10"
$ speed <list> ["10 ft.", "40 ft."]
$ strength <list> 21
$ dexterity <list> 9
$ constitution <list> 15
$ intelligence <list> 18
$ wisdom <list> 15
$ charisma <list> 18
$ proficiencies <list> [<data.frame[5 x 2]>]
$ damage_vulnerabilities <list> []
$ damage_resistances <list> []
$ damage_immunities <list> []
$ condition_immunities <list> []
$ senses <list> ["120 ft.", 20]
$ languages <list> "Deep Speech, telepathy 120 ft."
$ challenge_rating <list> 10
$ xp <list> 5900
$ special_abilities <list> [<data.frame[3 x 3]>]
$ actions <list> [<data.frame[4 x 7]>]
$ legendary_actions <list> [<data.frame[3 x 4]>]
$ url <list> "/api/monsters/aboleth"Great. This is more or less the structure we want. You might notice that all of our columns are lists rather than atomic vectors – we’ll deal with that later once we get all of the data.
Now that we know the basic process, we’ll just apply this to all of the monsters with data available through the API. To do that, I’ll write a function that executes the previous steps, get a list of all of the monsters available in the API, use map() to iterate the “fetch” function for each monster, and then bind all of the resulting rows together.
fetch_monster <- function(monster) {
dnd_url <- "https://www.dnd5eapi.co/api/monsters/"
ret <- fromJSON(paste0(dnd_url, monster)) %>%
enframe() %>%
pivot_wider(names_from = name,
values_from = value)
return(ret)
}
#this gets all of the monster indices to plug into the fetch function
mons <- fromJSON(dnd_base)$results %>%
pull(index)
monster_lists <- map(mons, fetch_monster)
mons_bind <- bind_rows(monster_lists)
glimpse(mons_bind)Rows: 332
Columns: 31
$ index <list> "aboleth", "acolyte", "adult-black-dragon", "a…
$ name <list> "Aboleth", "Acolyte", "Adult Black Dragon", "A…
$ size <list> "Large", "Medium", "Huge", "Huge", "Huge", "Hu…
$ type <list> "aberration", "humanoid", "dragon", "dragon", …
$ alignment <list> "lawful evil", "any alignment", "chaotic evil"…
$ armor_class <list> 17, 10, 19, 19, 18, 19, 18, 19, 19, 19, 19, 18…
$ hit_points <list> 135, 9, 195, 225, 172, 212, 184, 256, 207, 256…
$ hit_dice <list> "18d10", "2d8", "17d12", "18d12", "15d12", "17…
$ speed <list> ["10 ft.", "40 ft."], ["30 ft."], ["40 ft.", "…
$ strength <list> 21, 10, 23, 25, 23, 25, 23, 27, 23, 27, 27, 22…
$ dexterity <list> 9, 10, 14, 10, 10, 10, 12, 14, 12, 10, 10, 10,…
$ constitution <list> 15, 10, 21, 23, 21, 23, 21, 25, 21, 25, 25, 22…
$ intelligence <list> 18, 10, 14, 16, 14, 16, 18, 16, 18, 16, 16, 8,…
$ wisdom <list> 15, 14, 13, 15, 13, 15, 15, 15, 15, 13, 13, 12…
$ charisma <list> 18, 11, 17, 19, 17, 19, 17, 24, 17, 21, 21, 12…
$ proficiencies <list> [<data.frame[5 x 2]>], [<data.frame[2 x 2]>], …
$ damage_vulnerabilities <list> [], [], [], [], [], [], [], [], [], [], [], []…
$ damage_resistances <list> [], [], [], [], [], [], [], [], [], [], [], []…
$ damage_immunities <list> [], [], "acid", "lightning", "fire", "lightnin…
$ condition_immunities <list> [], [], [], [], [], [], [], [], [<data.frame[1…
$ senses <list> ["120 ft.", 20], [12], ["60 ft.", "120 ft.", 2…
$ languages <list> "Deep Speech, telepathy 120 ft.", "any one lan…
$ challenge_rating <list> 10, 0.25, 14, 16, 13, 15, 14, 17, 15, 17, 16, …
$ xp <list> 5900, 50, 11500, 15000, 10000, 13000, 11500, 1…
$ special_abilities <list> [<data.frame[3 x 3]>], [<data.frame[1 x 3]>], …
$ actions <list> [<data.frame[4 x 7]>], [<data.frame[1 x 4]>], …
$ legendary_actions <list> [<data.frame[3 x 4]>], [], [<data.frame[3 x 4]…
$ url <list> "/api/monsters/aboleth", "/api/monsters/acolyt…
$ subtype <list> <NULL>, "any race", <NULL>, <NULL>, <NULL>, <N…
$ reactions <list> <NULL>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>…
$ forms <list> <NULL>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>…Notice that we have the same structure as in the previous example, but now with 322 rows instead of 1. Now we can take care of coercing some of these list columns into atomic vectors.
Restructuring Data
One problem here, though, is that the possible variable values for each column differ depending on the monster (for some variables). Variables like strength, hit points, challenge rating, and xp will always be a single integer value, but variables like legendary_actions can differ greatly. People who play D&D will know that normal monsters don’t have any legendary actions, and so this will be NULL for those monsters. But some monsters might have 1 or 2 legendary actions, whereas big baddies like ancient dragons can have several. This same varying structure applies to columns like proficiencies, special abilities, reactions, etc. Ultimately, this means that a list column is probably the best way to represent this type of data, since lists are more flexible, whereas some columns can be represented as an atomic vector, and so we need to figure out how to address this.
To do this, we can write a couple of functions. The first, compare_lens() (below), will determine if the length of each element of a list is equal to whatever size we want to compare against (I’ve set the default to 1, which is what we want to use in this case). It then uses the all() function to determine if all of these comparisons are equal to TRUE, and will return a single value of TRUE if this is the case (and a single FALSE if not).
compare_lens <- function(x, size = 1) {
all(map_lgl(x, ~length(unlist(.x)) == size))
}Next, we’ll use the compare_lens() function as the test expression in another function, cond_unlist (or conditionally unlist), below. The idea here is if compare_lens() is TRUE, then we will unlist the list (simplify it to a vector) passed to the function; otherwise, we’ll leave it as is (as a list). Putting these functions together, the logic is:
- Determine if all elements of a list have a length equal to 1.
- If so, turn that list into a vector.
- If not, leave it as a list.
cond_unlist <- function(x) {
if (compare_lens(x) == TRUE) {
unlist(x)
} else {
x
}
}The final step is to apply this function to all of the columns (which, recall, are lists) in our mons_bind tibble. We can do this using a combination of mutate() and across(). After doing this, we’ll see that some of the columns in our data frame have been simplified to character, integer, and double vectors, whereas others remain lists (lists of lists, lists of data frames).
mons_df <- mons_bind %>%
mutate(across(.cols = everything(), ~cond_unlist(x = .x)))
glimpse(mons_df)Rows: 332
Columns: 31
$ index <chr> "aboleth", "acolyte", "adult-black-dragon", "ad…
$ name <chr> "Aboleth", "Acolyte", "Adult Black Dragon", "Ad…
$ size <chr> "Large", "Medium", "Huge", "Huge", "Huge", "Hug…
$ type <chr> "aberration", "humanoid", "dragon", "dragon", "…
$ alignment <chr> "lawful evil", "any alignment", "chaotic evil",…
$ armor_class <int> 17, 10, 19, 19, 18, 19, 18, 19, 19, 19, 19, 18,…
$ hit_points <int> 135, 9, 195, 225, 172, 212, 184, 256, 207, 256,…
$ hit_dice <chr> "18d10", "2d8", "17d12", "18d12", "15d12", "17d…
$ speed <list> ["10 ft.", "40 ft."], ["30 ft."], ["40 ft.", "…
$ strength <int> 21, 10, 23, 25, 23, 25, 23, 27, 23, 27, 27, 22,…
$ dexterity <int> 9, 10, 14, 10, 10, 10, 12, 14, 12, 10, 10, 10, …
$ constitution <int> 15, 10, 21, 23, 21, 23, 21, 25, 21, 25, 25, 22,…
$ intelligence <int> 18, 10, 14, 16, 14, 16, 18, 16, 18, 16, 16, 8, …
$ wisdom <int> 15, 14, 13, 15, 13, 15, 15, 15, 15, 13, 13, 12,…
$ charisma <int> 18, 11, 17, 19, 17, 19, 17, 24, 17, 21, 21, 12,…
$ proficiencies <list> [<data.frame[5 x 2]>], [<data.frame[2 x 2]>], …
$ damage_vulnerabilities <list> [], [], [], [], [], [], [], [], [], [], [], []…
$ damage_resistances <list> [], [], [], [], [], [], [], [], [], [], [], []…
$ damage_immunities <list> [], [], "acid", "lightning", "fire", "lightnin…
$ condition_immunities <list> [], [], [], [], [], [], [], [], [<data.frame[1…
$ senses <list> ["120 ft.", 20], [12], ["60 ft.", "120 ft.", 2…
$ languages <chr> "Deep Speech, telepathy 120 ft.", "any one lang…
$ challenge_rating <dbl> 10.00, 0.25, 14.00, 16.00, 13.00, 15.00, 14.00,…
$ xp <int> 5900, 50, 11500, 15000, 10000, 13000, 11500, 18…
$ special_abilities <list> [<data.frame[3 x 3]>], [<data.frame[1 x 3]>], …
$ actions <list> [<data.frame[4 x 7]>], [<data.frame[1 x 4]>], …
$ legendary_actions <list> [<data.frame[3 x 4]>], [], [<data.frame[3 x 4]…
$ url <chr> "/api/monsters/aboleth", "/api/monsters/acolyte…
$ subtype <list> <NULL>, "any race", <NULL>, <NULL>, <NULL>, <N…
$ reactions <list> <NULL>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>…
$ forms <list> <NULL>, <NULL>, <NULL>, <NULL>, <NULL>, <NULL>…And there we have it. Our data is now in a pretty good state for some analysis. Depending on what we’re interested in doing, we could also do some additional feature engineering on the list columns, but the choices there will be contingent on the analyses we want to do.
For my next blog in this series, I’ll use this data to do some exploratory analysis, which I hope to get to in the next week or so.
Reuse
Citation
BibTeX citation:
@online{ekholm2020,
author = {Ekholm, Eric},
title = {Dungeons and {Dragons} - {Part} 1},
date = {2020-12-18},
url = {https://www.ericekholm.com/posts/dungeons-and-dragons-part-1},
langid = {en}
}
For attribution, please cite this work as:
Ekholm, Eric. 2020. “Dungeons and Dragons - Part 1.”
December 18, 2020. https://www.ericekholm.com/posts/dungeons-and-dragons-part-1.