In-class Exercise 5a

5.1 An exploration of VAST Challenge 2024 - MC1 data

5.1.1 Loading the necessary R packages

pacman::p_load(tidyverse, readtext,
               quanteda, tidytext)

5.1.2 Loading the data

data_folder <- "data/MC1/articles"

5.1.3 Text sensing to extract text

text_data <- readtext(paste0("data/MC1/articles",
                "/*"))

OR the below code chunk can be utilised as well

text_data <- readtext("data/MC1/articles")

5.2 Basic tokenisation

usenet_words <- text_data %>%
  unnest_tokens(word, text) %>%  #reading the text data
  filter(str_detect(word, "[a-z']$"),
         !word %in% stop_words$word) #remove stop words

usenet_words %>%
  count(word, sort = TRUE)

readtext object consisting of 3260 documents and 0 docvars.
# A data frame: 3,260 × 3
  word             n text     
  <chr>        <int> <chr>    
1 fishing       2177 "\"\"..."
2 sustainable   1525 "\"\"..."
3 company       1036 "\"\"..."
4 practices      838 "\"\"..."
5 industry       715 "\"\"..."
6 transactions   696 "\"\"..."
# ℹ 3,254 more rows

Observations- Most common words are: fishing, sustainable and company

5.2.1 Creating a table to observe word counts

temp_table <- usenet_words %>%
  count(word, sort = TRUE)

5.2.2 Using corpus to read text data

corpus_text <- corpus(text_data)
summary(corpus_text, 5)

Corpus consisting of 338 documents, showing 5 documents:

                                   Text Types Tokens Sentences
 Alvarez PLC__0__0__Haacklee Herald.txt   206    433        18
    Alvarez PLC__0__0__Lomark Daily.txt   102    170        12
   Alvarez PLC__0__0__The News Buoy.txt    90    200         9
 Alvarez PLC__0__1__Haacklee Herald.txt    96    187         8
    Alvarez PLC__0__1__Lomark Daily.txt   241    504        21

To separate the data; into 2 columns X & Y.

text_data_splitted <- text_data %>%
  separate_wider_delim("doc_id",
                       delim = "__0__",
                       names = c("X", "Y"),
                       too_few = "align_end")

Some text are starting with “1” hence the split does not occur