GOAL | To unpack the data from nearly 1,000 pdf files into one cohesive file for analysis.

RESULTS | After writing many if/else statements and loops, one data table was produced for analysis.

RECOMMENDATION | To continue to use this code in the future when data needs to be scraped from PDF files.

Setup of Scraping Data

Every file had eight tables (Figure 1). Each table represented one of the eight competencies and the attributes that made up that competency. For example, the employee in Figure 1 was rated a 7 in three of the four attributes that make up the effort competency.

We needed the data from those PDF files. Initially, there seemed to be only one solution: go through each PDF file and manually type the data into an excel sheet. However, this process would have taken too many hours to complete. I knew that there must

be a way to scrape the data from the PDF files through R Studio. It took about a week to write a series of loops to pull the data from the PDF files and write the data to one large dataset. It was surprisingly easy to read the PDF files into R Studio with one line of code (Figure 2). R Studio simply read the lines of each table.

Three problems

However, I soon ran into three main problems.

1. Individuals with middle initials

2. Space where ratings should be

3. Employee ID number (i.e, “Perner”) and employee name were switched

Let's start with problem #3. 

I pulled the employee's name and ID number from the first page of the PDF. Unfortunately, some PDFs had the name and ID number in different places. Thus, I wrote an if/else statement (Figure 3) to help with this problem! If the first line contained letters, then it was the employee's name. If the first line did not have letters, then the first line was the employee's ID number. I have fixed Problem #3!

I wanted to pull the column headings only once from each file. But I needed a find a way to locate the column headings. To do this, I used the employee's name. So, first, I created a variable called "ind" (Figure 4) to find the location of the employee's name within the PDF file. After examing the output of "ind," the optimal row to pull the column headings was from the fourth mention of the employee's name. 

However, the inconsistencies caused by problem #1 warranted an if/else statement (see Figure 4). The first two "if" statements pulled the column headers from the line where the middle initial matched the spacing from the fourth mention of the employee's name. This way, all of the column headings would match no matter the length of the employee's name. I have fixed Problem #1!

Now I was ready to pull the employee competency ratings for all eight competencies. Starting with the first competency, "effort,"