Quick Refresher

It took a while for me to get back to this (sorry, a lot of things happening in the world right now). But this is the second entry on my approach on how to use {bookdown} to create a manuscript for a paper in the life sciences (biology, biomedical engineering, etc.).Last time I covered the reasons why we should stop using Word to write papers, and a basic structure for the project.

I realized I needed to create a Repo to keep track of the steps I am describing here, so if you click on the GitHub on top of this blog entry, it should direct you to the Repo that I will be using as an example. So far, our project looks like this:

We have created a project in R (using RStudio), and have directories for figures, data, code, and manuscripts. In this post, I will cover how we can structure the different sections of the paper.

Disclaimer A large part of the motivation for writing these entries about writing a manuscript comes from my desire to keep a record of the things I’ve done, it is possible that my suggestions here might not align with your needs or your own approach to writing a paper using the tools I describe here. However, I believe the overall description can be useful to people that are interested in creating reproducible documents using R, and you can certainly customize everything to fit your needs!

With that said, let me provide the rationale for the approach I’ll follow here.

The Approach

Let’s say that you have now a directory structure like the one in the image above, and you want to start writing the actual paper. If you have already have clean data files, and scripts for your Stats, what usually needs to be iterated (sometimes, a lot) are the paper sections themselves. That is, you might write a section, get some feedback and re-write such section again, but the data and Stats remain the same. If that is the case, one approach that has worked for me is to create another directory inside the manuscripts directory, which will contain the first version of the paper. Let’s say we call this new subdirectory manuscript_v1. Once you create it, your directory will look like this:

Inside this folder, we create another folder (sections) that will contain the file(s) that make the different parts of our manuscript. For example, the typical sections of a paper are:

• Introduction/Background
• Materials and Methods
• Results
• Discussion
• Conclusion

Why do we need to create different files? Having a file for each section makes things a lot easier! (The first time I used {bookdown} I started writing the whole paper on a single file and it gets messy and confusing quickly).

So let’s create another folder inside manuscript_v1 called sections.

The files that {bookdown} uses to create a document are RMarkdown files (*.Rmd). There are multiple ways to customize an Rmd file (more on that later), but for the time being we just need to create a plain, empty file. To create an empty Rmd file in RStudio, you need to go to File -> New File -> R Markdown -> Click on “Create Empty Document”. To keep things tidy, we will add a number to the files we will create. Once you have created and saved all your files, your directory should look like this:

Now the fun begins (writing)! How do we write in {bookdown}? Briefly, we only need to add some symbols when writing to indicate the sections of our document. For example, if we go to the 01_Introduction.Rmd file, we could start writing something like this:

Notice that we added a # before ‘Background’ and ## before ‘Another paragraph’. These symbols basically indicate that they are titles for a section and a subsection, respectively (I won’t cover all the details of Markdown syntax, as an excellent guide can be found here).

The take home point is that once you have created the files, you basically just need to start typing your text and indicate your section and subsection titles. Now, you have created the overall structure of the manuscript (I’ll cover references in another post), but we can’t get a document still, as we haven’t told {bookdown} what formatting options we want.

How we do it

Alright, so we need to create a file that will tell {bookdown} how to assemble the pieces of the manuscript, and what formatting options we want. To do this, we can create an Rmd file on the manuscript_v1 directory. The rationale for this is that in this way, we week the parts of the document on a different directory and this avoids having lots of files in the same location (i.e., you will only see and deal with files that are associated with formatting, but the pieces of the paper will be in the sections folder).

Let’s call the file that assembles the whole thing “Complete_paper.Rmd”, and let’s create this file on the manuscript_v1 directory. To do this, you will go to File -> New File -> R Markdown but, unlike previous times, just click “OK” without changing the default options that appear below:

(Also don’t worry if the Title appears as “Untitled”, we will fix this later).

Once you create the file, it should look like this:

The YAML Header

At the top the document is the YAML header, where we provide formatting instructions to {bookdown}. Notice that the default output at this point is HTML, but generally speaking, we are interested in creating a PDF document so we will edit the YAML section to pass some formatting options that are more convenient for a paper.

NOTE: The YAML section is very useful but it can be very tricky sometimes because blank spaces do matter. If the spaces in a line are wrong, the document may not be compiled or the formatting can be ignored. I can’t count the number of times I had a problem with a document and spent a lot of time trying to figure out what part of the YAML header was wrong!

You can ignore the content of the default Rmd template, as we will cover how to create figures later on, and we already know how to create title sections and add text.

A modified YAML header looks like this:

title: 'Manuscript'
output:  
  bookdown::pdf_document2:
    fontsize: 12pt
    fontfamily: helvetica
    keep_tex: yes #keep LaTeX file for submission
    fig_caption: yes #allows captions in figures
    toc: false #do not include table of contents
    extra_dependencies:
      subfig: null #allows for subfigures
      breqn: null #line breaks for long equations
      caption: ["font={small}"] #size of the figure captions
    includes:
      in_header: latex_docs/preamble.sty #additional LaTeX formatting
  bookdown::word_document2:
    fig_caption: yes #figure caption
    reference_docx: "docx_template.docx"
    keep_md: yes
  bookdown::html_document2:
    css: "html_docs/style.css" #style for the HTML document
csl: bibliography/american-medical-association.csl #style for references
bibliography: bibliography/refs.bib #references
link-citations: yes #adds links to the citations
urlcolor: blue
'': default
---

I have added comments to each line to help understand what each option does. In particular, with this YAML header we are able to create a PDF, docx, and HTML outputs if desired. The docx output is very useful for internal reviews, as is possible that other users not familiarized with {bookdown} find it easier to add comments in a Word document (formatting in Word is bad, though).

Additionally, we also now have a couple of additional directories, where formatting instructions for our different outputs will be stored. Note that we are including a LaTeX preamble.sty file. The reason for this is that to get a PDF, {bookdown} creates first a LaTeX file (more about how this is done here). You can edit the preamble file to add authors, or change formatting options if desired (each line also has comments to help understand the purpose of each LaTeX package that is being called).

The LaTeX preamble is just added so some options that are by default not available can be used as well (line numbers, double spacing, and a section for authors). LaTeX can be very complex (!), and in my experience, is not used much in the biomed/biology fields, but one of its major advantages is that it automates a lot of the formatting. Don’t freak out, we are trying to use the minimum possible of options to get a paper that is easy to ready and that looks great.

Additionally, we created a directory with a CSS file for HTML output formatting instructions, and added a file called “docx_template.docx”, which {bookdown} will use to create a Word output. Essentially, we have all of our bases covered regarding formatting!

Toward the end of the header, there are two lines that control the references and reference formatting, but we won’t cover that yet.

Assembling the Document

So much for a “short guide” eh? But we’re almost done with this part of the tutorial! The final part we need to look at is the body of our document, where we have added the instructions on how we want to assemble the final document. The body of the document now has some R code chunks that look like this:

{r child='sections/01-Introduction.Rmd'}

The purpose of this chunks is only to call the pieces of our paper in the order we want them to put them in a single document! This is great because in this way you compartmentalize your work, and if a section needs to be fixed, you open the corresponding file, edit it, and compile a new document. With a paper, where there are lot of text, images, and code, a single document can be very hard to navigate. Luckily for us {bookdown} eases the work.

Knitting the document

Finally, we are ready to compile the document! In this post I will only cover the PDF output, as I believe this is the most common type of document that suits a paper. Because in our YAML header we have added the option to create a pdf document (pdf_document2), we can select to create that output by clicking on arrow that is next to the “Knit” button:

By selecting “Knit to pdf_document2”, we will be able to get our desired PDF output. When the document is ready, a new window will open with the document! With the instructions we provided to {bookdown}, our output looks like this:

Pretty, isn’t it? Note that each section is numbered, we have line numbers (thanks LaTeX!), double space, and we have a section for the authors and their information. The rest of the sections beyond the Background in this case are empty, but each section and subsection (if added) will be formatted exactly in the same manner. A little lengthy at first, but I guess by now it is clear why using {bookdown} is largely beneficial when creating a paper: you are not fighting with formatting issues, and your document can be replicated if created in any other machine (for example, a reviewer may want to see if is possible to replicate your paper. Trust me, it happened to me!).

Recap

In this post I covered how to structure the sections of a paper using {bookdown}, adding some formatting options using YAML, and how a PDF output looks. In my next post, I will cover how to create figures using R, where you can combine images, and figures with statistical analyses together!

Why?

Let me start this post by sharing the reasons that lead me to write it. I have been an avid R user for some time now, and R has been my default option to clean, analyze, manipulate data, and create graphs. I started using {rmarkdown} to create reproducible reports for my analyses but it never occurred to me that I could write a complete paper with it. However, that changed a couple years ago I wrote a paper about generalized additive models for biomedical data using R. For that paper, I learned about {bookdown}, and how you could create books (or other long publication formats) with it. That was my first experience creating a paper using solely R, and I made many mistakes along the way (also learned a lot).

Specifically, it was hard for me to know how organize the document in a way that made things easier to track. Now, this organization issue probably wouldn’t had been a huge deal had I planned to only share the final paper, but back then I was starting to learn about Open Science and I decided that I had to make the whole project reproducible. By trial and error, I figured a way to structure the whole project in such a way that, if I wanted to recreate the whole paper, I only had to click the knit button once.

However, the next roadblock came when I worked in my final paper (as a PhD student) and I realized that probably one of the reasons why {bookdown} is not used more frequently by people in life sciences is because we deal with biological data and images. What I mean by this is that many people know about {rmarkdown}, {bookdown}, and other R packages are good for writing documents; however, these packages are typically associated with documents that are only composed of text and R-generated figures. But when you have imaging data and files (e.g., microscopy images, pictures of equipment, etc), you want to be able to create figures using those files and the first impression is that you can’t do that with {bookdown}. However, as I discovered, you can add imaging data to a document made with {bookdown}! Combining this fact and packages such as {patchwork} and good ’ole {ggplot2} it turns out you can compose great figures that combine statistical analyses, imaging data and more, and you can create a paper that is reproducible with one click. How neat is that?

Part 1

How the idea started

A couple weeks ago I read a Tweet from Alison Presmanes Hill where she commented on the beauty of Deisy Gysi’s site. And the site is truly beautiful, the background is mesmerizing! I was intrigued on how such animations could be implemented in a Hugo site.

I am no programming wizard (faaaar from it), but if there’s one thing I have learned during my PhD is to find information. So, I decided to start my quest by googling “hugo background animation”. Among the first results that I could see, there was one that caught my eye: “Customizing a Hugo theme with an animated P5.js background” (which can be found here).

Upon doing some reading two things were clear:

• Such customization required JavaScript

• The post was a little bit old (2018).

From seeing the structure of the folders described in the post I could tell that the Apero theme did not have the same structure. Moreover, because I knew that Hugo themes tend to have multiple changes over time, I was afraid that by making the proposed changes in the post I would end up messing things too much (it had happened before and I ended up burning everything to the ground from the Hugo Academic theme and migrated to the Apero theme, which had substantial benefits).

But one thing was clear, this was doable. However, the first thing I wanted to know was what was this p5.js thing. Well, turns out that p5 is a JavaScript (JS) library for creative coding. Watching the examples on the site was mindblowing! Take some time and look them up, because although most of them are in plain black and white, it’s easy to see the potential of using p5 to create beautiful artwork.

One of the examples is called “particles”, and when I saw it I realized that it was very similar to Geisy’s website background. Actually, there is a library called “particles.js” that makes it easy to use and customize (see here).

The quest continues

I had a starting point, I wanted to use p5.js, but I could not find a thorough explanation of how to use the library on a Hugo site. I noticed that some people had asked similar questions in the Hugo community, but honestly I lacked the programming experience to understand the explanations given, or they were too vague (multiple times I found something along the lines of “can be implemented later”).

However, one thing became apparent from reading these posts: If you are going to use JS on a Hugo site, the scripts need to be in a specific folder. I noticed most of the posts made reference to the root static/ folder as the place where .js files need to be placed. I also ended up learning that because of the way Hugo works, it is recommended that any changes you make on your site regarding style (CSS, fonts, etc.) are to be placed in folders in the root directory (such as static, assets, etc), rather than altering the theme itself.

When Hugo loads the site it looks on the root folder first and any changes you have made regarding style override the default theme settings. And because is easy to keep track of changes and additions that are made on the folders you create on the root directory rather than messing with the themes/hugo-apero folders, this saves you multiple headeaches later, which is something I can attest.

Back to the question at hand, I stumbled on this post, where someone asked about using something called D3.js in a Hugo site. Now, D3.js is not p5.js but there is a saying in Spanish that goes un diablo se parece a otro, which basically means that sometimes you can confuse two persons that look alike. In my naive mind, D3.js was close enough to p5.js and I thought that the procedure to implement the first had to work for the latter.

What you need to do

OK, so here is where things get interesting. Say you want to add the “particles” example from p5.js to a blog post. The first thing you need to do is to copy the code that appears below the example on the site and paste it in a file that we will call particles.js. The file needs to exist in a root folder called static, and for the sake of being organized we will further place it in a folder called js. In other words, the path is ../static/js/particles.js

So far, so good. Now, say you create a new blog post, which you will coveniently create using the Blogdown ‘new post’ Addin in RStudio. Say you create a blog post titled “Testing p5.js” and that you are going to make it using a .Rmd file.

Below the YAML header, you can write some text:

---
title: Trying a p5.js animation
author: ''
date: ''
slug: []
categories: []
tags: []
---

### Some clever text

Because HTML is supported in RMarkdown, first we need to call the p5.js library:

<script src="https://cdn.jsdelivr.net/npm/p5@1.4.0/lib/p5.js"></script> <!-- load p5.js from CDN--> 

And then we will call our particles.js script:

<script src = "/js/particles.js"></script> <!-- this will pick our script  -->

Just by adding those two lines, you should be able to see the animation running at the bottom of the page.

By default, p5.js objects (also called “sketches”) are added at the bottom of the page. Say you want to write some text, add a p5.js animation and continue writing below. How do you achieve this?

You need to do two things:

• In the .js animation file you created (in this case particles.js) and that now lives in static/ you need to find this line (assuming you haven’t changed anything yet):

createCanvas(720, 400);

And change it to:

var canvas = createCanvas(700, 470); //make it an object
canvas.parent('particles-holder');

As I said before, I am no JS expert, but intuitively, we’re making the canvas a type of “object” we can use a function on. p5.Element.parent() function allows to move the sketch inside an HTML file. So now, when we call it in our blog post, you can have something like this:

#### Some clever text

Here we have a particle simulation from p5.js.

```{=html}
<div id="particles-holder"></div>
<script src="https://cdn.jsdelivr.net/npm/p5@1.4.0/lib/p5.min.js"></script>
<script src="/js/particles.js"></script>
```

More text to come!

The result is what you see below, we have some text before the sketch, we call the sketch, and we write more text below.

As you can see, there is some work to do but it is honestly very rewardint to add such cool animations and visualizations to your site.

Final thoughts:

• There are multiple ways of doing this. This is what worked for me this time, but I am sure more improvements can be done (as when you want multiple sketches on a page, which is covered here).

• I recently migrated to the Hugo Apéro theme. I have to say that it is extremely well done. The Wowchemy Academic theme was usable, but things were confusing at times and documentation was, well, poor. It is nice to see a theme that makes customizations easy, and that is well structured.

Use “chapters”

When I started working with Bookdown I was making a single document for my manuscript. Doing it in that way was manageable at the beginning, but as I wrote more and added more code, scrolling down started to become painful (specially after I realized one day that I was at about 1000 lines in length between code/text). I decided that I wanted to break the document in pieces, but I didn’t know how. Reading through the Web I came to realize that you can piece a Bookdown document using the child option (see here for a thorough description). The child option means that you will break your document in “Chapters” but they will be knitted in a single document when you compile the PDF/HTML output. For this to work you need in the same working directory:

• A “central” document that will contain the YAML header and the instructions to call all the other documents.
• All your “chapters”

The “central” document will contain all the formatting instructions (YAML header with bibliography, references styles, options, etc) and chunks of code where you call the “chapters”. Suppose you call this “central” document Main.Rmd, and that you have in the same directory two “chapters”: Chapter 1.Rmd and Chapter 2.Rmd.

In Main.Rmd, after they YAML header and the chunk where the knitr options are specified, you need to add the chunks that will call the chapters, like this:

```{r child='Chapter 1.Rmd'} `r ''`
```

```{r child='Chapter 2.Rmd'} `r ''`
```

The very convenient thing about using this approach, is that you can move across the chapters and find things way more quickly than using a single document; you can also check bugs in your code without constantly scrolling up and down. Keep in mind that all your packages need to be loaded in Main.Rmd, the “chapters” do not have a YAML header or an R chunk for options, you can start them just with text. If you were to open the file for the first chapter it would look like this:

• Chapter 1.Rmd

# Chapter 1: Introduction

In this section we will review the current status of the field of interest.

“Why don’t you try RMarkdown?” and moving to Bookdown

A while ago I started to work on a manuscript, and during a meeting with the co-authors one of them said “Why don’t you try RMarkdown to write this project?”. Until that moment, I had only used Word to write scientific documents, and a quick look on the Internet confirmed my initial suspicion that using RMarkdown was a complete different experience than using Word. Nonetheless, I also had a feeling that learning to do things in RMarkdown had to be beneficial, so I decided that it was worth to give it a shot. If I didn’t like how things were going after a week, I would go back to Word.

Fast forward a couple days later and I’m starting to understand how things work on RMarkdown, and I also realize that I need to cross-reference an equation in the manuscript. A Google search indicates that to do this, I should probably use Bookdown. A quick glimpse on the Bookdown manual also indicates that Bookdown offers more capabilities than RMarkdown to reference equations and figures. So I start working on Bookdown. As I work on the manuscript more and more, I find that using Bookdown is substantially more beneficial to write a document than Word. Equations? Easy, and unlike the equations made in Word, here they look gorgeous, textbook-like. Citations? A small learning curve, but you understand what you are doing and control the amount of information you have for each reference. Above all, one of the major advantages is that when creating figures in R you don’t have to mess with importing the figure and fighting with its placement in the document, which is always a pain in Word!

However, about a week ago I decided to work on some minor things, and had to add the author information on the manuscript (which hencefort I will call Manuscript.Rmd). There are three authors, two of which (including myself) are from the same academic institution. I try the option of listing the authors in the YAML header as the Bookdown manual suggests (title and author information in this case are fake):

---
title: '**A longitudinal analysis of Twitter trends in the year 2020**'
subtitle: _Political and sociological considerations_
author:
- Benicio del Ox^[Department of Sociology, University of the Insula]
- Jake A. Motaungh^[Department of Sociology, University of the Insula]
- Frederick P. Montego^[Department of Psychology, University of Mars]
---

The main problem here is that when rendered, the header looked like this:

While the footnote looked like this:

Its clear that every author was being assigned a “different” institution. I was naive enough to think that Bookdown would “detect and merge” the affiliation of twor or more authors when it was the same, but that was clearly not the case.

I couldn’t find a satisfactory answer to this problem in the Bookdown Cookbook or in the forums I consulted, but what I did found was that there is a package called authblk that allows for better control on the author information. This search also lead me to this post on Rbloggers with a proposed solution on how to use authblk in a Bookdown document.

However, the syntax offered by the post didn’t work for me, and after repeatedly trying to make the YAML header to behave (YAML headers are quite temperamental and picky about indentations!) I started to think that I would probably have to submit a question about this issue in some specialized forum (which would involve creating a reprex, creating an account, registering, and the list goes on…).

After reading for a while again, I discover (yes, I know this is pretty well known to almost anyone, but it felt like a great discover to me, nonetheless) that more control over the creation of the document can be achieved by using a .sty file. In short, the .sty file can contain some code that allows to customize the final document (longer explanation here). This discovery was really convenient as this approach would allow me add line numbers, which is something I also wanted to do on the document.

So I proceeded as follows:

• Creating a .sty file called “preamble” that was the same directory where Manuscript.Rmd was located.

• In preamble.sty:

\usepackage{lineno}
\usepackage{authblk}

%command for the package lineno to put line numbers in the manuscript
\linenumbers

%authors
\author[1]{Benicio del Ox}
\author[2]{Jake A. Motaungh}
\author[1]{Frederick P. Montego}
\affil[1]{\footnotesize Department of Sociology, University of the Insula}
\affil[2]{\footnotesize Department of Psychology, University of Mars}

Then, in the YAML header of Manuscript.Rmd I added two lines to indicate that code is in preamble.sty. The two lines are includes and in_header:

---
title: '**A longitudinal analysis of Twitter trends in the year 2020**'
subtitle: _Political and sociological considerations_
output:  
  bookdown::pdf_document2:
    keep_tex: yes
    includes:
      in_header: preamble.sty
---

Alright! And the document looked like…

Huh… that was odd. It was nice to see that the document had line numbers, but you can notice that there is an additional “and” after the last author along with a superscript for the affiliation. This was the most frustrating thing, I am no expert and I couldn’t find the cause of the problem, although I suspected that there was a conflict between the Pandoc LaTeX template and what the authblk package was doing. For this one, I was out of ideas after several hours of trying to fix the issue and decided to submit an issue on the Bookdown GitHub site.

I got a very detailed response from the Bookdown team the next day (yesterday). So I decided to add the explanation and the solution here before I forget. It goes like this: RMarkdown uses a Pandoc template (which can be found here)to build documents, and in that template there is an ‘author’ line that includes an \and command. In that template the field is not conditional, meaning that it will always appear even if you already have the information in a separate .sty or .tex file. Because in this case I have no ‘author’ information to pass to the template (as the author information is in the .sty file) the template author field remain blank but the \and will always appear.

There are two solutions which the Bookdown team provided (here):

• Use a custom LaTeX template that does not include the \author line.
• Add a function in the .Rmd document that post-processes the file through bookdown.post.latex(more information about this function here)

I tried both options. In the first case, you only need to edit the original template, save with a custom name in the same directory where your .Rmd file is and call it in the YAML header like this:

bookdown::pdf_document2:
  template: my-template.tex

However, it seems that maintaining a custom template can be tricky if there are updates over time. So I also tried the second option, which is the function provided by the Bookdown team and that you have to place in a setup chunk at the beginning of the document. It can also be placed in the chunk where the knitr::opts line would be. The function is:

remove_author <- function(x) {
  # identify empty author line
  i <- grep("^\\\\author\\{\\}$", x)
  # be sure it is the one pandoc inserts
  if(length(i) != 0 && grepl('^\\\\date\\{', x[i+1])) x <- x[-i]
  x
}
options(bookdown.post.latex = remove_author)

In this way, the default template is called, but the function will remove the author line. I found that this worked nicely, with the advantage of not having to worry about issues with a custom template. After compiling the PDF document using the function the final result looks like this:

Which is what I wanted. Quite a journey to have this resolved, but I learned a bunch of stuff along the way! I hope this post helps anyone going through the same issues.

A couple of final thoughts:

• is challenging but there’s whole world of things to experiment with!
• YAML headers are temperamental. Sometimes it’s good to use a YAML validator to check it everything works. With more and more options and custom things in it, there is more chance that an indentation is not right and you can spend hours trying to find the problem (trust me, I’ve been there)
• Also, I am a convert. Only Bookdown to write documents from now on!

<script src="penrose.js"></script>