Pages

Showing posts with label markdown. Show all posts
Showing posts with label markdown. Show all posts

Tuesday, April 14, 2015

Breaking Up With Powerpoint

I’m breaking up with powerpoint. I’ve known this day would come for a while now, but its shocking its finally here. There are academic arguments for its abandonment, but none really compelled me. The honest truth is I’ve finally found something better.

Two weeks ago I did something new, I wrote and delivered a presentation on graph theory and interactive data analysis to a mixed crowd of upper year undergrads and grad students. What was special about this presentation was that powerpoint was nowhere to be seen, not a familiar microsoft trapping in sight. This presentation was a beamer presentation. I wrote it in R Markdow supplemented with some raw LaTeX, and now I don’t think I can ever back. I’m breaking up with powerpoint, and I think you should too.

Background

If you’re not familiar yet with R Markdown, I recommend you go back and read my introduction to Knitr and R Markdown. It has become so natural to do the bulk of my work without leaving the comfortable confines of R Studio that I keep looking for more tasks I can do without switching software. Presentations were a logical next step. Previously I had dipped my toes into the problem of authoring presentations with markdown. I used ioslides (another presentation format offered by R Markdown), but was unsatisfied with the level of customization I could achieve (with my primitive knowledge of javascript). So I tried again, but this time with beamer. Beamer for those who haven’t spent much time swimming in the LaTeX pools is a convenient package for rendering LaTeX code as pdf slide-decks. I encountered beamer for first when I tried learning LaTeX originally but never had enough time or drive to master it. However, now with the added ease offered by R Markdown I decided to give it another shot.

These are the results:

The code is available from my github

Other than the relatively uninspiring title page the document came out beautifully. Figures rendered wonderfully, code seamlessly integrated into the slides, natural sub-sectioning, I can’t wait to write more like it. I recommend you quickly scroll through the document to see just how simple the document turned out to be (after code headers).

Getting Started

I won’t lie, there were quite a few gotchas1 along the way, but you get the opportunity to learn from my mistakes. To start a beamer presentation in R Studio, create a new markdown document as I discussed in the post about markdown, but instead of choosing the default settings, click the panel labelled presentation, then select beamer and ok

R Studio throws in some demonstration slides to give you a taste for how to make your presentation, you can go ahead an delete that (though keep the yaml block at the very top [the stuff enclosed by — ]) because I’ll walk you through how to write a really simple presentation.

First Gotcha, YAML Headers, and Themes

A problem I ran in to (yet haven’t done my due dilligence and reported it) was that I couldn’t resize the code in my document. As there isn’t a burgeoning community of R Markdown –> Knitr –> Beamer users, tracking down which component of the pipeline isn’t working right and finding a fix is challenging. I found references to a workaround by Yuihui Xie (the creator of Knitr) for getting the code to the right size, but it didn’t work for me, and supposedly is no longer necessary anyway. He was using Knitr –> Beamer so the issue could be in R Markdown. I created a work-around that made the code font smaller but left the output font gargantuan, it was sufficient for my purposes. You can grab the modified template I used from my github by running:

library(RCurl)
gistUrl <- "https://gist.githubusercontent.com/cfhammill/b5ba7767d7729bd676a2/raw/987d43694eda1fc263efdd38af03f846db80e690/resizeTemplate.beamer"

write(getURL(gistUrl), "resizeTemplate.beamer") 

Then you can add template: resizeTemplate.beamer to your yaml header. Also if you’re interested in using a theme to beautify your document you can add that in the header as well:

---
title: "A title"
author: "Your Name"
date: 'Today's date'
output:
  beamer_presentation:
    theme: "Boadilla"
    template: resizeTemplate.beamer
---

I used the theme boadilla but there are many others to choose from. To find the theme that’s right for you check out the gallery by Ian Blaines to see one presentation rendered in many different themes.

Slides

Once that is set up, you can start writing your presentation. By default, new slides begin with every level 2 header, or line break. To create two slides (plus your title slide) you can add the following code to get a titled slide and an untitled slide:

## Slide 1

Some Slide Contents!

------------------

Untitled slide 2

Images

Next thing you might want to try is to add some images into your documents.

To add pictures, you can use the default markdown code:

![](path/to/pic.png)

But I found myself unsatisfied with the default sizing and positioning. I wanted a centred picture of a certain size. To achieve that I needed to write some raw LaTeX

\centering \scalebox{0.45}{\includegraphics{path/to/pic.png}}

Centering indicates the line should be centered, and since LaTeX treates included graphics as large characters that will center your image. The \scalebox command resizes the image as you’d expect (with numbers larger than one expanding it). All and all not too complicated.

Bullet Points and Sequence

To have a series of bullet points in your slide you just need to create a bulleted list the default markdown way

#Bulleted Slide

- Isn't
- This 
- Easy

And you’ll get a nice bulleted slide. If you’re like me and want some but not all of your bullet points to come in sequentially you can add incremental = true to your yaml block after beamer_presentation: but I found it easier to leave that out and specify manually where I’d like my bullets to be sequential. To force sequential bullets (or if incremental is true to force static bullets, which isn’t documented on R Markdown’s webpage) you just need to add the greater than sign before the bullet.

#Sequential Bullet Slide

>- Wait for it
>- .
>- ..
>- ...
>- Point!

Images can be made sequential too by putting them in a sequential bullet.

Bullet Spacing

The applies to line spacing in general, markdown ignores extra white space by default, so trying to force extra space between points isn’t as easy as one might hope (although there is probably a way to do it with your LaTeX header or yaml header). The solution I found was to manually include LaTeX line-breaks

#Spaced Out Bullets

- Point 1 \newline
- Point 2 \newline
- Point 3 \newline

Which is useful if you, like me, try to keep text to a minimum so using white space effectively is key.

Resizing Font

To resize font in your document you can use LaTeX’s font sizing codes e.g. (\large{your text}, \Large{your text}, \tiny{your text}, etc.)

This was useful for me to make better use of the slide space with sparse text (lots of line spacing and a bigger font), and for emphasis without using headers which can trigger some unwanted stylistic changes.

Outro

With that you now know about as much as I do about creating presentations with Beamer via Knitr via R Markdown. It’s pretty straight forward, if you ever do presentations that involve code, equations, and figures I can’t recommend it enough. I hope you’re inspired to try your next presentation without powerpoint.

-Chris

Bonus Trick For Those interested: in the presentation, the red X and green check mark were made using grid graphics directly from within R. I previously wrote a little about using ggplot2 in unexpected ways, this used some of those lessons. By using the grid package directly you can draw whatever you like on a plot canvas, check out the presentation code for how I did it.


  1. “Gotcha” is a programming term for a little irksome quirk of a language or tool that cause it to perform in unexpected or counter intuitive ways

Tuesday, February 24, 2015

Writing a Knitr Engine to run code through an interpreter of your choice!

Preamble

Recently I took a job with the Department of Fisheries and Oceans translating some statistical software from python into R. The package uses kernel smoothing to estimate the probability distribution of a variable of interest relative to one or more covariates. I’m employed for the next month or so, and hopefully by the end we’ll be submitting the package to CRAN. I’ll write a little more about that as it gets closer to happening.

I’ve gotten to the point in the development process where I need to think hard about optimization and benchmarking. I’m in the habit of keeping my bosses/colleagues up to date on my work through html files I make using R markdown (the same way I write this blog). R markdown has drastically changed the way I work, allowing me to weave narrative and code together to make a more compelling exploration of my progress. I was hoping to benchmark my code against the original python code to see how they stack up speedwise and thought it would be great to have the python benchmarks right in there with my R code. I knew knitr provided facillities for including non-R code in documents so I set to work figuring our how to do it.


Back to basics

For those of you who don’t know the fantastic one-two punch that is R markdown and Knitr let me tell you a little bit about it. R, since at least the early 2000’s, has provided facillities for (s)weaving code and documents together. The aim was to mesh analysis and reporting into one cohesive action in the name of reproducibility. Though there is still much progress to be made moving toward this ideal the tools have gotten better.

In 2012, Yihui Xie introduced the world to knitr, a neater knitter of code and documentation. It has (almost?) completely supplanted sweave as the primary tool to acheive this goal. The benefits of knitr are many, it allows you to create html and pdfs, include code, figures, and LaTeX all in the same document, and rerun the analysis any time the included code changes (but only if you want it to). Needless to say I’m a big fan.

The next major innovation in the reproducible research game was R Markdown. Markdown is family of markup languages introduced by John Gruber in 2004 designed to facillitate writing html documents in plain text. The goal was to have readable plain text documents where the html tags could be generated on demand. R markdown is variant designed to work primarily with R, created by the folks at R Studio. If you use R Studio, the integration of R markdown is seamless, you write an R markdown formatted file, the documents is converted to a format knitr understands, and then knitr makes you a beautiful document with all of your work ready to be shared with the world. I highly recommend using these tools for just about any authoring job short of manuscript preparation.

Now that I’ve given you a brief introduction, lets dive into my oddessy making a knitr engine.


The problem

As I mentioned in the preamble, I wanted to run R and python code side-by-side in my document so that both sets of benchmarks could be readily compared. Knitr comes pre-loaded with a quite a few engines for other languages:

library(knitr)
names(knit_engines$get())
##  [1] "awk"       "bash"      "coffee"    "gawk"      "haskell"  
##  [6] "node"      "perl"      "python"    "Rscript"   "ruby"     
## [11] "sas"       "scala"     "sed"       "sh"        "zsh"      
## [16] "highlight" "Rcpp"      "tikz"      "dot"       "c"        
## [21] "fortran"   "asy"       "cat"       "asis"

A quick scan of the available engines shows that knitr should be perfectly capable of executing python code. So I tried:

```{r, engine = “python”}
#My python code
```

Which sadly did not work as I had hoped. The python interpreter used by knitr in that situation was the stock python interpreter. It failed to properly import the modules I needed and was untenable for my objectives. “Alas, if only there was an ipython engine” I thought to myself, and then promptly decided I should probably build one.


Taking apart the engine

To figure out how to build my own engine, I decided to have a look at the engine provided for me. A convenience of R that I think more people should take advantage of is the ability to get code for any function you like. Provided it’s written in R in many cases you can figure out how it works. You can skim over this code, I’ll get back to the important bits.

knit_engines$get("python")
## function (options) 
## {
##     engine = options$engine
##     code = if (engine %in% c("highlight", "Rscript", "sas", "haskell")) {
##         f = basename(tempfile(engine, ".", switch(engine, sas = ".sas", 
##             Rscript = ".R", ".txt")))
##         writeLines(c(switch(engine, sas = "OPTIONS NONUMBER NODATE PAGESIZE = MAX FORMCHAR = '|----|+|---+=|-/<>*' FORMDLIM=' ';", 
##             haskell = ":set +m"), options$code), f)
##         on.exit(unlink(f))
##         switch(engine, sas = {
##             saslst = sub("[.]sas$", ".lst", f)
##             on.exit(unlink(c(saslst, sub("[.]sas$", ".log", f))), 
##                 add = TRUE)
##             f
##         }, haskell = paste("-e", shQuote(paste(":script", f))), 
##             f)
##     }
##     else paste(switch(engine, bash = "-c", coffee = "-e", node = "-e", 
##         perl = "-e", python = "-c", ruby = "-e", scala = "-e", 
##         sh = "-c", zsh = "-c", NULL), shQuote(paste(options$code, 
##         collapse = "\n")))
##     code = if (engine %in% c("awk", "gawk", "sed", "sas")) 
##         paste(code, options$engine.opts)
##     else paste(options$engine.opts, code)
##     cmd = options$engine.path %n% engine
##     out = if (options$eval) {
##         message("running: ", cmd, " ", code)
##         system2(cmd, code, stdout = TRUE, stderr = TRUE)
##     }
##     else ""
##     if (!options$error && !is.null(attr(out, "status"))) 
##         stop(paste(out, collapse = "\n"))
##     if (options$eval && engine == "sas" && file.exists(saslst)) 
##         out = c(readLines(saslst), out)
##     engine_output(options, options$code, out)
## }
## <environment: namespace:knitr>

First good sign, all the code is R with no references to byte-code and it’s short enough to not take all day to figure out. First thing to note about the code is it’s a function, and not only that it’s a function of one argument, options. The first line of code takes a named element of options called engine (so options is a named list containing things like the name of the engine). My intuition was that options was a list of every component of chunk, and the rest of the function largely confirmed that intuition. Other named elements of options used in the function include:

  • code
  • engine.opts
  • engine.path
  • error

The shape of the code the begins to come into focus. The function primarily takes your code out of the chunk (element code), converts it into shell command that runs engine with some flags and a quoted argument containing your code. This gets executed by the function system2 which is used to execute shell commands and retrieve output. This output is then passed to a function engine_output along with the code and options, the results of which are used to build your document. So from this point it seamed obvious what needed to happen.

  1. Concatenate engine.path and engine to create the command that needs to be run
  2. Convert code into a string appropriate for use in shell commands
  3. Paste flags onto code so that the engine knows to expect a string of code
  4. Run the command and the modified code through the engine with system2, saving the output
  5. Pass everything to engine_output

Pretty straight forward really, and except for some simple housekeeping that’s all I needed to write.


Writing the new engine

Now I began writing my own engine (and it didn’t work the first time, some of the housekeeping I didn’t figure out right away, but lets pretend it did). I called my engine ipythonKnitEngine but you can call your engine whatever you want.

ipythonKnitEngine <- function(options){
  
  enginePath <- options$engine.path
  if(grepl("/$", enginePath)) enginePath <- paste0(enginePath, "/")
  
  engine <- paste0(enginePath, options$engine)
  
  code <- paste("-c", shQuote(paste(options$code,
                                    collapse = "\n")))
  
  out <- system2(engine, code, stdout = TRUE, stderr = TRUE)
  
  engine_output(options, options$code, out)
  } 

As of writing this, the engine requires you specify the path to your ipython executable, its a kink I’ll probably iron out in the near future, but this was a quick fix to finish a document, so it still needs some polish.

For those who aren’t familiar shQuote converts a character vector to a string useable in your shell. The other potentially obtuse bit is the collapse = "\n" to paste. When paste takes a character vector of length n, by default it makes a vector of length n, when collapse is not NULL, all of those elements are concatenated together separated by the collapse argument. Otherwise pretty standard stuff.

The last necessary little bit of funny business is engine_output isn’t actually available from the knitr package to users, but you can force your engine to use knitr’s namespace by setting your engine’s environment:

environment(ipythonKnitEngine) <- environment(knit_engines$set)

Now the environment of the function is set to the knitr namespace and it will be able to find engine_output.

And finally we can set the new engine and begin using it!

knit_engines$set(ipython = ipythonKnitEngine)


Using the engine

To use the engine in your R Markdown / Knitr documents just run library(knitr) in your first code chunk, and include the function declaration for your engine along with the environment and knit_engines$set statements above and then you can use your engine in your document.

Now we can run

```{r, engine = “ipython”, engine.path = “path/to/ipython/”}
#Python code to execute
```
with impunity and it works like a charm!.


Wrap-Up

I hope you liked this little tutorial. If you like to say “nuts to writing my own, I’ll just use yours” you can find all this code in a convenient format on github.

Chris