Introduction to Bayesian Statistics
About this course
This is a week-long short course meeting for 3 hours each day; it is a course offered through UT's Department of Statistics and Data Science, with students of all ages and backgrounds attending (though it is reasonable to assume that interest in a course of this nature presupposes familiarity with basic statistics).
This course (1) introduces students to probability basics (joint, marginal, and conditional probabilities), Bayes' rule, probability distributions (PDFs) and likelihood functions, Bayes' rule as applied to both discrete and continuous probability distributions (prior, likelihood, and posterior); (2) applies this knowledge to real statistical scenarios that arise in day-to-day research.
(note that all of the text in this box is summarized in the document above)
Before coming to class:
Students will watch 1-3 short videos covering foundational pre-requisites and illustrating concepts to be covered in the upcoming class; this ensures pre-processing of the material and makes it more likely that all participants will be “on the same page” in class. These videos and built-in exercises will also be used to off-load practice honing basic skills in order to make the best use of limited class time. The assessment questions built into these videos will allow for me to evaluate how students are keeping up, and will also allow them to monitor their own understanding; the first two days of the course will have the largest quiz component.
In class, the videos will be recapped in the form of a mini-lecture that embellishes the online component. I will generate graphics summarizing students' performance on the assessments, and we will take the necessary time address any issues that remain unclear.
During class (face-to-face)
Mini-lectures will be directly followed by application assignments in which students are given a statistical scenario and required to carry out the correct analysis on a sample dataset using the techniques presented. These will be scaffolded at first, but assistance will gradually be faded out: at first, I will walk them through the analysis step by step, having them enter commands into R, but by the end of the second day, students will be comfortable enough to use the R-script themselves and to make their own modifications.
After each illustrative example, students will conduct similar analyses independently; then small groups will compare their results with one-another. Then each small group will reveal their solution; if there are any differences, these will be puzzled out by the class as a whole.
Extending, applying, and reflecting on learning
For most of the last day, students will apply Bayesian analytical techniques to a dataset of their own choosing, with optional play datasets for those who do not have their own. The only requirements are (1) that students use a hierarchical model with thoughtful choice of priors (either uninformed or informed by results in their field), and (2) that students use MCMC through R/Jags to sample form their posterior distribution(s).
Students will work individually at first; about half-way through, students will form small groups based on the types of analysis they have chosen to perform and explain to others the nature of their design and help one another troubleshoot any issues that have arisen. This will take 30-minutes to an hour; the remaining time will allow students to finish their analyses, and we will finish with a class discussion of how to write-up and report the results of Bayesian analyses. Students will finally fill out course instructor surveys; before doing so, they will be made to undestand that I take this feedback very seriously, and their suggestions will be used to shape the future development of the course.
In terms of final products to take away from the course, the students will have (1) R and Jags installed on their personal computers, with basic knowledge of how to run them, (2) an R-script of commands for conducting all of the analyses from scratch again, including the many in-class worked examples of implementations on play datasets, (3) access to all of the videos and mini-lecture slides associated with the course, and (4) a finished (or at least mostly finished) set of analyses they have performed on their own dataset.
Planning, prepping, story-boarding...
Below is a picture of some of the story-boarding that went into producing the video lessons. It was actually really tricky to find a nice balance between scripting (so that you don't forget anything important) and spontaneity (so that you sound like a normal human being with all the conversational hums and haws in the right places). Speaking of conversation, it was a bit odd at first to be sitting alone in a room talking to my computer screen!
Producing the slides
Producing the slides was relatively easy for me because I was expanding on my live teaching practice lesson. Some of changes I made were in direct response to the feedback I received from instructors and peers, including (1) to make an effort to ensure that everyone is brought up to the same level of prior knowledge with respect to notation, jargon, etc, and (2) to do a better job of assessing students' understanding in real time.
To do this, I added a "basics and notation" section to the beginning of the lesson, where I thoroughly cover probability conventions (independence, 0<p<1) and notations (p(A|B), etc). I encluded 2 questions to assess students understanding of these concepts, which are delivered through Zaption.
One change I did not (yet) make in response to my peer's feedback is the inclusion of some explicit advance organizer directly within the lesson. I see why something like this is important, but at the same time for my own purposes (a 4 day short-course with an in-class), I will be able to convey this through other means.
Besides, I was really trying my hardest to keep the videos short! Research on the human attention span has shown that 5-10 minutes is really about the limit, even for the most focused students. But my video kept going over! I ended up paring them down with the YouTube editor (see next section) and breaking them into three separate videos (while at first I first thought I might be able to get away with one!). These are the first three of about 15 (~5 optional, ~10 required) that I will eventually make for my blended short-course. I am happiest with the first two, though the third one didn't turn out too bad either.
We used Screencast-O-Matic to overlay our slides with audio/video commentary.
I quickly discovered that making the videos like these requires a bit of voice-acting! Online, you have to exaggerate everything you say in order to engage as effectively as in real-life and to avoid being misunderstood; while this is clearly true of text, I didn't realize the extent to which it would be true with disembodied audio.
Here was my first take of the first part; this was done during class-time, so we limited ourselves to the first three slides: http://zapt.io/t4jqkvw4
(n.b., as of 9/2016 ZAPTION is no longer available)
Not a terrible first start, but there were clearly some kinks to be worked out. I had not figured out Zaption's feedback mechanism, for one; for two, the sound-quality was atrocious because of the mic I was using, which made my bumbling even worse!
Here are the final three video lessons for Day 1 (to be viewed at home, prior to class):
Lesson 3 (Solutions):
Editing and producing the videos
While I was editing the videos, I realized the importance of recording as much as possible in a single go; if you break up your recordings, you forget what you have already talked about in depth and you end up covering the same ground. I think this is why my first two videos were 5 minutes long, while my third was over 10; I recorded the third separately and a couple days after the others!
One caveat here, though, is that doing it all in one take means extensive editing may be necessary, and editing was for me by far the most time-consuming aspect of the whole thing. I had to cut out the mistakes, cut out the repetitions, the fluff, the wastes of time; no matter how hard you try, there will always be more to whittle down. To make matters worse, I thought I would use YouTube's built-in movie editor to save time, but it ended up being extremely clunky and hard to use. I kept having to refresh, and I found myself unable to click on certain parts of the timeline, etc. These little frustrations added a considerable amount of time to an already time-consuming task.
Unfortunately, Digication only lets you embed media from a small number of sites and does not support Zaption (which I guess is OK since Zaption just got bought up and will apparently not be supported after 9/30/2016). This is why you see a series of links below instead of media embedded directly in the page. If Digication wants to be competetive it needs to be useful to people; my advice would be to allow people to work directly in html; it thwarted me again down below when I tried to embed the Shiny app I'd been working on!
Zaption (effects added after videos have been produced)
I used Zaption for
- Creating end-caps for my lessons (static slides at the beginning and end of the video that discribe the lesson or summarize and segue into the next lesson, respectively).
- Asking both open-ended (drawing & numeric entry) and recognition (select all that apply) type questions during lecture. Not only does this serve to break up lecture and get students actively doing something with what they are learning, it also allows me to see where everyone is at before the in-class component the next day.
- Delivering correct-answer feedback: all of the questions have feedback to students (where appropriate); I decided that, if I go over the solution in the video, I would withhold an explanation from those who had given a wrong answer but encourage them to watch the video for the full answer worked out. If this wasn't appropriate (e.g., if there was no video explanation for a particular question), then I would give an explanation directly in the feedback message.
(see links to Zaption video lessons above)
First take of introductory video:
Instead of a slidedeck, I got the idea to just take people through my social media, show them pictures of my cats, etc. We had to make our first video during class up at school; we had 20 minutes. Since I didn't have a web-cam with me, I went to the computer lab on the 4th floor of Sanchez and just let it rip. Screencast-o-matic, the software we used to record our lessons, has some annoying features like a freemium watermark but it is also extremely convenient in that you can run it on any computer without administrator privileges. Anyway, the results of this dry run are here, for better or worse: https://www.youtube.com/watch?v=F0BasHUcCJ0
Final introductory video (see video below):
I waited until I got home so I could find a quiet moment to make the real-deal intro video (see video below); I kept my time low, which is good, but I didn't get everything right. For one, I forgot to make my video "future proof" (I mentioned graduating "a month ago", etc) so I will need to recreate them when I actually teach this course. But that's OK! By then different things will be going on that I want to show my students and different things I want them to know about me. Fortunately, I learned from this error and made a conscious effort to leave out any mention of time in my lessons.
This is a Shiny app I've been working on that illustrates graphically the dependencies among conditional probabilities and also how Bayes rule works. I could easily design an assignment using this, but I haven't gotten all the kinks worked out yet. I got it up and running for this project though, which has emboldened me to begin playing more with Shiny; their server lets you host your apps there for free; infortunately, Digication is extremely rigid with respect to what it allows you to embed in iFrames--otherwise I could have stuck the app right into this page.