Three reactions to the academic job talk debate sparked by Dan Nexon. (Nexon argued job talks are not useful for hiring.)
1. I agree with Dan Drezner that the Q&A can be very illuminating. For instance, how well does the candidate know the topic and previous scholarship? What questions have they anticipated? Are they open to alternate views or is it my-way-or-the-highway? I should also note it often lets me see how colleagues, some of whom may know a ton about the topic, are reacting. I may not get that by reading on my own.
2. I am not aware of scholarship that has assessed the correlation between the components of the job hiring process and the most common measures of faculty success, A) publications and B) tenure. That is crucial data that is probably next to impossible to collect for a host of reasons (e.g. privacy, liability, collegiality, diversion from more interesting questions, human dignity). But if someone who studies such matters can shed light, I for one would be interested.
3. Given my first point, here’s a (wacky) suggestion, because I do agree with Nexon that there is a reticence to read anything especially if one is not on the search committee. What if the candidate distributed a paper in advance, spoke for only 10-15 minutes, and then took extended questions and comments for 45-60 minutes? That would give time to go deep, for each questioner to have some back and forth if desired.
Perhaps even crazier: I once saw an APSA panel (I am thinking it involved David Waldner and Allison Stanger among others) where the discussants went first and then the paper authors responded. What if a member of the department critiqued the paper for 15 minutes and then the author/job candidate responded? OK, I know, a little crazy and the internal departmental politics of the hiring process could muck it up. But I am enamored with the importance of questions and answers.
That said, if a department might have 10-15 job candidates in a season, asking people to read that many papers is not insignificant. The status quo may have staying power for a reason.