This is the archived version
of the post titled "How the Liberal
Media Myth is Created - Part 4"
which appeared at The
Left Coaster prior to 4/16/05. An
updated version has been posted in its place at the
This is the continuation of a series on how the "liberal
media" myth is created. Previous installments covered how this
myth is created using "tone" of media coverage (Part
1), using "catch-phrases" like 'right-wing
extremist' v. 'left-wing extremist' (Part
2), and using "newspaper headlines" (Part
3). In this part, I address a fourth (superficial) approach
used for creating a myth of "liberal media" - "topics"
covered. [A few more parts can be expected over the rest of this
The focus of this part is the 2004 paper, "Being
the New York Times: The Political Behaviour of a Newspaper"
by Riccardo Puglisi of the London School of Economics (LSE) (which I
discovered via Marginal
Revolution). I have provided a more systematic critique of this
paper at Illiberal
Conservative Media (ICM) - Sec.
2.10; here I will highlight some portions of that critique.
[*See footnote for an update]
The issue of topic choice is important in media bias analysis, but
like everything else it has to be treated with some sophistication to
eliminate false results/conclusions. As I have indicated before at ICM:
Topic choice is certainly a function of editorial bias, but it
also a function of numerous other confounding factors -
source credibility, events, circumstances, issues of public
interest, issues of interest to politicians or policy-makers, issues
of interest to the media outlet to ensure their revenues and profits
in the markets they compete in, etc. So, it would be much more
difficult to credibly demonstrate editorial bias on topic
choice, by itself.
With that sentiment, let's look at Puglisi's paper, starting with
his abstract (bold text is my emphasis):
I analyze a dataset of news from the New York Times,
from 1946 to 1994. Controlling for the incumbent President’s
activity across issues, I find that during the presidential campaign
the New York Times gives more emphasis to topics that are owned
by the Democratic party (civil rights, health care, labour and
social welfare), when the incumbent President is a Republican. This
is consistent with the hypothesis that the New York Times has a
Democratic partisanship, with some "watchdog" aspects,
in that it gives more emphasis to issues over which the (Republican)
incumbent is weak. Moreover, out of the presidential campaign, there
are more stories about Democratic topics when the incumbent
President is a Democrat.
Now, you can call me silly, but when someone derives such a
sweeping conclusion based purely on a study of "topics"
covered, my GIGO* detector goes on Orange alert. As I read more of the
paper, I sadly realized the alert level had to be upgraded to Red. In
my detailed critique of this paper, I've pointed out what I believe
are six major problems with this paper (I,
I can't do full justice to all of those points here - so I am simply
going to condense my points here and refer interested readers to the full
critique for details.
First, here are some assumptions stated by Puglisi for his
As briefly anticipated in the introduction, the
empirical analysis performed here and the interpretation of its
findings are based on the following set of identifying assumptions:
(1) The issue ownership hypothesis holds.
(2) “All publicity is good publicity”.
(3) The relative share of Executive Orders about a
subset of issues proxies the relative intensity of the activity of
the incumbent President with respect to those issues.
The issue ownership hypothesis, which Puglisi bases on
historical polling data and mentions throughout, is the following:
Democratic topics comprise Civil Rights, Health
Care, Labor & Employment and Social Welfare. Republican topics
comprise Defense and Law & Crime.
Now, it may be convenient to assign such ownership
because it helps make the analysis more interesting, but really,
someone "owning" the issue often has little to do
with whether the publicity/coverage that person gets on that issue is
good or bad. Thus, the second assumption, that "All publicity is
good publicity" (referring to "owned issue" coverage
for the person who owns it) simply makes no sense. For example, was
"Health Care" coverage always "good publicity" for
Bill Clinton (Democrat)? Was "Defense" and "Law and
Crime" coverage always "good publicity" for Richard
Nixon (Republican) and Ronald Reagan (Republican)? Was
"Employment" and "Social Security" coverage
necessarily always "bad" publicity for the Reagan
administration? In other words, the assumption that if a
newspaper reports on topics "owned" by a party, it
automatically means that party benefits, makes no sense because such
an assumption fails to account for the fact that newspapers, can and
do issue reports on "owned" topics that may not be positive
at all to the "owning" party. (If this seems like an odd
assumption, wait till you see the second one, which effectively
scuttles the papers' conclusions.)
Second, consider these "definitions" offered from
Definition 1 A newspaper has a Democratic
(Republican) partisanship if during the presidential campaign it
devotes more space to issues owned by the Democratic (Republican)
party, at the expense of neutral or Republican (Democratic) issues.
In fact, over and above the electoral partisanship of the
newspaper, as described by definition 1, the political colour of the
incumbent President could be given an interpretation within a
lapdog/watchdog dichotomy. The idea is the following: if it turns
out that -during the presidential campaign- the New York Times gives
less emphasis to Democratic topics and/or more emphasis to
Republican topics when the incumbent is a Democrat, over and above
his Democratic or Republican or lack of partisanship, this is
consistent with the fact that the newsaper [sic] acts as an
electoral watchdog with respect to the incumbent President.
Definition 2 The newspaper is an electoral
lapdog of the incumbent President if ceteris paribus during the
presidential campaign it devotes more space to the issues over which
the incumbent is strong, and/or less to issues over which the
incumbent is weak.
Definition 3 The newspaper acts as an
electoral watchdog if ceteris paribus during the presidential
campaign it dedicates more space to the issues over which the
incumbent is weak, and/or less space to the issues over which the
incumbent President is strong.
Where do I begin?
I don't mean this insultingly at all, but these
definitions are ridiculously wrong because not only are they
inconsistent with each other, the latter definitions are wrong on
their face. For example, Puglisi's definition of "watchdog"
in Definition 3 is highly convenient (and reveals possible built-in
bias in his thinking about this problem) because I can just as
well argue based on his Definition 1 that the newspaper is no
"watchdog" but just a shill for the candidate opposing the
incumbent and is therefore displaying "partisanship" in
favor of the challenger. In fact, let's ignore Definition 1
completely and consider Definition 3 on its own. It is
Puglisi's *opinion* that the newspaper serves as a
"watchdog" by focusing on the topics that supposedly favor
the challenger. One can easily have a different *opinion* that a
newspaper doing this is a partisan supporter of the challenger and not
a "watchdog"! (Talk about pre-ordaining the results!)
This is the natural (and fully expected) problem with
studies of this nature which don't bother to actually analyze the content
of the news articles. Thus, Puglisi's assumptions and definitions are
wrong because at a very fundamental level, they neglect the actual
nature of the coverage (accurate or inaccurate). So,
combining Problem I and Problem II, this study and the interpretation
of its results totally break down even before we get to the actual
Third, by Puglisi's own admission (Tables 2 and 3), when we
look at "All stories" that appeared in the New York Times in
the period 1946-1994, the so-called Republican topics and so-called
Democratic topics were only 21.7% (8.37% + 13.36%) of the total. Thus,
this study claims to show "Democratic partisanship" (or
otherwise) based on a study that essentially ignores over 78% of all
stories published in the New York Times. Stunning.
For example, "Banking, Finance and Dom.
Commerce" (14.66% of all stories) and "International
Affairs" (13.22% of all stories) are not part of Puglisi's model
because they are not "owned" by Republicans or Democrats.
What category would "taxes" or "spending" or
"budget deficits" fall under? This is one of the most
important topics in all Presidential campaigns - which often make or
break campaigns - and there's no mention of it in the analysis. Also,
what category would draft-avoidance or alleged extra-marital affairs
fall in? Other? Or is it "Law and Crime?" There's a whole
slew of topics relating to the individuals or their policies, that
fall into the supposed "non-owned" issue category, which
have a habit of coming up frequently during campaigns. It is
extraordinarily odd that one would simply ignore all that and still
feel comfortable reaching sweeping conclusions of the kind the author
Fourth, Puglisi's paper does not consider seriously the fact
that major events happen which have nothing to do with the
"strength" of Democrats or Republicans. For example, George
Bush Sr. started
significant cuts to defense spending at the end of the Cold War
and Bill Clinton continued this effort. When there are no major wars
and when there is no overarching concern about national
defense, there is no reason for papers to simply keep writing more
articles about "defense" just because a Democrat is in
power. This same argument applies to every topic under the sun.
It is also obvious that many topics are raised,
especially in electoral campaigns, by the politicians who are
campaigning. Not to mention, one of Puglisi's "findings"
is that the coverage of "Republican topics" actually goes up
significantly in the campaign coverage when the challenger is a
Republican. This takes us right back to Problem II. Either the NYT has
"Democratic partisanship" or it doesn't. It makes no sense
to claim that it has "Democratic partisanship" and
simultaneously say that "...under a Democratic incumbent there
are more stories about Republican topics when the presidential
campaign kicks in. This effect is quite strong in magnitude...". Why
is the latter considered a "watchdog" behavior rather than
"Republican partisanship"? After all, if part of the
"results" point one way, it is sufficient for Puglisi to
label it "partisanship" of one kind; yet, when another part
of the "results" points in another direction, it is not
partisanship in the other direction - it is "watchdog"ism!
If that isn't
bias convenient, I don't know what is.
Fifth, when I look at Puglisi's basic data tables 3 and 4
(in his paper - see footnote), even if one makes the assumption that
Executive Orders get proportional coverage in the NY Times (as he
does), the numbers
I derived suggests that even when the New York Times'
topics-coverage is normalized to Executive Orders, it provided more
coverage overall on the "Republican" topics than on the
"Democratic" topics (I invite readers who are more
statistics-aware to comment on whether I made any mistakes in my
assumptions/calculations because I am not a statistics expert). This
seems to partly contradict his main conclusions (even if you ignore
the fundamental flaws I discussed above).
Sixth, Puglisi's study lacks any *real* control for
comparison. Even if we assume that the results of this study are
correct (which they are not), how can someone claim that a paper is
partisan without even evaluating another paper - with an ideology
known to be conservative - to see whether that paper's topic coverage
was similar or the opposite?
All in all, this is a deeply flawed paper that certainly does
NOT prove ANY liberal bias or Democratic "partisanship" on
the part of the New York Times. But it helps us learn yet another way
media bias myths are propagated.
P.S.: GIGO = Garbage In Garbage Out
*FOOTNOTE ADDED 4/4/05: It has been brought to my attention that
the version of the paper I
had originally linked to and analyzed is not the final version of
Puglisi's paper. The latest version is available for download here.
I apologize for this inadvertent /unintentional error. Given this, I
have made appropriate (minor) modifications in my detailed analysis at
ICM, and in the
post above, to reflect the content and pagination in the final version
of the paper. Having said that, Puglisi's conclusions or my
critiques of his assumptions, data or conclusions have not changed
with the latest version of his paper. Thus, the substance
of my critique remains unchanged.