top of page

3rd Wave (2025-2029)

BACK MENU

Content Analysis

 

We will conduct a content analysis of the news published in the most important newspapers, websites, radio, and TV news programs in the participating countries. 

 

Sampling

​

Researchers in each of the 59 participating countries will select two to three news media outlets per type of platform. The criteria used to select the media outlets were audience size, reach, and level of influence in agenda setting. Preference was given to outlets that were national in scope, though regional and local outlets were included in the sample in countries in which they were considered important to the media landscape.

 

Given that the structure and format of media systems differ in many ways across countries including size, audience orientation, ownership, political leaning, and the presence of more than one language in a territory, researchers were asked to ensure as much as possible that the selected outlets represented the diversity of each country’s media system. Team members had to consider the fact that the number of media outlets included may vary from country to country and that greater heterogeneity in the media system would lead researchers to include more outlets, and vice versa. 

​

Thus, for example, certain criteria were used in the selection of outlets: (1) While popular-oriented news outlets do not exist in some countries, they are highly prevalent in others and need to be included in the sample. (2) In some countries, all media outlets are private/commercial, while others have both private and public (mostly broadcast) media. Some of the countries included in the study also have state-run media outlets. The sample should include all types. (3) Researchers attempt to include media outlets that reflect all dominant languages in multilingual countries in which language is an important feature of the media system. (4) While preference was given to outlets that were national in scope (in contrast with “local” or “regional” scope), in countries where regional and local outlets are as important as national media outlets, the sample should include both types.

​

To control for the potential overrepresentation and/or underrepresentation of certain types of media, due to some media including more stories in the sample than others, we weigh the data by medium for each country. This guarantees that each media type – TV, radio, online news, and newspapers – within each country would have an equivalent weight in the results. 

 

Using the constructed week method, a two-week stratified-systematic sample is selected for all outlets from January 2 to December 31, 2026. The same days are analyzed in all the countries. Because daily and monthly variations are important factors in studying news content, we divide the year into two six-month periods: January-June and July-December. For each six-month period, we create a constructed week, randomly selecting starting dates on a Monday in January and a Monday in July. Then, using three to four-week skip intervals, we select each of the subsequent six days: a Tuesday, a Wednesday, a Thursday, a Friday, a Saturday, and a Sunday. This procedure allows us to include seven days in each six-month period for a total sample of 14 days during the year, ensuring that one issue/edition/program from each of the seven days of the week is selected for each half-year and that all months are represented by at least one day.

​

Some news outlets do not report the news on weekends (Sundays, Saturdays, Fridays) or present the news using formats and/or time slots that differ from those used on weekdays. Thus, in some countries, there are no newspapers published or no news programs on television and/or radio on some weekend days. Those cases are considered “missing data.” However, if the time of a television and/or radio news is changed (e.g., due to special events), the newscast is coded using the actual time slot.

 

Each national team determines the specific sampling unit for the selected outlets, based on the following criteria: for television - the most watched newscast within each selected channel; for radio - the news program with the greatest audience in the selected channels; for newspapers - the full issue; and for online news - the entire homepage of the selected websites (including links contained therein). 

 

Whereas television and radio news programs and newspapers are “static” in the sense that each edition is unique and appears at fixed times, online news is dynamic and changes constantly. We therefore “capture” the homepages of the websites at two fixed points during the sampled days: 11:00 a.m. and 11:00 p.m. (local times) excluding the stories already considered in the morning cut (except if the story was updated or contains more developments). This 12-hour gap between the two was assumed to provide the maximum content variability for each day. The homepages and their respective links are opened and saved in real time.

 

The unit of analysis is the news item, defined as a set of contiguous verbal and, if applicable, audio and/or visual elements that refer to the same event/issue/person. For each of the selected issues/editions/news programs of newspapers, television, radio, and online news websites, all current affairs news are coded. Specifically, the following topics are coded: government/legislatures, campaigns/elections/politics, economy and business, police and crime, court, defense/military/national security, war and conflict, education, health, environment, energy, transportation, housing, accidents and natural disasters, religion and churches, labor and employment, demonstration and protests, social issues, and media and communication. 

 

An event, issue, or statement may be reported by a media outlet in more than one news item on a given day. When the same event, statement, or issue is covered in more than one item, these are considered to be unique stories and are coded separately. 

​

Not included in the study are editorials, opinion columns, weather forecasts, horoscopes, movie (or other cultural) reviews, puzzles, social pages, and similar content on radio and TV. We also excluded supplements/magazines/special features programs and headlines on newspaper front pages and at the beginning of TV and radio newscasts. 

 

In the case of online media, we only code news items that appeared on the extended home page by clicking on, and thereby opening, each of the relevant items. Items that included embedded video or audio clips were also coded.

 

Measurements

​

​The coding is based on the operationalization proposed by Mellado (2015) and validated in subsequent studies (Mellado et al., 2017; Mellado and van Dalen, 2017; Mellado et al., 2020) to measure professional roles in news content. 

​

The codebook includes operational definitions of the performance of the watchdog, interventionist, loyal-facilitator, service, infotainment, and civic roles based on the relationship between journalism and the de facto power, the presence of the journalistic voice in the story, and the way the journalists address the audience. Each of these roles/dimensions is characterized by different measures of professional practices, reporting styles, and narrative schemes. 

​

The codebook to be used in this wave of the project was refined by the participants in a collaborative manner to ensure intercultural validity. The codebook will be applied by all national teams in its original language (English).

 

Five indicators measure the presence of the “interventionist” role, seven indicators measure the “watchdog,” eight indicators measure the “loyal-facilitator” role, five indicators measure the “service” role, five indicators measure the “infotainment” role, and nine indicators measure the “civic” role. 

JRP 3RD WAVE.png

Each indicator is measured on a presence or absence basis. In the case of some indicators, we also code for the actor or action toward which the journalist’s or source’s comment was directed. We transform those indicators into dichotomous variables to calculate the main roles and place all measures on the same scale. 

​

Based on previous research, we consider role dimensions as reflective measurement models, where (a) the dimensions exist regardless of the measures used, (b) the variation in item measures does not cause a variation in the construct, and (c) adding/dropping an item does not change the conceptual domain of the construct (Wirth & Kolb, 2012). Additionally, based on the theoretical rationale of role performance literature (Lynch, 2007; Mellado, Hellmueller & Donsbach 2017), we assume that journalistic roles coexist. Therefore, the measures are treated as non-mutually exclusive. 

​

Prior to conducting our main analyses, we complete confirmatory factor analyses to test whether news stories reflect a latent role manifested through concurrent concrete indicators. Within that framework, we empirically test competing measurement models.

 

We also test for measurement invariance for categorical outcomes, conducting multi-group analyses of measurement invariance.

 

Based on the CFA results, the individual indicators comprising each dimension are combined to generate a final role score for each item. For descriptive purposes, we calculate raw scores, based on the total points divided by the number of indicators in each role. A higher score expresses a higher presence of each journalistic role in the news, and vice versa. Meanwhile, we use factor scores to test for differences in the performance of the roles analyzed. 

​

The content analysis also includes measures related to (1) general information on each news item, such as the type of medium in which it was published, the news outlet, the date of publication, story type, and story placement; (2) the characteristics of the story, such as the topic of the news item and the location where the news story takes place; and (3) the sources cited, including number of sources, source type, the diversity of type of sources, and the diversity of points of view. For the 3rd wave, we added extra variables that will help to problematize the presence of the roles in the news, such as authorship –including those created or assisted by IA—, the explicit acknowledgment of fact-checking processes in the news, and a clearer emphasis on whether the news focus addresses problems or solutions.

 

Data collection

​

The sample search process, as well as the news item coding are conducted by native speakers in each country. At least one national principal investigator is responsible for conducting all phases of the study, including liaising with the general project coordinator. National teams are trained by the project coordinator, particularly in order to obtain a common understanding of the operational definitions for all variables of the codebook. Training sessions are conducted in person or online. 

 

Coders in each country code the news stories directly into a specially designed online interface. The corpus of news items in each country is randomly divided among coders to reduce bias and avoid a situation in which a coder would code an entire outlet.

 

Coding Reliability

 

Since several concepts are inevitably culturally bound, we follow a three-step strategy to test for intercoder reliability between and within countries. First, we conduct a pre-test among principal investigators across countries to ensure that they have a shared understanding of the codebook. Second, national teams run pilot tests based on news items not included in the actual sample. As many training sessions are conducted as necessary until coders attain acceptable intercoder reliability coefficients. Coders are also closely monitored during the coding process. During the coding period, each team’s progress is monitored on a monthly basis.

​

Once the coding is completed, a post-test is conducted in each country in order to calculate a final global overall intercoder reliability.

bottom of page