Source Triangulation Skills and the Future of Digital Inclusion: How Information Literacy Policy Can Address Misinformation and Disinformation Challenges

This is the sixth installment of a new white paper series on alternative regulatory responses to misinformation. The articles, which were produced with the editing and support of the WIII Initiative, will be presented at a series of events on March 8, 10, and 12th. You can find more information, and registration details for our launch events, here.

I. Introduction

As we navigate treacherous information environments finding answers to questions about politics, healthcare, nutrition, and beyond, the ability to distinguish fact from fallacy and even fake is vital. To help people develop this ability, policy responses to misinformation and disinformation challenges should promote equitable, distributive supports^[1] to encourage source triangulation. The source triangulation concept^[2] refers to information validation through access and assessment of multiple sources with the aim of finding credible commonality.^[3] This is what Google nudges us to do when posting links to “more resources” adjacent to YouTube videos, or what Twitter invites us to consider by adding links to alternate perspectives (see Figures 1 and 2). As an individual learns source triangulation, they develop one component of an information literacy education. Information literacy is a broader, multi-layered concept which includes various forms of knowledge and skill for distinguishing helpful from unhelpful information.^[4] Source triangulation supports could complement platform-down content moderation efforts,^[5] concurrently enhancing user agency^[6] from the bottom-up.

Any new proposal connecting technology and education must acknowledge that U.S. policy tends to focus primarily on closing first-level digital divides (e.g. computer and internet access).^[7] While funding must continue to address access, this proposal asserts that it is time to also address what the United Nations calls “meaningful universal connectivity.”^[8] As the Secretary-General of the International Telecommunications Union explains, “digital inclusion can only be meaningful and effective if and when Internet users feel empowered to use the technology."^[9] While the context of these comments is second-level digital divides^[10]outside of the United States, some American digital inclusion initiatives advance similar goals. For example, the National Digital Inclusion Alliance (NDIA) connects digital inclusion^[11] and digital equity^[12] with digital literacy.^[13] Access and use of technologies are central to NDIA’s work, suggesting digital inclusion “requires intentional strategies and investments to reduce and eliminate historical, institutional and structural barriers.”^[14] One attempt at addressing these barriers is The Digital Equity Act of 2019.^[15] With direction from NDIA’s definitions for digital inclusion and digital equity,^[16] the Act proposes $1.25 billion in funding for two Federal programs facilitating grants through the National Telecommunications and Information Administration “to promote digital equity nationwide.”^[17] Indeed, coupling connectivity and use initiatives is essential as misinformation and disinformation challenges proliferate, and as ongoing research reveals that the state of individual ability to address these challenges is “bleak.”^[18]

This paper offers two policy recommendations:

1) U.S. federal government agencies addressing technology and education questions should develop an expansive vision of digital inclusion that includes the use of technologies, with a focus on information literacy initiatives in general, and source triangulation outcomes in particular; and

2) The work of the National Digital Inclusion Alliance should be complemented by funding efforts to realize information literacy skills via schools, libraries, and additional non/for-profit platform and infomediation initiatives to address misinformation and disinformation challenges.

These efforts should follow a distributive justice approach,^[19] with the aim of achieving an equitable distribution of resources that acknowledges the challenges unique to marginalized and vulnerable communities.^[20] In particular, we must address gender-based divides such as those UNESCO identifies,^[21] struggles unique to individuals whose internet experiences are mobile-only or mobile-mostly,^[22] and growing concerns about Big Data discrimination.^[23] It is essential that misinformation and disinformation challenges be understood as including, for example, biased algorithms, data sets, and resulting eligibility determinations that can contribute to Big Data discrimination. Recognizing the extent and the variations is important if we are to stop the damage. We must address the threat mosaic of deteriorating information environments, especially for those most likely to be victimized. The consequences of inaction may be residence in online and offline worlds that are both dangerous and treacherous, where we cannot navigate, understand, or govern.

The next section briefly describes findings about the state of online information literacy in the United States. To begin to answer the information literacy challenge, a conceptualization of source triangulation is presented, and a set of policy proposals for future information literacy efforts in the United States follows. The conclusion suggests that these proposals might serve as a model for efforts internationally.

II. The Lack of Online Information Literacy in the United States

The state of online information literacy in the United States appears problematic. Assessments of the extent of the problem include studies from Stanford University’s “Civic Online Reasoning”^[24] project (a concept synonymous with online information literacy). In a 2016 report,^[25] the group summarizes various assessments of thousands of middle-school, high school and university students at both poorly and well-resourced schools across a variety of U.S. states. A primary focus was the extent to which students could distinguish between helpful and unhelpful information online, check sources, as well as identify and compare evidence to test for credibility. Results suggest that “[o]verall, young people’s ability to reason about the information on the Internet can be summed up in one word: bleak.”^[26] A follow-up study in 2019^[27] of 3,446 students, “a national sample that matches the demographic profile of high school students in the United States,”^[28] similarly suggests that despite policy efforts put in place since the first study, the results of the second study are “troubling.” The authors summarize that “[n]early all students floundered […] Education moves slowly. Technology doesn’t. If we don’t act with urgency, our students’ ability to engage in civic life will be the casualty.”^[29]

PEW research suggests these concerns extend to older individuals as well. A recent report found that when classifying five factual statements that were “news-related” and five opinion statements, only 26 percent of more than 5,000 participants could correctly identify the factual statements, and 35 percent for the opinions.^[30] More concerning is what PEW calls a “democratic deficit,”^[31] suggesting the struggle “extends to a growing and troubling public distrust in each other’s ability to make informed decisions about democratic leadership.”^[32] Beyond the American context, UNESCO emphasizes that “[t]oday billions of people have access to affordable devices and broadband networks, but do not have the requisite skills to take advantage of this technology to improve their lives.”^[33]

The online information literacy challenge is not only about the ability to read and understand content. Analyzing, critiquing and comparing content are initial steps, but more is necessary. For robust information literacy, we must also understand information environments where filter bubbles,^[34] echo chambers,^[35] bots,^[36] trolls,^[37] and our own confirmation bias^[38] introduce challenges. Beyond this is also the need to understand the policy, political economy, and infrastructure that contributes to the evolution of the internet.^[39] Thus, while the policy recommendations herein suggest the need for source triangulation skills, to achieve a thorough online information literacy, multiple layers to the necessary education are vital if we are to achieve digital inclusion and “meaningful universal connectivity.”^[40]

III. Source Triangulation as a Learning Outcome

There seems to be comfort in the efficiency of accessing a single source of information we feel we can trust.^[41] One problem with blindly trusting a single source however, is that its information may be questionable and require checking. If a problematic source provides material that is comforting, or reinforces prior beliefs, it can be challenging to break from it.^[42]

Instead of learning outcomes that result from reliance on a single source, the ability to find, evaluate, and draw accurate information by “comparison and corroboration”^[43] of multiple sources should be a primary focus of educational investment.^[44] Source triangulation is one aspect of information literacy development which involves learning to validate sources by conducting research and comparing multiple sources to find a commonality that is credible.^[45] The term “credible” is important, as credibility is context-specific. What is credible in a corporate context differs from what is credible in a critical studies context, which connects us to one of the oldest philosophical and political economic debates about manufactured consent,^[46] mediums as messages,^[47] “pictures in our heads,”^[48] and shadows on cave walls.^[49] A necessary caveat must clarify that truth will not necessarily emerge via commonality alone. Many important truths are with sources, some with access to the marketplace of ideas, some without, that are not understood or known broadly. Warnings about echo chambers are also important, which is why source triangulation skill is just one aspect of broader information literacy training necessary for navigating treacherous online information environments.

Despite the challenges with credible commonality, source triangulation is consistently viewed by information literacy experts and advocates as a primary strategy for helping users find the information necessary for democracy to function.^[50] The triangulation concept draws from the practice of surveying land.^[51] As Patton states:

Knowing a single landmark only locates you somewhere along a line in a direction from the landmark, whereas with two landmarks you can take bearings in two directions and locate yourself at their intersection. The notion of triangulating also works metaphorically to call to mind the world's strongest geometric shape-the triangle.^[52] (emphasis added)

Central to determining information-strength via triangulation is the challenge that single sources are often easily refutable by alternative explanations.^[53] Critical analyses of multiple sources, and checking for consistencies across them, supports individuals as they attempt to determine credible commonality.^[54] It is also important to identify facts and perspectives that may be missing from the consideration of a single source. The intention is both to help people identify “information problems”^[55] while also learning to “take charge of the information allowed into their minds and decision-making schemas.”^[56]

One place to begin when teaching source triangulation is with Stanford University’s “Civic Online Reasoning” project.^[57] In addition to the curriculum available on the project platform,^[58] published studies identify skills similar to source triangulation. For example, upon arriving at a piece of information online, the research recommends first to “take bearings”^[59] (note the language consistent with Patton above^[60]) or the process of taking time to determine a plan for evaluating the information available. Rushing to trust a single source is discouraged. A similar skill is “click restraint,”^[61] relevant to evaluating search results or social media feeds, where individuals should begin to assess information before clicking, considering, for example, differences associated with the curation of results.^[62] These two skills emphasize that information literacy efforts should help individuals learn to evaluate information sources as well as the platforms that curate access to those sources. These initial steps prepare the individual to then attempt a third, which is similar to source triangulation - “lateral reading.”^[63] As the authors note: “When reading laterally, one leaves a website and opens new tabs along the browser’s horizontal axis, drawing on the resources of the Internet to learn more about a site and its claims.”^[64] Remaining on the site and only evaluating its content to determine credibility would be “reading vertically,”^[65] and introduces the risks associated with trusting single sources. In a study of how fact-checkers engaged in lateral reading, “(f)act checkers, in short, learned most about a site by leaving it.”^[66]

This returns the discussion to the question of how to determine information credibility, especially when attempting to triangulate sources during lateral reading. Indeed, both the pedagogy and the learning outcomes for developing information literacy in the twenty-first century are areas in need of further research. At the same time, there is an abundance of resources (and related terms) available online about addressing misinformation and disinformation, and at different stages of development. Should we teach digital literacy? Media literacy? Metaliteracy? Critical information literacy? This requires educators to have information literacy skills just to navigate the resources for teaching information literacy. The need for further research and resource organization justifies the call for government leadership central to the policy recommendations that follow.

IV. Policy Recommendations

Information literacy and source triangulation skills are not primary goals of U.S. government funding mechanisms connecting technology and education. Funding efforts in this space often focus on closing first-level digital divides—differences in access to higher-quality hardware, software, and internet connections.^[67] While access to technology is essential, particularly technology that gets users beyond mobile-only or mobile-mostly experiences,^[68] considerable investment in a more “meaningful universal connectivity,”^[69] with a specific focus on internet use is vital.

The Obama administration’s digitalliteracy.gov project, now offline,^[70] suggests government-funded online information literacy programs were at one time possible. The initiative was a collaboration between the National Telecommunications and Information Administration (Department of Commerce), the Department of Education, the Institute of Museum and Library Services, the Federal Communications Commission, and the Department of Housing and Urban Development, among others.^[71] While the focus of the Obama administration’s efforts appear to have emphasized skills for the digital economy, such as professional development training, the few materials still available suggest programs prioritizing internet use were possible.

Policy responses should now revive and expand these efforts, funding research and implementation of a variety of supports to train and engage people in source triangulation. In particular, this paper recommends: 1) that U.S. federal government agencies already connecting technology and education should develop and promote a vision of digital inclusion that prioritizes internet use through information literacy initiatives such as source triangulation training, and 2) these agencies should allocate funding to schools, libraries, and additional for/non-profit efforts to deliver source triangulation and other information literacy learning outcomes. This should serve both to support as well as to complement the efforts of groups including the National Digital Inclusion Alliance.

The next section addresses how U.S. federal agencies connecting technology with educational opportunities might pursue these recommendations. A brief discussion follows of additional for/non-profit approaches involving platforms and infomediaries.

IV.i Vision and Funding from Federal Agencies for Information Literacy Supports

Libraries and centers for information and library science scholarship already lead efforts promoting online information literacy to address misinformation and disinformation challenges. To enhance and support these efforts in the United States, the federal agency known as the Institute of Museum and Library Services (IMLS) should prioritize source triangulation skills as a part of its vision for libraries across the country and fund efforts to develop and promote that vision. While the agency lists information literacy as a “twenty-first century skill,” it is not one of the agency’s “priority areas.”^[72] While the Library Services and Technology Act,^[73] which governs the agency, appears to prioritize access, professional development and STEM development are IMLS priority areas, suggesting that information literacy initiatives would be consistent with current practice. Beyond a clear vision prioritizing source triangulation, the funding programs the IMLS oversees, such as the Grants to States program which distributes hundreds of millions of dollars to libraries across the U.S., should support these learning outcomes. Such a shift might encourage all State Library Administrative Agencies to update priorities to include online information literacy initiatives.

Another federal agency that should prioritize and fund source triangulation efforts is the U.S. Department of Education’s Office of Educational Technology. This agency “develops national educational technology policy and establishes the vision for how technology can be used to transform teaching and learning and how to make everywhere, all-the-time learning possible for early learners through K-12, higher education, and adult education.”^[74] (emphasis added) Source triangulation should be central to that vision. For example, while the 2017 National Education Technology Plan^[75] makes one mention of information literacy in passing, source triangulation should be central to plan updates.

This agency also oversees massive funding efforts that focus primarily on closing first-level digital divides. For example, a recent initiative associated with the Coronavirus Aid, Relief, and Economic Security (CARES) Act, involves an allocation of $30.75 billion to support the emergency shift to online and blended forms of learning during the pandemic.^[76] While funds should not be diverted from emergency efforts, it is important to emphasize that the Office of Educational Technology, and its various funding mechanisms, could be a place for more robust information literacy programs to emerge.

It is important to clarify that while state and local entities should have the freedom to independently pursue initiatives, ensuring flexibility within federal granting contexts, federal leadership remains vital. Instead of focusing on the specific details of programs or curricula, federal government agencies should prioritize the development of a clear and normative vision, as articulated in 20 U.S.C. 3425 - Office of Educational Technology:

The Director of the Office of Educational Technology […] shall— (1) in support of the overall national technology policy and in consultation with other Federal departments or agencies which the Director determines appropriate, provide leadership to the Nation in the use of technology to promote achievement of the National Education Goals and to increase opportunities for all students to achieve State content and challenging State student performance standards.^[77] (emphasis added)

The Office of Education Technology’s website explains that the details of the National Education Technology Plan “align to the Activities to Support the Effective Use of Technology (Title IV A) of Every Student Succeeds Act as authorized by Congress in December 2015.”^[78] This section of the Act, emphasizing “21^st Century Schools,” states with regard to the allocation of funds:

ACTIVITIES TO SUPPORT THE EFFECTIVE USE OF TECHNOLOGY (a) Uses of Funds. --Subject to section 4106(f), each local educational agency, or consortium of such agencies, that receives an allocation under section 4015(a) shall use a portion of such funds to improve the use of technology to improve the academic achievement, academic growth, and digital literacy of all students.^[79]

Returning to the list of agencies supportive of President Obama’s digital literacy efforts,^[80] there are a variety of additional agencies that might provide leadership and funding. One is the Federal Communications Commission (FCC). Via the Universal Service Fund (USF), and specifically, the E-Rate program, which the FCC oversees, eligible libraries and schools receive funds to create and enhance internet connections. While historically the view was that USF-programs should make closing first-level digital divides a priority,^[81] there is evidence that FCC efforts to support information literacy might be also possible. In updates to its “Lifeline Program for Low-Income Consumers,” the FCC prioritized “digital inclusion” making it a “national priority.”^[82] The updates make specific reference to the efforts of the NDIA, and clarify that with the changes the FCC is starting “an ongoing campaign to build the Commission’s digital literacy capacity.”^[83] The update tasked the Consumer and Governmental Affairs Bureau to develop a plan for achieving various digital inclusion outcomes. The resulting plan,^[84] released in 2017, mentions digital literacy repeatedly, making clear that teaching individuals to use the internet must be central to efforts that aim to promote digital inclusion. This suggests that the FCC could make information literacy and source triangulation part of its priorities and funding efforts. It is worth considering how historical interpretations of the public interest, as a measure for determining access to the radio spectrum, for example, might be updated to establish new requirements for organizations operating within this model.^[85]

To achieve source triangulation learning outcomes across the country there will need to be investments in the development of pedagogical approaches and accompanying software. Public-private partnerships may be one approach to pursue these efforts. As David L. Cohen, Senior Executive Vice President at Comcast, comments about the FCC’s Lifeline updates:

[W]e will not move the needle on broadband adoption in this country unless there is a robust commitment to digital literacy and relevancy training. […] Closing the digital divide is a great challenge, but I’m convinced that, by working together, we can get more Americans connected to the amazing and transformative power of the Internet.^[86]

It appears the focus on connectivity in federal funding mechanisms is slowly shifting towards programs for teaching people how to use the internet. Federal agencies should help lead this important work and ensure the vision and funding to address misinformation and disinformation challenges.

IV.ii Information Literacy Supports Via Digital Platforms and Infomediaries

Another approach to supporting information literacy in general and source triangulation in particular is through supplementary platform efforts. As noted earlier, Google and Twitter are providing links to supplementary information next to YouTube videos and tweets.^[87] By nudging individuals to “read beyond,” and “consider the source,”^[88] platforms are beginning to convey that users should learn not to trust single sources and instead consider multiple sources as a default information engagement strategy.

These efforts could expand to include more dynamic engagement and educational opportunities, similar to how experimental forms of dynamic online consent processes are being considered as part of initiatives to help users realize digital privacy protections.^[89] Public-private partnerships will likely be necessary, as software, user-interface design, and embedded pedagogy are the subject of further research.

New strategies for achieving information literacy outcomes, as well as generating demand for skill development, should be considered. For example, with funding from the U.S. government, imagine if Jeff Bezos and Jack Dorsey organize a coalition of media companies to develop software and pedagogy for engaging users in source triangulation education. With Bezos leading the initiative, a subscription to The Washington Post could be a benefit of user participation. The program could develop a new online platform that would become central to K-12 information literacy curricula, if states chose to participate. One potential concern with this approach is the possibility of unfairly privileging dominant media players, such as The Washington Post.^[90] Ensuring participation in the coalition for smaller media companies, as well as those representing a diversity of ownership and content might help mitigate this.^[91] This potential revitalization of the fourth estate,^[92] might combine providing individuals with the information they require to be self-governing in a democracy (central to historical views of journalism^[93]), with supports for developing information literacy.

Another possibility is the development and funding of infomediation services.^[94] These entities might emulate similar services emerging in the online reputation management industry, helping individuals address the challenges of understanding and even controlling online information. As the management of misinformation and disinformation grows as a challenge relevant to individual as well as organizational reputations, various research, market, and non-market responses should be the focus of government funding. At this early stage, one goal might be to support infomediation services working to prevent organizations from making biased decisions based on flawed data. For example, consultants with expertise in source triangulation could assist human resource professionals as they attempt to navigate biased data and algorithms when evaluating job candidates.^[95] These efforts might evolve into an array of private and non-profit infomediation services that could also support individuals as they encounter misinformation and disinformation challenges.^[96] What is clear is that policy efforts must follow a distributive justice approach, acknowledging the unique challenges faced by those most likely to be harmed by the treacherous information environments online.

V. Conclusion

While this paper focuses on developing policy recommendations for implementation in a U.S. framework, the relevance of these proposals goes beyond any single national context. In January 2017, the International Federation of Library Associations and Institutions (IFLA) posted an infographic entitled “How to Spot Fake News”^[97] (see Figure 3) to the global resource, Wikimedia Commons.^[98] The first four recommendations all connect to the source triangulation concept, suggesting individuals should read critically and review multiple sources before finding answers to questions. Even “check your biases”^[99] asks individuals to consider whether less-rigorous information evaluation strategies, perhaps linked to the trust of individual sources or confirmation bias,^[100] should be reconsidered.

Self-described as “the global voice of the library and information profession”^[101] the IFLA claims to have at least 1,500 members in more than 150 countries.^[102] A credible commonality check reveals that the IFLA’s Facebook page is followed by more than 50,000 individuals.^[103] The American Library Association notes the IFLA “wants to show the power of libraries in achieving the UN Sustainable Development Goals (SDGs) and has asked library associations around the world to join in a global effort.”^[104] These two additional pieces of information suggest the IFLA and its statements may be credible.

This brief exercise suggests the following. First, while the IFLA resource is a place to start, more research is necessary to determine what information literacy skills are important and how they might be taught. The “Civic Online Reasoning” curriculum^[105] provides some helpful resources, but more are needed. Second, the presence of the IFLA and its commitment to information literacy education suggests that the policy proposals herein may extend internationally. As the United Nations works to promote “meaningful universal connectivity”^[106] it is vital that international digital inclusion initiatives prioritize the vision and the funding to support individuals as they attempt to distinguish between fact, fallacy, and fake.

As technology changes, imagining new governance models can be difficult, especially when current (and ineffective) approaches have been in place for some time.^[107] Answers to these questions require further research into how source triangulation may contribute to democratic discourses internationally. One thing is clear: we are unprepared to meet the “information problems”^[108] present today and threatened by challenges soon to be realized in a quickly emerging tomorrow.^[109]