Skip to Main Content

FLEET LIBRARY | Research Guides

Rhode Island School of Design

Generative Artificial Intelligence

An overview for students and faculty

Evaluating Generative AI

Information literacy is a key skill that students, professors, and researchers rely on in their studies. Fleet Library even made a guide on it for your reference: https://risd.libguides.com/s101.

Just as you should evaluate sources of information such as journal articles, books, websites, etc., it is also important to think critically about generative AI software. Software that uses generative AI, like chatbots and image generators, is relatively new and still being shaped by forces throughout society, from the software's creators, to its investors, to the public (and users like you). Nothing is inevitable about the use of generative AI. If you choose to use generative AI, remember that you are contributing to its development by providing free data to the company operating the software. You may want to ask yourself: can I trust this company? Can I rely on this software? Is this software really helping me?

When deciding whether to use generative AI, it is important to understand it practically and contextually. Below is a simple yet comprehensive tool we created to help you evaluate software that uses generative AI on both levels.

The POTATO Test

  • Purpose
    • What is the stated purpose/function of the software? How does it achieve this purpose?
  • Ownership
    • Who owns the software?
    • What are the owners' stated goals in developing this software?
    • What external information exists about the founders, investors, and leaders developing this software?
  • Transparency
    • Are the data sources used in the software's training disclosed anywhere?
    • Is there information available on how the generative AI was trained?
    • Are potential biases and flaws with the software disclosed?
  • Accuracy
    • Can the information generated by the software be independently verified? [Geared towards chatbots]
    • Does the software's output contain obvious biases?
  • Transhumanism
    • Why are you using this software? Is it helping you in a clear way?
    • How does using this software impact your cognitive and/or creative abilities?
    • What are the trade-offs in using this technology? Will using it make you rely on it more or less?
  • Operability
    • Does the software's output reflect the elements of your prompt?
    • How easy and intuitive is it to interact with the software?