The Guide to the Software Engineering Body of Knowledge describes prototyping as a method of requirements validation as well as requirements elicitation in Chapter 2, Section 6, Subsection 2:
Prototyping is commonly a means for validating the software engineer's
interpretation of the software requirements, as well as for eliciting
new requirements. As with elicitation, there is a range of prototyping
techniques and a number of points in the process where prototype
validation may be appropriate. The advantage of prototypes is that
they can make it easier to interpret the software engineer's
assumptions and, where needed, give useful feedback on why they are
This passage cites Software Requirements: Objects, Functions, and States, Requirements Engineering: Processes and Techniques, and Software Requirements Engineering.
From Object-Oriented Software Engineering: Practical Software Development using UML and Java, 2nd Edition, which was the textbook used when I took my undergraduate Introduction to Software Engineering course provides this definition in Chapter 4 (Developing Requirements):
A prototype is a program that is rapidly implemented and contains only a small part of the anticipated functionality of a complete system. Its purpose is to gather requirements by allowing software engineers to obtain early feedback about their ideas.
The book then goes on to describe paper prototypes and rapid prototypes.
Ian Sommerville's Software Engineering (8th edition) provides this high-level definition of prototyping in Chapter 7.3:
In this approach to validation, an executable model of the system is demonstrated to end-users and customers. They can experiment with this model to see if it meets their real needs.
In Chapter 17.4, Sommerville discusses the use of prototyping during requirements engineering to elicit and validate requirements, in system design to explore particular solutions, and in testing to run tests with the system that will be delivered. He cites a study by V.S. Gordon and J.M. Bieman, published in 1995, titled "Rapid prototyping: lessons learned", which was published in IEEE Software, which found that prototyping leads to improved usability, a closer match to user needs, improved design quality, improved maintainability, and reduced development effort.
In Software Requirements, Karl Wiegers defines prototyping as:
A partial, preliminary, or possible implementation of a program. Used to explore and validate requirements and design approaches. Types of prototypes include evolutionary, throwaway, paper, horizontal, and vertical. These can be combined, as in an evolutionary vertical prototype.
Chapter 13 is dedicated to reducing the risk of a software project through the use of prototyping and discusses horizontal, vertical, throwaway, evolutionary, paper, and electronic prototyping methods. He also discusses how to evaluate prototypes and the risks that could be incurred because of the use of prototyping. One of the largest traps of using a prototype that he identifies is a stakeholder who thinks that a prototype is an early version of production software.
From a project management perspective, Steve McConnell discusses prototyping in Rapid Development: Taming Wild Software Schedules. A few key points are the use of evolutionary prototyping as a lifecycle model, useful for situations with rapidly changing requirements or for systems that are not well understood. He also discusses evolutionary delivery, which is a blend of techniques from the evolutionary prototyping and staged delivery lifecycle models.
Some of McConnell's best practices include evolutionary prototyping, throwaway prototyping, and user interface prototyping. For each, he identifies the impacts of these techniques on schedule, visibility, risk, and project success, the major risks and trade-offs, and when using these techniques is appropriate and when it is not appropriate.