Back to LessWrong

Information hazard

From Lesswrongwiki

Jump to: navigation, search

An information hazard is a concept coined by Nick Bostrom in a 2011 paper[1] for Review of Contemporary Philosophy. He defines it as follows;

Information hazard: A risk that arises from the dissemination or the potential dissemination of (true) information that may cause harm or enable some agent to cause harm.

Bostrom points out that this is in contrast to the generally accepted principle of information freedom and that, while rare, the possibility of information hazards needs to be considered when making information policies. He proceeds to categorize and define a large number of sub-types of information hazards. For example, he defines artificial intelligence hazard as

Artificial intelligence hazard: There could be computer-related risks in which the threat would derive primarily from the cognitive sophistication of the program rather than the specific properties of any actuators to which the system initially has access.

The table below is reproduced from (Bostrom 2011).


TYPOLOGY OF INFORMATION HAZARDS
I. By information transfer mode
Data hazard
Idea hazard
Attention hazard
Template hazard
Signaling hazard
Evocation hazard
II. By effect
TYPE SUBTYPE
ADVERSARIAL RISKS Competiveness hazard Enemy Hazard
Intellectual property hazard
Commitment hazard
Knowing-too-much hazard
RISKS TO SOCIAL ORGANIZATION AND MARKETS Norm hazard Information asymmetry Hazard
Unveiling hazard
Recognition hazard
RISKS OF IRRATIONALITY AND ERROR Ideological hazard
Distraction and temptation hazard
Role model hazard
Biasing hazard
De-biasing hazard
Neuropsychological hazard
Information-burying hazard
RISKS TO VALUABLE STATES AND ACTIVITIES Psychological reaction hazard Disappointment hazard
Spoiler hazard
Mindset hazard
Belief-constituted value hazard
(mixed) Embarrassment hazard
RISKS FROM INFORMATION TECHNOLOGY SYSTEMS Information system hazard Information infrastructure failure hazard
Information infrastructure misuse hazard
Artificial intelligence hazard
RISKS FROM DEVELOPMENT Development hazard


See also

References

  1. Bostrom, N. (2011). "Information Hazards: A Typology of Potential Harms from Knowledge". Review of Contemporary Philosophy 10: 44-79. http://www.nickbostrom.com/information-hazards.pdf.