Nonstandard Errors

Albert J. Menkveld; Ulf Nielsson; Oliver-Alexander Press; Anders Bjerre Trolle; Stefan Voigt; Thomas Lindner

doi:10.1111/jofi.13337

Nonstandard Errors

Albert J. Menkveld^*, Ulf Nielsson, Oliver-Alexander Press, Anders Bjerre Trolle, Stefan Voigt, Thomas Lindner

^*Corresponding author for this work

Research output: Contribution to journal › Journal article › Research › peer-review

56 Downloads (Pure)

Abstract

In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

Original language	English
Journal	Journal of Finance
Volume	79
Issue number	3
Pages (from-to)	2339-2390
Number of pages	52
ISSN	0022-1082
DOIs	https://doi.org/10.1111/jofi.13337
Publication status	Published - Jun 2024

Bibliographical note

343 authors have contributed to this article.
The authors included in this registration are the corresponding author, CBS-affiliated authors, as well as authors from other Danish universities.

Access to Document

10.1111/jofi.13337Licence: CC BY

Full TextFinal published version, 1.16 MBLicence: CC BY

Handle.net

Persistent link

Cite this

@article{7ba45f84f5174089b4dd711979b4e41d,

title = "Nonstandard Errors",

abstract = "In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.",

author = "Menkveld, {Albert J.} and Ulf Nielsson and Oliver-Alexander Press and Trolle, {Anders Bjerre} and Stefan Voigt and Thomas Lindner",

note = "343 authors have contributed to this article. The authors included in this registration are the corresponding author, CBS-affiliated authors, as well as authors from other Danish universities.",

year = "2024",

month = jun,

doi = "10.1111/jofi.13337",

language = "English",

volume = "79",

pages = "2339--2390",

journal = "Journal of Finance",

issn = "0022-1082",

publisher = "Wiley",

number = "3",

}

TY - JOUR

T1 - Nonstandard Errors

AU - Menkveld, Albert J.

AU - Nielsson, Ulf

AU - Press, Oliver-Alexander

AU - Trolle, Anders Bjerre

AU - Voigt, Stefan

AU - Lindner, Thomas

N1 - 343 authors have contributed to this article. The authors included in this registration are the corresponding author, CBS-affiliated authors, as well as authors from other Danish universities.

PY - 2024/6

Y1 - 2024/6

N2 - In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

AB - In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

U2 - 10.1111/jofi.13337

DO - 10.1111/jofi.13337

M3 - Journal article

SN - 0022-1082

VL - 79

SP - 2339

EP - 2390

JO - Journal of Finance

JF - Journal of Finance

IS - 3

ER -