Published 2022-10-26T10:07:00.005Z by Physics Derivation Graph
Knowledge progresses when one person can leverage the insights of another person. There are levels of reproducibility that require different levels investment on the part of the person looking to build on the initial knowledge.
The levels described below are ranked from "requires lots of work to build upon" to "very easy to leverage."
example: "My design for this car tire supports operational speeds of 50 miles per hour and will be usable for 50,000 miles."
No software or analytical calculations are provided. No explanation of how claim was arrived at.
What distinguishes 2 from 1: Advertising that code was written, or math was done.
Most peer-reviewed scientific papers are written at this level or, if you're lucky, level 2.
Most presentations of experiments (e.g., at conferences and lectures) also are made at this level or level 2.
software-based example: Python script provided by the author to back up claim. No configuration file or random seed value. Library dependencies and versions need to be determined through trial and error (aka digital archeology).
analytical example: a few key equations (from a much more complex derivation) are mentioned, as are a few (not all) of the assumptions.
consequence: If you're smart and diligent, you may be able to recover statistically similar (though not exact) behavior for stochastic models, assuming neither you nor the original author had any bugs in the implementation.
Caveat: the levels described above are not actually linear. There are a few meandering paths that get from 0 to 6.