navigation / documentation overview / design documentation

Recommendation: Read the user documentation and FAQ first. This page assumes familiarity with the jargon used in the Physics Derivation Graph.

This page provides background context for design decisions and implementation choices associated with the Physics Derivation Graph (PDG). Contributions to the project are welcome; see CONTRIBUTING.md on how to get started. The Physics Derivation Graph is covered by the Creative Commons Attribution 4.0 International License, so if you don't like a choice that was made you are welcome to fork the Physics Derivation Graph project.

Design Principles Documentation for the Physics Derivation Graph

This page enumerates design principles and goals for the Physics Derivation Graph (PDG).
The list is unordered.

Goals

Therefore

Expressions, symbols, operators, and relations exist only once in the database.

Expressions, symbols, operators, and relations are assigned unique identifiers; these are akin to Godel numbering.

Design Choices for the Physics Derivation Graph

Contents

 

Goal

Restating the goal from the front page of https://allofphysics.com/,

"Write down all known mathematical physics in a way that can be both read by humans and checked by a computer algebra system."

This page describes the current status and historical evolution of design decisions critical to the Physics Derivation Graph.

 

 

Decisions are not made independently; one choice informs others. Below is a visual overview of the relations among various decisions.

legend for shapes:

colors:

rule of connectivity for this graph:

 

 

Decision: What constitutes "all known mathematical physics"?

By topics

For a list of topics, see https://en.wikipedia.org/wiki/Branches_of_physics.

Googling major domains of physics and the answer is

classical physics (classical mechanics, thermodynamics, electromagnetism, optics) and
modern physics (quantum mechanics, relativity, atomic/nuclear/particle physics, condensed matter).

"Spanning all of Physics" would then require having a derivation involving pairs of domains. Not necessarily all pairs fully connected by derivations, but a minimum set of edges such that any domain can be reached from any other domain.

By assumptions

There is another approach that focuses on the dichotomy of assumptions

Notations

Notationally, mathematical physics includes Dirac notation, calculus, differential equations, algebra, trigonometry. (There is https://en.wikipedia.org/wiki/List_of_common_physics_notations but there's not comprehensive coverage.
See also Supported mathematical features

Notation isn't comprehensive: Geometry is certainly part of mathematical physics but the developer of this project hasn't figure out how to incorporate geometry.
See the full list of not in scope.

 

 

Decision: What constitutes "spanning" for all known mathematical physics?

Merely having derivations in each domain (e.g., quantum, classical, relativistic) does not suffice.

Suppose we have a derivation in the domain of quantum mechanics and a derivation in the domain of classical mechanics. Suppose that there was a variable (e.g., length) common to the two derivations.

Suppose we have a derivation in the domain of quantum mechanics and a derivation in the domain of classical mechanics. Suppose that there was an expression common to the two derivations.

Is there a single derivation that involves expressions that are quantum and expressions that are classical?

See also spanning the topics and assumptions of Physics, dichotomy of assumptions, and finding major edges of the Physics Derivation Graph.

 

 

Decision: What does "human-readable" mean?

Reasonable for a human to understand without use of specialized knowledge.

Raw Latex (like \int_0^{\infty}) is not understandable to everyone, whereas \( \int_0^{\infty} \) is). Similarly, raw contentML is not human-readable.

 

 

Decision: What does "checkable by CAS" mean?

In a derivation the mathematical steps can be verified as correct using symbolic mathematical software. (See which CAS for examples of applicable Computer Algebra Systems.)

Once a CAS is introduced, there are multiple aspects that can be checked:

The relevance of checking the math is to distinguish from just writing down symbols and expressions.

Why this matters: Reliability of machine-verified logic, Reproducibility, and Accessibility (no leaps of logic).

Decision: a CAS is not sufficient verification

A CAS is not sufficient as it may report \( 1 = x/x \) as true, even though \( x = 0 \) is false. That is why a proof assistant is necessary. (See which proof assistant?.)

 

 

Decision: Character support: ASCII or Unicode?

current status: Only ASCII is supported.

Why only ASCII?

 

 

Decision: Which Computer Algebra System(s) to use?

current status: using SymPy.

https://github.com/allofphysicsgraph/task-tracker/issues/117

See comparison of CAS like Mathematica, MathCad, Sage, Maple, SymPy.

 

 

Decision: Which proof assistant(s) to use?

current status: using Lean 4.

https://github.com/allofphysicsgraph/task-tracker/issues/106

See comparison of Rocq, Isabelle, Lean

 

 

Decision: How to represent expressions?

current status: using Latex.

See comparison of Latex, Content MathML, Presentation ML

 

 

Decision: Are feeds single use?

Suppose a step has feed "x/2". Should other steps re-use that feed? Should other derivations re-use that feed? Or should feeds only be connected to a single step?

current status: Feeds should only be used once.

Cypher query to find feeds connected to more than one step: 

  MATCH (f:feed)--(s:step)
WITH f, count(s) AS stepCount
WHERE stepCount > 1
RETURN f, stepCount

 

 

Decision: What parts of an expression are required to be specified?

Made 2024-05-28

design decision: Constrain user input of expressions to separately provide lhs, relation, rhs, when.

For example: 

F = m a when v<


(Previously (in v7) I accepted arbitrary Latex as input.)


 




 



Decision: Should user have to review existing content before adding new?



Made 2024-05-28


design decision: force user to see all expressions before adding new


The landing page intentionally does not have links to "create new expression" and "create new symbol" and "create new inference rule.

The user is forced to see all existing symbols (and expressions and inference rules) before creating a new instance.


The same reasoning applies to where "create new symbol" is placed on the page -- at the bottom, which forces the user to scroll past all existing symbols. 

Same for expressions and inference rules.



 




 



Decision: Neo4j schema for nodes?



Made 2024-05-28


design decision: treat "symbol" as a node type, rather than have separate node types for subcategories like 
"symbol_operation", "symbol_scalar", "symbol_vector", "symbol_matrix", "symbol_quantum_operator"


Each of these subcategories have distinct properties


https://physicsderivationgraph.blogspot.com/2024/05/distinguishing-scalars-vectors-and.html



 




 



Decision: Neo4j schema for nodes?



Made 2024-05-28


design decision: feed is a distinct node type


in v7 JSON each step has zero or more feeds, and each feed was in the same category as expressions. 


Feeds do not have the same attributes as symbols, nor do feeds have the same attributes as expressions. 

The category of symbols includes operators, scalars, vectors, matrices. 

Feeds can be composed of symbols, e.g., "x/a".

Feeds should not have equality or inequality and cannot be checked for dimensional consistency.


Feeds are specific to a step: "multiply both sides by" with the feed "x/a" used in two different steps refers to distinct feeds. 

However, a step could feature multiple feeds, and each feed has separate properties.




 




 



Decision: Should the server cache results or only serve live content?



Made 2024-05-28


design decision: live assessments are preferred over caching.


I don't care much about performance of page loads since page loads are less than 5 seconds. Therefore I see no value in caching results.


Also, caching could give a false response if the cache isn't refreshed appropriately.


Live assessments provide immediate indicators of failure (rather than obfuscating).


Caching would be additional complexity for the design and implementation.




 




 



Decision: validate expressions and steps at page-load time or off-line?



Made 2024-05-28


design decision: validate content at page-load time 


The cost of validating expressions and steps at page-load time is  longer page-load durations. 


The cost of validating expressions and steps off-line (e.g., at time of entry or modification) is lower certainty that nothing has changed (e.g., to a symbol)


I'm going to do validation at page-load time because validity isn't an intrinsic property of the node.




 




 



Decision: Should the sequence order number for steps be a property of the edge or node for steps?



Made 2024-05-28


design decision: property of edge.


Steps in a derivation are ordered. Two options for specifying the order:



I'm not currently clear which is correct, so I've chosen the edge property.



Sequence order also matters for arguments of an inference rule: there may be multiple input expressions. In this case sequence order must be an edge property since order isn't a property of either the inference rule or the expressions.


Sequence order matters for Abstract Syntax Trees for an expression. For example, 

"integrate  from  to  over "

requires a sequence of symbols -- they cannot be unordered. 

Again, in this case case sequence order must be an edge property since order isn't a property of either the expression or the symbol.


Based on the order number being an edge property for the two cases of inference rule and ASTs, I'm going with edge property between derivations and steps.



 




 



Decision: symbols in new expressions 



Made 2024-05-24


design decision: ordering symbols versus SymPy for new expressions


In v7 JSON the workflow for new expressions is
1) user provides Latex expression
2) user provides SymPy expression
3) user provides symbols


The reason for that order was that the symbol detection was coming from SymPy's parsing of the Latex.


In v8 Neo4j the workflow is
1) user provides Latex expression
2) user provides symbols
3) user provides SymPy expression


The new sequence (symbols from latex-as-string first, then send to sympy) might be 
advantageous: if the symbols are identified first, that might make SymPy's parsing more likely to be right. For example, parsing

a = b



using SymPy is more likely to succeed than

a = r_{\rm Earth}



which fails in SymPy. 



 




 



Decision: creating new expressions in isolation versus part of derivation 



Made 2024-05-24

design decision: creating new expressions as part of derivation.


In the v7 JSON implementation, the only way to create a new expression was to have the expression be associated with a specific derivation in a (new) step. The intent of this was to ensure there were no unconnected expressions. As a positive side-effect, associating a new expression with a derivation helped with guessing the symbols.


In the new (as of 2024) v8 Neo4j design, expressions can not be created during the specification of a derivation step. 

Now an expression has to be created prior to being referenced in a step. The value of separating these actions is that the interaction with the user is easier when processing one expression (rather than a bunch concurrently all related to the step).

Two consequences from this approach:




Interestingly, in the v7 JSON implementation a user could add new symbols that were not referenced by any expression.




 




 




Decision: separate tests from implementation or keep tests next to implementation?


Based on a recommendation from Gemini 3.5 Flash, tests are separate from the implementation.


Justifications:
  
    
    
  • Avoid Production Dependency Issues
    
The tests rely on testing frameworks (like pytest and playwright). If test code is in the same file as routes, then the production environment will parse the test-related imports. This means you would have to install development-only testing tools on your production server, which increases your deployment size and security attack surface.

    
  • Automatic Test Discovery
    
Testing tools like pytest are designed to automatically scan your project, find a directory named tests/, and run any file that starts with test_ (e.g., test_routes.py). Keeping them separate makes it easy to run all your tests with a single command (just typing pytest in your terminal).


  • Cleaner Deployment
    
When you package your application for production (such as building a Docker container or deploying to a cloud provider), you can easily exclude the tests/ directory. This keeps your production build lightweight and ensures you aren't deploying code to production that will never be executed there.


  • Readability and Maintainability
    
A route file can quickly grow to hundreds of lines of code as you add more endpoints. If you mix in 5 to 10 tests for every route, the file will quickly become bloated and difficult to navigate. Keeping them separate allows you to focus on writing routes in one file and testing them in another.




 




 





Decision: How to render graphs visually?



current status:  using d3js and graphviz.


https://github.com/allofphysicsgraph/task-tracker/issues/97



d3js, graphviz, networkx





 




 



Decision: Which Property Graph database?



current status:  using Neo4j.


https://github.com/allofphysicsgraph/task-tracker/issues/43



See comparison of Neo4j



 




 




Decision: Whether to cache results



current status: Not caching generated information.


Caching could lower latency for users of the website. However, to eliminate a risk of incorrect caching, validation or checking or queries are done at the query time.


 




 



Historical Decision: Upgrade path from JSON/SQL to Neo4j



The repo ui_v7 used JSON/SQL and ui_v8 used Neo4j. The ui_v7 repo had a working implementation of Google authentication. I had trouble getting Google authentication working in ui_v8, and I didn't want to have to refactor all the static content from ui_v7, so I decided to create a new repo, "allofphysics.com" (now renamed to https://github.com/allofphysicsgraph/combined_v7_JSON_and_v8_neo4j.



Although getting Neo4j into ui_v7 was do-able, the "mash together to repos" ended up being a bad decision from a troubleshooting and cleanliness-of-design experience.



 




 



Decision: Naming of git repository

  

I decided to name the repo "allofphysics.com" to make it clear which repo hosted the website on the Internet. Also, the alternative repo name "a mashup of v7 and v8" wasn't a good name, though it would have been more descriptive.



  
In retrospect, that was a bad naming convention because I later reverted to using ui_v7 for the website.



The "allofphysics.com" has been renamed to https://github.com/allofphysicsgraph/combined_v7_JSON_and_v8_neo4j.



As of 2026-02-06, ui_v8 is in use.


 




 



Decision: Page scope



current status:  using page-per-decision.


Rendered HTML pages should have a single scope. (As opposed to a single-page website.)



 




 




Decision: How to display Latex on webpages?



current status:  using MathJax.





 




 



Decision: Which VPS provider service company to use?



current status: Currently using Hetzner.


Example options: DigitalOcean, AWS, Oracle, Azure



https://github.com/allofphysicsgraph/task-tracker/issues/56



https://physicsderivationgraph.blogspot.com/2026/01/vps-price-comparison-september-2024.html

  https://physicsderivationgraph.blogspot.com/2026/01/vps-price-comparison-january-2026.html



 




 



Decision: Separate some pages into a separate Python file?



current status: flask routes are in single file (pdg_app.py).


Options: have different categories of routes in separate .py scripts, or have all routes in a single .py script.


Having all the routes in a single file (e.g., pdg_app.py) results in a huge file with thousands of lines.

To make this more managable, Flask provides a way to separate routes into separate files (e.g., pdg_other_routes.py) using blueprints.



As an example of how this woule be enacted, suppose the file pdg_app.py contains

from pdg_other_routes import other_routes_bp
web_app.register_blueprint(other_routes_bp)



and pdg_other_routes.py contains

other_routes_bp = Blueprint("other_routes", __name__)

@other_routes_bp.route("/api_via_js")
def to_api_via_js() -> str:
    return render_template("js_with_api/api_js.html")




The cost of this separation is that Flask namespaces the endpoints to ensure there are no name collisions between different files. 
  The url_for function in Jinja2 templates (and in the Python code) now expects the format:



other_routes.function_name




That would require figuring out which url_for functions point to routes in pdg_other_routes.py and which routes are in pdg_app.py




Therefore, having all routes in pdg_app.py is easier.




 




 



Decision: Which languages to use?



current status: using Python, HTML, Latex.


Languages I'm comfortable with and are widely used - Python; HTML; Docker




 




 



Decision: Is an interface needed?



current status: Direct write access to Neo4j is not allowed.


One could imagine allowing read-write access to the property graph database. That would allow users to make changes that violate the schema.
A property graph is overly permissive regarding what nodes, edges, and properties are allowable. 


Having an interface enforces compliance with the schema. The interface enforces
  



The backdoor to this is exporting the database and then importing modified content.


 




 



Decision: Which interface modality?



current status: primarily using web UI to read and write to Neo4j database. Exploring use of API.


Plan to support API.


Why web ui? Why not a GUI? Or command line?

Accessibility. Most users wouldn't be comfortable with command line UI. The web UI enforces the Neo4j schema.



The motive for adding the API is so that other developers can provide a better UI.


The reason to start with the web interface is to figure out which actions should be allowed and which actions should not be allowed. 
The same set of constraints and workflow will apply to the API.


 




 




Decision: Not hiring contractors to enact features



current status: I am not hiring a designer for the web front-end because that would be premature at this point. I'm still in the exploration prototyping phase to figure out what the front end will need to be able to do.



 




 



Decision: Which data structure to use in the Physics Derivation Graph



current status: the Physics Derivation Graph is stored in a Neo4j Property Graph database.



(There are intermediate data structures in pdg_app.py that interface with the jinja2 HTML pages.)

Not clear whether the site could just be using Neo4j directly by the web UI. 




  The many alternatives data storage options (e.g., SQLite, Redis, Python Pickle, CSV, GraphML) offer trade-offs, a few of which have been explored.


 




 



Decision: Why are there different node type?



In neo4j_query.py there are many node types:




The motive for having distinct node types is each has a specific set of property keys. The property keys of :scalar are different from the property keys relevant to :matrix.


The :scalar, :vector, :matrix nodes are grouped under :symbol. 
  There's an argument to be made that another hierarchical layer would be :operation, :relation, :quantum_operator, :value_with_units, :symbol. 
  Not clear what that grouping would be called. "Things that appear in expressions and feeds" is too long.


A counter-proposal would be that :operation, :relation, and :quantum_operator should be nested under :symbol. Currently those nodes are connected to :expression and :feed by :IS_COMPRISED_OF (rather than :HAS_RELATION and :HAS_OPERATION).


 




 



Decision: Choice of edge labels



The Physics Derivation Graph previously used HAS_SYMBOL between :expression and :symbol nodes. That label is bad as an edge because it implies a category of node. IS_COMPRISED_OF is better.


Could every edge be IS_COMPRISED_OF?

No. For example, HAS_STEP is separate because of the property key sequence index.
Similarly, HAS_INPUT, HAS_OUTPUT, HAS_FEED are right and each also should have an index .



 




 



Decision: Node IDs should be unique



When I started using Neo4j I enacted

:begin
CREATE CONSTRAINT ON (node:derivation) ASSERT (node.id) IS UNIQUE;
CREATE CONSTRAINT ON (node:step) ASSERT (node.id) IS UNIQUE;
CREATE CONSTRAINT ON (node:inference_rule) ASSERT (node.id) IS UNIQUE;
CREATE CONSTRAINT ON (node:feed) ASSERT (node.id) IS UNIQUE;
CREATE CONSTRAINT ON (node:expression) ASSERT (node.id) IS UNIQUE;
CREATE CONSTRAINT ON (node:value_with_units) ASSERT (node.id) IS UNIQUE;
CREATE CONSTRAINT ON (node:operation) ASSERT (node.id) IS UNIQUE;
CREATE CONSTRAINT ON (node:relation) ASSERT (node.id) IS UNIQUE;
CREATE CONSTRAINT ON (node:scalar) ASSERT (node.id) IS UNIQUE;
CREATE CONSTRAINT ON (node:vector) ASSERT (node.id) IS UNIQUE;
CREATE CONSTRAINT ON (node:matrix) ASSERT (node.id) IS UNIQUE;
:commit



However, I learned that merely means the id property is unique within that label. 


To enact globally-unique values I had to add a new label

CREATE CONSTRAINT ON (node:a_node) ASSERT (node.id) IS UNIQUE;




This is bug 131.



 




 



\(\rm\LaTeX\) representation of expressions



  There are multiple choices of how to represent a mathematical expression.
  The choices feature trade-offs between conciseness, ability to express the range of notations necessary for Physics, semantic meaning, and ability to use the expression in a computer algebra system (CAS).
  See the comparison of syntax.
  \(\rm\LaTeX\) was selected primarily because of the common use in Physics, display of complex math, conciseness, and expressiveness.
  The use of \(\rm\LaTeX\) means other tasks like parsing symbols and resolving ambiguity are harder.




 




 



\(\rm\LaTeX\) or SymPy as the primary representation of expressions?



Is the latex representation primary, or SymPy primary, or is one canonical? 


They are co-equals and intended to be equivalent.


 




 




Decision: Which objects should be represented?



There are a few obvious objects that need to be accounted for, like derivation, steps, inference rule, feed, 
  and expression.


Beyond those there are objects that could be a either node in the graph or a property of a node. For example, should (LHS, relation, RHS) be separate nodes or properties of an expression? A framing that motivates the choice is whether a user may want to query LHS separately from the expression. The trade-off is that additional nodes better support custom queries but then incur more queries to extract information relevant for typical workflows. 

  
Another framing to motivate the node-or-property decision is the nodes can have properties but properties cannot have edges. For example, if LHS is a property of expression, then a symbol-as-node has to be related to the expression rather than LHS.
    
:expression (LHS) -> IS_COMPRISED_OF -> :symbol (x)

    
versus
    
:expression -> HAS_SIDE -> LHS -> IS_COMPRISED_OF -> :symbol (x)



If "symbol" is a node, then is a relation is a symbol? Should the relation be a property of the expression-as-node, or should the schema be
  
:expression -> HAS_RELATION -> :relation (=)



 




 



Decision: Supported Mathematical features



Many but not all symbols are supported. Here are some categories of supported symbols. (Symbols exist on the left side or right side of an expression and can also appear in a feed.)




Operators. These act on symbols (listed above) and are part of either the left side or right side of an expression. Operators can also appear in feeds.




Relations. These evaluate to True for an expression. Relations do not appear in a feed.



To check the above documentation against the code, see https://github.com/allofphysicsgraph/ui_v8_website_flask_neo4j/blob/gh-pages/webserver_for_pdg/library/list_of_valid.py


 




 




Decision: Outside of Current Scope



Although the Physics Derivation Graph is intended to be comprehensive
  across domains, there are aspect of Physics not within the current scope of the project:



These aspects could be included if the data structure and workflow were adapted
to an expanded scope.