From 47030f01912c2a5fd4c6b6d4db788dd664947fa4 Mon Sep 17 00:00:00 2001 From: Stefan Plantikow Date: Mon, 7 May 2018 14:20:31 +0200 Subject: [PATCH 1/5] Initial version of catalog administration lifted from multiple graphs CIP --- .../CIP2018-05-03-catalog-administration.adoc | 131 ++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 cip/1.accepted/CIP2018-05-03-catalog-administration.adoc diff --git a/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc b/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc new file mode 100644 index 0000000000..bd1dfdd7b0 --- /dev/null +++ b/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc @@ -0,0 +1,131 @@ += CIP2018-05-03 Creating and administrating graphs and views +:numbered: +:toc: +:toc-placement: macro +:source-highlighter: codemirror + +*Author:* Stefan Plantikow , Andres Taylor , Petra Selmer + +This material is based on internal contributions from Alastair Green , Mats Rydberg , Martin Junghanns , Tobias Lindaaker + +[abstract] +.Abstract +-- +This CIP extends Cypher with support for creating and administrating graphs and views in the catalog. + +This CIP has been created in tandem with `CIP2017-06-18` for adding support for working with multiple graphs to Cypher and `CIP2016-06-22` for nested subqueries and relies on the material introduced in these proposals. +Therefore this CIP is based on the assumption that `CIP2017-06-18` and `CIP2016-06-22` will be accepted. +-- + +toc::[] + + + +== Introduction + +`CIP2017-06-18` introduces the notion of a catalog and adds support to Cypher for working with multiple graphs that may be either obtained from the catalog by name or constructed dynamically by a query. + +This proposal adds three additional capabilities: Managing the graphs in the catalog, managing views in the catalog and using them in a query, and finally query local declarations. + + +=== Related work + +This CIP has been developed in tandem with the following CIPs; as such, it is recommended to read all four CIPs in conjunction with each other. + + * `CIP2016-06-22`: Nested subqueries + * `CIP2018-05-04`: Equivalence operators, copy pattern, and related auxiliary functions + * `CIP2017-06-18`: Multiple graphs + + +== Managing graphs in the catalog + + +=== Creating graphs + +Creating new graphs in the catalog is done via the new schema command `CREATE GRAPH`. + +There are four forms of `CREATE GRAPH`: + +1. `CREATE GRAPH ` creates an empty graph in the catalog with the name ``. + +2. `CREATE GRAPH { }` creates a new graph in the catalog that is a copy of the graph that was returned by ``. +An error is raised if `` does not return a graph. + +3. `CREATE GRAPH myProc(...) AS ` creates a new graph in the catalog that is a copy of the graph that was returned by the call to `myProc`. +An error is raised if `myProc` does not return a graph. + +4. Multiple graphs may be created with one graph statement by separating multiple `GRAPH` subclauses of the forms 1, 2, or 3 with `,` (e.g. `CREATE GRAPH foo, GRAPH bar { ... }`) + +5. An error is raised when a graph is created with the name `` and there already is a graph with that name in the catalog. + + +=== Delete graph + +The catalog command `DELETE GRAPH ` deletes the graph with the name `` from the catalog. +If `` is not the name of a graph that already exists in the catalog, an error is raised. + + +=== Copy graph + +The catalog command `COPY TO ` copies the content and the schema of the graph with the name `` in the catalog to a new graph with the name `` in the catalog. +If `` is not the name of a graph that already exists in the catalog, an error is raised. +If `` is the name of a graph that already exists in the catalog, an error is raised. + + +=== Rename graph + +The catalog command `RENAME TO ` removes the graph with the name `` from the catalog and adds it as a new graph with the name `` in the catalog. +If `` is not the name of a graph that already exists in the catalog, an error is raised. +If `` is the name of a graph that already exists in the catalog, an error is raised. + + +=== Truncating graphs + +The catalog command `TRUNCATE ` truncates the graph with the name `` in the catalog. + +Truncating a graph deletes all its nodes and relationships but retains any additional schema information like constraints. + + +=== Determining the name of a graph + +The `catalog()` function returns the catalog name for the working graph or `NULL` if the working graph is a dynamically constructed graph. + + The `catalog(g)` function returns the catalog name for the graph identity `g` or `NULL` if `g` is a dynamically constructed graph. + +Note:: `toString(graph())` may be used to just generate a human readable name for the working graph. + + + +== Considerations + +=== Interaction with existing features + +None known. + + +=== Alternatives + +Instead of adding additional clauses the major part of the proposed functionality could be expressed using procedures. +However, catalog management was felt central enough to warrant proper inclusion into the language. + + +=== Syntax variations + +* `DROP GRAPH` instead of `DELETE GRAPH` + + +=== What others do + +SQL has followed a similar approach. + + +=== Benefits to this proposal + +Catalog management can be expressed using the Cypher language (instead of having to rely on implementation specific means). + + +=== Caveats to this proposal + +The size of the language is increased. +This makes it harder to learn Cypher. +However the chosen syntax is quite intuitive which is expected to at leat reduce the impact of this change on readability. From 5453c83c8c24a55fcf53f7bf849c9148eaa4ef54 Mon Sep 17 00:00:00 2001 From: Stefan Plantikow Date: Mon, 7 May 2018 15:39:33 +0200 Subject: [PATCH 2/5] Parameterized views and local declarations --- .../CIP2018-05-03-catalog-administration.adoc | 145 ++++++++++++++++-- 1 file changed, 131 insertions(+), 14 deletions(-) diff --git a/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc b/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc index bd1dfdd7b0..c043832a46 100644 --- a/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc +++ b/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc @@ -46,42 +46,42 @@ Creating new graphs in the catalog is done via the new schema command `CREATE GR There are four forms of `CREATE GRAPH`: -1. `CREATE GRAPH ` creates an empty graph in the catalog with the name ``. +1. `CREATE GRAPH ` creates an empty graph in the catalog with the name ``. -2. `CREATE GRAPH { }` creates a new graph in the catalog that is a copy of the graph that was returned by ``. -An error is raised if `` does not return a graph. +2. `CREATE GRAPH { }` creates a new graph in the catalog that is a copy of the graph that was returned by ``. +An error is raised if `` does not return a graph. -3. `CREATE GRAPH myProc(...) AS ` creates a new graph in the catalog that is a copy of the graph that was returned by the call to `myProc`. +3. `CREATE GRAPH myProc(...) AS ` creates a new graph in the catalog that is a copy of the graph that was returned by the call to `myProc`. An error is raised if `myProc` does not return a graph. 4. Multiple graphs may be created with one graph statement by separating multiple `GRAPH` subclauses of the forms 1, 2, or 3 with `,` (e.g. `CREATE GRAPH foo, GRAPH bar { ... }`) -5. An error is raised when a graph is created with the name `` and there already is a graph with that name in the catalog. +5. An error is raised when a graph is created with the name `` and there already is a graph with that name in the catalog. === Delete graph -The catalog command `DELETE GRAPH ` deletes the graph with the name `` from the catalog. -If `` is not the name of a graph that already exists in the catalog, an error is raised. +The catalog command `DELETE GRAPH ` deletes the graph with the name `` from the catalog. +If `` is not the name of a graph that already exists in the catalog, an error is raised. === Copy graph -The catalog command `COPY TO ` copies the content and the schema of the graph with the name `` in the catalog to a new graph with the name `` in the catalog. +The catalog command `COPY TO ` copies the content and the schema of the graph with the name `` in the catalog to a new graph with the name `` in the catalog. If `` is not the name of a graph that already exists in the catalog, an error is raised. If `` is the name of a graph that already exists in the catalog, an error is raised. === Rename graph -The catalog command `RENAME TO ` removes the graph with the name `` from the catalog and adds it as a new graph with the name `` in the catalog. -If `` is not the name of a graph that already exists in the catalog, an error is raised. -If `` is the name of a graph that already exists in the catalog, an error is raised. +The catalog command `RENAME TO ` removes the graph with the name `` from the catalog and adds it as a new graph with the name `` in the catalog. +If `` is not the name of a graph that already exists in the catalog, an error is raised. +If `` is the name of a graph that already exists in the catalog, an error is raised. === Truncating graphs -The catalog command `TRUNCATE ` truncates the graph with the name `` in the catalog. +The catalog command `TRUNCATE ` truncates the graph with the name `` in the catalog. Truncating a graph deletes all its nodes and relationships but retains any additional schema information like constraints. @@ -90,14 +90,131 @@ Truncating a graph deletes all its nodes and relationships but retains any addit The `catalog()` function returns the catalog name for the working graph or `NULL` if the working graph is a dynamically constructed graph. - The `catalog(g)` function returns the catalog name for the graph identity `g` or `NULL` if `g` is a dynamically constructed graph. +The `catalog(g)` function returns the catalog name for the graph identity `g` or `NULL` if `g` is a dynamically constructed graph. Note:: `toString(graph())` may be used to just generate a human readable name for the working graph. +== Views + +A view is a query that is stored in the catalog (in the same way as graphs) but is re-evaluated whenever it is referenced. +This is called an activation. + + +=== Creating views in the catalog + +Views are created in the catalog using syntax `CREATE QUERY { }`. + +Multiple views and graphs may be created by one `CREATE GRAPH/QUERY` schema command. + + +=== Managing views in the catalog + +Views in the catalog can be managed with `COPY`, `RENAME`, `DELETE QUERY` in the same way as graphs. + +An error is raised when attempting to delete a graph using `DELETE QUERY` or a view using `DELETE GRAPH`. + + +== View activation + +_Definition_: A view is activated whenever it is referenced from within a reading or updating statement. + +View activation executes the query that was associated with the view and returns the graph as the query result for actual use. + +The following forms of view activation currently exist in Cypher: + +1. `FROM ` +2. `UPDATE ` +3. `RETURN CALL ` +4. `RETURN GRAPH ` + + +== Local declarations + +Boths graphs and views may be declared at the start of a composite statement. + +The syntax for local graph declarations is + +[source, cypher] +---- +GRAPH < local graph name > { AS < local graph name > +GRAPH myProc(...) AS < local graph name > +---- + +The syntax for local graph declarations is + +[source, cypher] +---- +QUERY < local view name > { < nested subquery > } +---- + +To avoid conflicts, it is suggested that `` and `` are identifiers that start with a `_`. + +Semantics: + +1. `` must not be a correlated subqueries. + +2. An error is raised, if a local declaration would shadow an already exisint local declaration. + +Note:: Restriction 1 is likely going to be lifted in the future. + + +== Parameterized views + +Both views stored in the catalog and locally declared views may be parameterized with view arguments + +[source, cypher] +---- +QUERY _myView(, ...) { + +} +---- + +Activation of a parameterized view requires providing view arguments to the activation. + +1. View arguments use the same namespace as parameters. + +2. View arguments may be evaluated from any valid constant expression, i.e. an expression that only references literals or parameters in scope. +However grouped nested subqueries may be used to make additional parameters available inside a subquery. + +3. An error is raised if a local view declares a view argument that is already bound (either passed as a parameter or via a grouped nested subquery). + +It is recommended that a warning is raised if a catalog view references a parameter that is not an explicitly bound view argument. + +Furthermore, views may express expectations on the passed bindings: + +[source, cypher] +---- +QUERY _myView(args) { + WITH a, b + +} +---- + +This alternative form of argument passing is needed for grouped nested subqueries in order to distinguish between arguments that are evaluated over parameters and the grouping key and variable bindings available in all records for the same grouping key. + +Activation of a view with binding expectations may rename those bindings: + +[source, cypher] +---- +QUERY _myView($constant) { + WITH a, b + +} +MATCH (x)-[r:KNOWS]->(y) +CALL PER since _myView(5 WITH x AS a, y AS b) YIELD ... +... +---- + == Considerations +This CIP aims to bring together different concepts and syntactic ideas introduced in the design of Cypher for multiple graphs and the CIP for nested subqueries. + +It therefore tries to respect the guiding principles already expressed in those CIPs and other related proposals. + + === Interaction with existing features None known. @@ -116,7 +233,7 @@ However, catalog management was felt central enough to warrant proper inclusion === What others do -SQL has followed a similar approach. +SQL has followed a similar approach in that it allows to register both views and tables in a global catalog. === Benefits to this proposal From 514678b1ec47519fdf4a8940acd73b38129c2c58 Mon Sep 17 00:00:00 2001 From: Stefan Plantikow Date: Mon, 7 May 2018 16:00:20 +0200 Subject: [PATCH 3/5] Sketched grammar --- .../CIP2018-05-03-catalog-administration.adoc | 39 +++++++++++++++++-- 1 file changed, 36 insertions(+), 3 deletions(-) diff --git a/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc b/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc index c043832a46..0a23e2b642 100644 --- a/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc +++ b/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc @@ -146,14 +146,14 @@ The syntax for local graph declarations is [source, cypher] ---- -QUERY < local view name > { < nested subquery > } +QUERY < local name > { < composite statement > } ---- -To avoid conflicts, it is suggested that `` and `` are identifiers that start with a `_`. +`` are identifiers that start with a `_`. Semantics: -1. `` must not be a correlated subqueries. +1. `` must not be a correlated subqueries. 2. An error is raised, if a local declaration would shadow an already exisint local declaration. @@ -207,6 +207,39 @@ CALL PER since _myView(5 WITH x AS a, y AS b) YIELD ... ... ---- +== Grammar + +[source, ebnf] +---- + ::= CREATE < catalog item list > + | COPY TO + | RENAME TO + | TRUNCATE + | DELETE GRAPH + ; + + ::= < local declaration > [ { `,` < local declaration > } ; + + { < composite statement > } + | GRAPH < local name > [ { < composite statement > } ] + | GRAPH < invocation > AS < local name > + ; + + ::= < view name > [ `(` < view arguments > `)` ] ; + + ::= < expr > [ { `,` < expr > } ... < table arguments > ] ; + + ::= [ WITH < item > [ { `,` < item > } ] ; + + ::= < expr > AS < identifier > ; + + ::= < catalog name > | < local name > ; + +-- no leading _ allowed + ::= identifier [ { `.` identifier } ... ] ; + + ::= `_` identifier ; +---- == Considerations From d4164f2228e1e7257613cc30683ef7dd16eebbfa Mon Sep 17 00:00:00 2001 From: Stefan Plantikow Date: Mon, 7 May 2018 16:05:28 +0200 Subject: [PATCH 4/5] Clarification on passed through binding expectations --- cip/1.accepted/CIP2018-05-03-catalog-administration.adoc | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc b/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc index 0a23e2b642..8a59682c2f 100644 --- a/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc +++ b/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc @@ -194,7 +194,7 @@ QUERY _myView(args) { This alternative form of argument passing is needed for grouped nested subqueries in order to distinguish between arguments that are evaluated over parameters and the grouping key and variable bindings available in all records for the same grouping key. -Activation of a view with binding expectations may rename those bindings: +Activation of a view with binding expectations may provide those bindings via renaming. [source, cypher] ---- @@ -207,6 +207,9 @@ CALL PER since _myView(5 WITH x AS a, y AS b) YIELD ... ... ---- +If binding expectations as just passed through by the inner query, they are _not_ added as additional bindings in the result records. + + == Grammar [source, ebnf] From 10b146b5416be8990a7f08fecb1937d05bba6e60 Mon Sep 17 00:00:00 2001 From: Stefan Plantikow Date: Tue, 8 May 2018 10:07:30 +0200 Subject: [PATCH 5/5] Fixed title --- cip/1.accepted/CIP2018-05-03-catalog-administration.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc b/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc index 8a59682c2f..6dfb9fc6a5 100644 --- a/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc +++ b/cip/1.accepted/CIP2018-05-03-catalog-administration.adoc @@ -1,4 +1,4 @@ -= CIP2018-05-03 Creating and administrating graphs and views += CIP2018-05-03 Creating and managing graphs and views :numbered: :toc: :toc-placement: macro