Querying with Cypher¶

There are two ways to run Cypher, and most code wants the first:

The examples on this page share this setup:

import pyarrow as pa
from lance_csr import CypherEngine, GraphConfig

config = (
    GraphConfig.builder()
    .node_table("Person", "id")
    .node_table("City", "id")
    .rel_table("KNOWS", "Person", "Person", "src", "dst")
    .rel_table("LIVES_IN", "Person", "City", "src", "dst")
    .build()
)
datasets = {
    "Person": pa.table({
        "id": [1, 2, 3, 4],
        "name": ["Alice", "Bob", "Carol", "David"],
        "age": [28, 34, 29, 42],
    }),
    "City": pa.table({"id": [10, 20], "name": ["London", "Sydney"]}),
    "KNOWS": pa.table({"src": [1, 1, 2, 3], "dst": [2, 3, 4, 4]}),
    "LIVES_IN": pa.table({"src": [1, 2, 3, 4], "dst": [10, 10, 20, 20]}),
}
engine = CypherEngine(config, datasets, use_csr=True)

CypherEngine¶

CypherEngine(config, datasets, use_csr=False)

The constructor validates the datasets against the config, registers them with a DataFusion session, and, with use_csr=True, builds the CSR index. After that, every query reuses the same catalog and session.

execute(query, params=None)¶

Runs the query and returns the whole result as a pyarrow.Table:

table = engine.execute("""
    MATCH (p:Person)-[:LIVES_IN]->(c:City)
    RETURN c.name AS city, collect(p.name) AS people, avg(p.age) AS avg_age
    ORDER BY city
""")
print(table.to_pylist())
# [{'city': 'London', 'people': ['Alice', 'Bob'], 'avg_age': 31.0},
#  {'city': 'Sydney', 'people': ['Carol', 'David'], 'avg_age': 35.5}]

Output columns are named by their AS alias, or by the expression text (p.name) when there is none.

Parameters¶

Write $name in the query and pass a dict as the second argument. Parameter values can be numbers, strings, booleans, None, lists, string-keyed maps, and datetime.datetime or datetime.date values:

import datetime

engine.execute(
    "MATCH (p:Person) WHERE p.age >= $min_age AND p.name <> $skip RETURN p.name AS name",
    {"min_age": 30, "skip": "Bob"},
).to_pylist()  # [{'name': 'David'}]

Lists retain integer IDs, and maps can contain nested lists/maps and nulls:

engine.execute(
    "MATCH (p:Person) WHERE p.id IN $ids RETURN p.name AS name ORDER BY name",
    {"ids": [1, 3]},
).to_pylist()  # [{'name': 'Alice'}, {'name': 'Carol'}]

Pagination also accepts parameters, for example RETURN 1 AS n LIMIT $take with {"take": 1}. WITH ... LIMIT $take uses the same rules. Missing, null, boolean, fractional, negative and overflowing counts raise ValueError; arbitrary expressions and WITH SKIP remain unsupported.

Map keys must be strings, integer parameters must fit signed 64-bit values, and nesting is limited to 128 levels. Cyclic containers and unsupported Python objects raise ValueError. Lists follow literal-list coercion: mixed string/numeric values become strings because Arrow lists have one element type. Containers are expression values; inline node/relationship property filters still require scalar parameters.

Datetimes compare against timestamp columns, and dates against date columns. Parameters are substituted before planning, so they are also allowed in inline property maps: (p:Person {id: $id}).

execute_batches(query, params=None)¶

Streams the result as a pyarrow.RecordBatchReader instead of building one table:

reader = engine.execute_batches("MATCH (p:Person) RETURN p.id AS id, p.name AS name")
rows = 0
for batch in reader:  # each batch is a pyarrow.RecordBatch
    rows += batch.num_rows
print(rows)  # 4

• Batches are computed as you read them. Scans, filters and projections stream with bounded memory; joins, sorts and aggregations may still buffer their inputs.
• Planning errors (a syntax error, an unknown label) raise immediately. Execution errors raise while reading.
• The reader keeps its snapshot alive, so a concurrent refresh() doesn't affect it.
• Dropping the reader cancels the remaining work.

explain(query, params=None)¶

Returns the plan as text: the graph logical plan, DataFusion's logical plan and the physical plan. It's the quickest way to check whether a hop used the CSR (CsrExpandExec) or a join (HashJoinExec), and whether filters reached the Lance scan:

plan = engine.explain("MATCH (a:Person)-[:KNOWS]->(b:Person) RETURN b.name")
print("CsrExpandExec" in plan)  # True

Other methods¶

CypherQuery¶

CypherQuery parses a query up front and runs it against whatever datasets you hand it. Each builder method returns a new query:

from lance_csr import CypherQuery

query = (
    CypherQuery("MATCH (p:Person) WHERE p.age > $age RETURN p.name AS name ORDER BY name")
    .with_config(config)
    .with_parameter("age", 30)
)
print(query.execute(datasets).to_pydict())  # {'name': ['Bob', 'David']}

strategy takes an ExecutionStrategy. ExecutionStrategy.DataFusion is the default and the only one implemented; ExecutionStrategy.LanceNative is reserved and raises an error.

Supported Cypher¶

The Graph Data Council (GDC), formerly known as the Linked Data Benchmark Council (LDBC), maintains the GDC SNB Interactive complex queries. lance-csr implements a large read-only subset of openCypher, enough to run all 14 (IC14 without reduce()).

Clauses¶

Integer list literals retain signed 64-bit values: RETURN [1, 2, 3] AS ids returns integers, and UNWIND [1131, 19791209300775] AS id RETURN id preserves both IDs exactly. Numeric literals with a decimal point or exponent, such as [0.1, 1e-3], form float32 vectors. Whole-number lists can also be used as query vectors; conversion to float32 happens inside the vector operation. A configured engine is still required for standalone RETURN queries.

UNION removes duplicate rows; UNION ALL preserves them. Branches have independent variable scopes and keep their own ORDER BY, SKIP and LIMIT. Combined output has no guaranteed order. Mixed UNION modes are evaluated from left to right. Numeric columns can promote to a common numeric type; incompatible column counts, names or types raise ValueError. This follows the existing homogeneous Arrow value model.

engine.execute("RETURN 1 AS n UNION RETURN 1 AS n").to_pylist()  # [{'n': 1}]

Patterns¶

// Direction: outgoing, incoming or either
MATCH (a:Person)-[:KNOWS]->(b)     // outgoing
MATCH (a:Person)<-[:KNOWS]-(b)     // incoming
MATCH (a:Person)-[:KNOWS]-(b)      // either direction

// Variable length: *, *n, *n..m, *..m, *n..
MATCH (a:Person {id: 1})-[:KNOWS*1..3]->(b:Person)

// Alternation, also with variable length
MATCH (p:Person)-[:LIKES_POST|LIKES_COMMENT]->(m)

// Relationship variables and properties
MATCH (a:Person)-[k:KNOWS]->(b:Person) WHERE k.since > 2020

// Shortest paths (one relationship type, both ends bound)
MATCH p = shortestPath((a:Person {id: 1})-[:KNOWS*]-(b:Person {id: 4}))
RETURN length(p), nodes(p)

• Variable length defaults to a minimum of 1 and a maximum of 20 hops; a maximum above 20 is rejected, and zero-length paths (*0..) are not supported. Each length becomes one branch of a union.
• Undirected variable-length paths skip immediate backtracking (a-b-a).
• shortestPath and allShortestPaths need exactly one relationship type, no relationship variable, and named end nodes. Their length is unbounded unless you give a maximum. nodes(p) returns the node keys. With no path, MATCH drops the row and OPTIONAL MATCH returns null.
• Named paths (p = ...) are only supported with shortestPath and allShortestPaths.

Expressions and predicates¶

Functions¶

The string and list functions follow the Cypher v9 reference, within the list representation described above. size counts Unicode characters in a string or the outer elements of a list. left(text, length), right(text, length) and substring(text, start [, length]) count characters; the substring start is zero-based. Lengths and starts must be non-negative integers. A null input string returns null; a null start or length on a non-null string raises an error. trim, lTrim and rTrim remove whitespace, and replace replaces all literal matches. split preserves empty fields; an empty delimiter splits into characters. Splitting an empty string returns [''].

last([]) returns null; tail([]) and reverse([]) return empty lists. reverse also accepts strings. range(start, end [, step]) includes the end when reached, defaults to step 1, and accepts negative steps. A step pointing away from the end gives an empty list; zero steps and non-integer arguments raise an error. Unsigned Arrow arguments outside Int64 range also raise an error; they are not converted to null. Null arguments return null.

result = engine.execute("""
    RETURN substring('a猫😀z', 1, 2) AS text,
           reverse(range(1, 5, 2)) AS numbers
""")
print(result.to_pylist())  # [{'text': '猫😀', 'numbers': [5, 3, 1]}]

abs preserves integer results; the other math functions return floating point values. Integer abs raises an error on absolute-value overflow or unsigned inputs outside Int64 range. The other math functions convert numeric inputs, including UInt64, to Float64. round accepts one argument, with ties toward positive infinity (round(-1.5) is -1.0), as specified in the Neo4j 4.4 numeric reference. sqrt of a negative number returns NaN. These functions return null for null input and reject non-numeric values, including strings and booleans.

result = engine.execute("RETURN abs(-5) AS magnitude, round(-1.5) AS rounded, sqrt(9) AS root")
print(result.to_pylist())  # [{'magnitude': 5, 'rounded': -1.0, 'root': 3.0}]

Any other function raises ValueError with Cypher function 'name' is not implemented. This is a bounded function set, not a claim of full Cypher conformance. Remaining functions include keys, labels, reduce, the list predicates any / all / none / single, and relationships(p).

Example: a little of everything¶

result = engine.execute("""
    MATCH (p:Person)
    OPTIONAL MATCH (p)-[:KNOWS]->(f:Person)
    WITH p, count(f) AS friends
    WHERE EXISTS { MATCH (p)-[:LIVES_IN]->(:City {name: 'London'}) }
    RETURN p.name AS name,
           friends,
           CASE WHEN p.age >= 30 THEN 'senior' ELSE 'junior' END AS band
    ORDER BY name
""")
print(result.to_pylist())
# [{'name': 'Alice', 'friends': 2, 'band': 'junior'},
#  {'name': 'Bob', 'friends': 1, 'band': 'senior'}]

Case sensitivity¶

Keywords, function names, variables, labels, relationship types and property names are all case-insensitive. match (P:person) return p.NAME works, and the output column keeps the spelling you wrote.

An output alias may reuse an input variable's name. Expressions in the same RETURN still use the input variable; the following ORDER BY uses the output alias. For example, RETURN p.name AS p, p.id AS id ORDER BY p returns names and IDs, sorted by name. This also works with aggregation and OPTIONAL MATCH.

Errors¶