Fast Random Projection

Most real-world graphs contain node properties which store information about the nodes and what they represent. The FastRP algorithm in the GDS library extends the original FastRP algorithm with a capability to take node properties into account. The resulting embeddings can therefore represent the graph more accurately.

The node property aware aspect of the algorithm is configured via the parameters featureProperties and propertyRatio. Each node property in featureProperties is associated with a randomly generated vector of dimension propertyDimension, where propertyDimension = embeddingDimension * propertyRatio. Each node is then initialized with a vector of size embeddingDimension formed by concatenation of two parts:

The algorithm then proceeds with the same logic as the FastRP algorithm. Therefore, the algorithm will output arrays of size embeddingDimension. The last propertyDimension coordinates in the embedding captures information about property values of nearby nodes (the "property part" below), and the remaining coordinates (embeddingDimension - propertyDimension of them; "topology part") captures information about nearby presence of nodes.

It may be useful to generate node embeddings with FastRP as a node property step in a machine learning pipeline (like Link prediction pipelines and Node property prediction).

In order for a machine learning model to be able to make useful predictions, it is important that features produced during prediction are of a similar distribution to the features produced during training of the model. Moreover, node property steps (whether FastRP or not) added to a pipeline are executed both during training, and during the prediction by the trained model. It is therefore problematic when a pipeline contains an embedding step which yields all too dissimilar embeddings during training and prediction.

This has some implications on how to use FastRP as a node property step. In general, if a pipeline is trained using FastRP as a node property step on some graph "g", then the resulting trained model should only be applied to graphs that are not too dissimilar to "g".

If propertyRatio<1.0, most of the nodes in the graph that a prediction is being run on, must be the same nodes (in the database sense) as in the original graph "g" that was used during training. The reason for this is that FastRP is a random algorithm, and in this case is seeded based on the nodes' ids in the Neo4j database from whence the nodes came.

If propertyRatio=1.0 however, the random initial node embeddings are derived from node property vectors only, so there is no random seeding based on node ids.

Additionally, in order for the initial random vectors (independent of propertyRatio used) to be consistent between runs (training and prediction calls), a value for the randomSeed configuration parameter must be provided when adding the FastRP node property step to the training pipeline.

In summary, if you set a random seed and set propertyRatio to 1, FastRP is inductive because embeddings are based only on node properties projected deterministically into vectors.

The embedding dimension is the length of the produced vectors. A greater dimension offers a greater precision, but is more costly to operate over.

The optimal embedding dimension depends on the number of nodes in the graph. Since the amount of information the embedding can encode is limited by its dimension, a larger graph will tend to require a greater embedding dimension. A typical value is a power of two in the range 128 - 1024. A value of at least 256 gives good results on graphs in the order of 10⁵ nodes, but in general increasing the dimension improves results. Increasing embedding dimension will however increase memory requirements and runtime linearly.

The normalization strength is used to control how node degrees influence the embedding. Using a negative value will downplay the importance of high degree neighbors, while a positive value will instead increase their importance. The optimal normalization strength depends on the graph and on the task that the embeddings will be used for. In the original paper, hyperparameter tuning was done in the range of [-1,0] (no positive values), but we have found cases where a positive normalization strengths gives better results.

The iteration weights parameter control two aspects: the number of iterations, and their relative impact on the final node embedding. The parameter is a list of numbers, indicating one iteration per number where the number is the weight applied to that iteration.

In each iteration, the algorithm will expand across all relationships in the graph. This has some implications:

It is good to have at least one non-zero weight in an even and in an odd position. Typically, using at least a few iterations, for example three, is recommended. However, a too high value will consider nodes far away and may not be informative or even be detrimental. The intuition here is that as the projections reach further away from the node, the less specific the neighborhood becomes. Of course, a greater number of iterations will also take more time to complete.

Node Self Influence is a variation of the original FastRP algorithm.

How much a node’s embedding is affected by the intermediate embedding at iteration i is controlled by the i'th element of iterationWeights. This can also be seen as how much the initial random vectors, or projections, of nodes that can be reached in i hops from a node affect the embedding of the node. Similarly, nodeSelfInfluence behaves like an iteration weight for a 0 th iteration, or the amount of influence the projection of a node has on the embedding of the same node.

A reason for setting this parameter to a non-zero value is if your graph has low connectivity or a significant amount of isolated nodes. Isolated nodes combined with using propertyRatio = 0.0 leads to embeddings that contain all zeros. However using node properties along with node self influence can thus produce more meaningful embeddings for such nodes. This can be seen as producing fallback features when graph structure is (locally) missing. Moreover, sometimes a node’s own properties are simply informative features and are good to include even if connectivity is high. Finally, node self influence can be used for pure dimensionality reduction to compress node properties used for node classification.

If node properties are not used, using nodeSelfInfluence may also have a positive effect, depending on other settings and on the problem.

Choosing the right orientation when creating the graph may have the single greatest impact. The FastRP algorithm is designed to work with undirected graphs, and we expect this to be the best in most cases. If you expect only outgoing or incoming relationships to be informative for a prediction task, then you may want to try using the orientations NATURAL or REVERSE respectively.

By default, the algorithm treats the graph relationships as unweighted. You can specify a relationship weight with the relationshipWeightProperty parameter to instruct the algorithm to compute weighted averages of the neighboring embeddings.

First off, we will estimate the cost of running the algorithm using the estimate procedure. This can be done with any execution mode. We will use the stream mode in this example. Estimating the algorithm is useful to understand the memory impact that running the algorithm on your graph will have. When you later actually run the algorithm in one of the execution modes the system will perform an estimation. If the estimation shows that there is a very high probability of the execution going over its memory limitations, the execution is prohibited. To read more about this, see Automatic estimation and execution blocking.

For more details on estimate in general, see Memory Estimation.

In the stream execution mode, the algorithm returns the embedding for each node. This allows us to inspect the results directly or post-process them in Cypher without any side effects. For example, we can collect the results and pass them into a similarity algorithm.

For more details on the stream mode in general, see Stream.

The results of the algorithm are not very intuitively interpretable, as the node embedding format is a mathematical abstraction of the node within its neighborhood, designed for machine learning programs. What we can see is that the embeddings have four elements (as configured using embeddingDimension) and that the numbers are relatively small (they all fit in the range of [-2, 2]). The magnitude of the numbers is controlled by the embeddingDimension, the number of nodes in the graph, and by the fact that FastRP performs euclidean normalization on the intermediate embedding vectors.

In the stats execution mode, the algorithm returns a single row containing a summary of the algorithm result. This execution mode does not have any side effects. It can be useful for evaluating algorithm performance by inspecting the computeMillis return item. In the examples below we will omit returning the timings. The full signature of the procedure can be found in the syntax section.

For more details on the stats mode in general, see Stats.

The stats mode does not currently offer any statistical results for the embeddings themselves. We can however see that the algorithm has successfully processed all seven nodes in our example graph.

The mutate execution mode extends the stats mode with an important side effect: updating the named graph with a new node property containing the embedding for that node. The name of the new property is specified using the mandatory configuration parameter mutateProperty. The result is a single summary row, similar to stats, but with some additional metrics. The mutate mode is especially useful when multiple algorithms are used in conjunction.

For more details on the mutate mode in general, see Mutate.

The returned result is similar to the stats example. Additionally, the graph 'persons' now has a node property fastrp-embedding which stores the node embedding for each node. To find out how to inspect the new schema of the in-memory graph, see Listing graphs.

The write execution mode extends the stats mode with an important side effect: writing the embedding for each node as a property to the Neo4j database. The name of the new property is specified using the mandatory configuration parameter writeProperty. The result is a single summary row, similar to stats, but with some additional metrics. The write mode enables directly persisting the results to the database.

For more details on the write mode in general, see Write.

The returned result is similar to the stats example. Additionally, each of the seven nodes now has a new property fastrp-embedding in the Neo4j database, containing the node embedding for that node.

Below is an example of running the weighted variant of algorithm.

Since the initial state of the algorithm is randomised, it isn’t possible to intuitively analyse the effect of the relationship weights.

To explain the novel initialization using node properties, let us consider an example where embeddingDimension is 10, propertyRatio is 0.2. The dimension of the embedded properties, propertyDimension is thus 2. Assume we have a property f1 of scalar type, and a property f2 storing arrays of length 2. This means that there are 3 features which we order like f1 followed by the two values of f2. For each of these three features we sample a two dimensional random vector. Let’s say these are p1=[0.0, 2.4], p2=[-2.4, 0.0] and p3=[2.4, 0.0]. Consider now a node (n {f1: 0.5, f2: [1.0, -1.0]}). The linear combination mentioned above, is in concrete terms 0.5 * p1 + 1.0 * p2 - 1.0 * p3 = [-4.8, 1.2]. The initial random vector for the node n contains first 8 values sampled as in the original FastRP paper, and then our computed values -4.8 and 1.2, totalling 10 entries.

In the example below, we again set the embedding dimension to 2, but we set propertyRatio to 1, which means the embedding is computed from node properties only.

In this example, the embeddings are based on the age property. Because of L2 normalization which is applied to each iteration (here only one iteration), all nodes have the same embedding despite having different age values (apart from rounding errors).