Simplicial Complex Convolutional Neural Network Layer.

class topomodelx.nn.simplicial.sccnn_layer.SCCNNLayer(in_channels, out_channels, conv_order, sc_order, aggr_norm: bool = False, update_func=None, initialization: str = 'xavier_normal')

Layer of a Simplicial Complex Convolutional Neural Network.

Parameters:

in_channelstuple of int

Dimensions of input features on nodes, edges, and triangles.

out_channelstuple of int

Dimensions of output features on nodes, edges, and triangles.

conv_orderint

Convolution order of the simplicial filters. To avoid too many parameters, we consider them to be the same.

sc_orderint

SC order.

aggr_normbool, default = False

Whether to normalize the aggregated message by the neighborhood size.

update_funcstr, default = None

Activation function used in aggregation layers.

initializationstr, default = “xavier_normal”

Weight initialization method.

Methods

add_module(name, module)	Adds a child module to the current module.
aggr_norm_func(conv_operator, x)	Perform aggregation normalization.
apply(fn)	Applies fn recursively to every submodule (as returned by .children()) as well as self.
bfloat16()	Casts all floating point parameters and buffers to bfloat16 datatype.
buffers([recurse])	Returns an iterator over module buffers.
chebyshev_conv(conv_operator, conv_order, x)	Perform Chebyshev convolution.
children()	Returns an iterator over immediate children modules.
cpu()	Moves all model parameters and buffers to the CPU.
cuda([device])	Moves all model parameters and buffers to the GPU.
double()	Casts all floating point parameters and buffers to double datatype.
eval()	Sets the module in evaluation mode.
extra_repr()	Set the extra representation of the module
float()	Casts all floating point parameters and buffers to float datatype.
forward(x_all, laplacian_all, incidence_all)	Forward computation (see [1]_).
get_buffer(target)	Returns the buffer given by target if it exists, otherwise throws an error.
get_extra_state()	Returns any extra state to include in the module's state_dict.
get_parameter(target)	Returns the parameter given by target if it exists, otherwise throws an error.
get_submodule(target)	Returns the submodule given by target if it exists, otherwise throws an error.
half()	Casts all floating point parameters and buffers to half datatype.
ipu([device])	Moves all model parameters and buffers to the IPU.
load_state_dict(state_dict[, strict])	Copies parameters and buffers from state_dict into this module and its descendants.
modules()	Returns an iterator over all modules in the network.
named_buffers([prefix, recurse, ...])	Returns an iterator over module buffers, yielding both the name of the buffer as well as the buffer itself.
named_children()	Returns an iterator over immediate children modules, yielding both the name of the module as well as the module itself.
named_modules([memo, prefix, remove_duplicate])	Returns an iterator over all modules in the network, yielding both the name of the module as well as the module itself.
named_parameters([prefix, recurse, ...])	Returns an iterator over module parameters, yielding both the name of the parameter as well as the parameter itself.
parameters([recurse])	Returns an iterator over module parameters.
register_backward_hook(hook)	Registers a backward hook on the module.
register_buffer(name, tensor[, persistent])	Adds a buffer to the module.
register_forward_hook(hook, *[, prepend, ...])	Registers a forward hook on the module.
register_forward_pre_hook(hook, *[, ...])	Registers a forward pre-hook on the module.
register_full_backward_hook(hook[, prepend])	Registers a backward hook on the module.
register_full_backward_pre_hook(hook[, prepend])	Registers a backward pre-hook on the module.
register_load_state_dict_post_hook(hook)	Registers a post hook to be run after module's load_state_dict is called.
register_module(name, module)	Alias for add_module().
register_parameter(name, param)	Adds a parameter to the module.
register_state_dict_pre_hook(hook)	These hooks will be called with arguments: self, prefix, and keep_vars before calling state_dict on self.
requires_grad_([requires_grad])	Change if autograd should record operations on parameters in this module.
reset_parameters([gain])	Reset learnable parameters.
set_extra_state(state)	This function is called from load_state_dict() to handle any extra state found within the state_dict.
share_memory()	See torch.Tensor.share_memory_()
state_dict(*args[, destination, prefix, ...])	Returns a dictionary containing references to the whole state of the module.
to(*args, **kwargs)	Moves and/or casts the parameters and buffers.
to_empty(*, device)	Moves the parameters and buffers to the specified device without copying storage.
train([mode])	Sets the module in training mode.
type(dst_type)	Casts all parameters and buffers to dst_type.
update(x)	Update embeddings on each cell (step 4).
xpu([device])	Moves all model parameters and buffers to the XPU.
zero_grad([set_to_none])	Sets gradients of all model parameters to zero.

__call__

References

[1]

Papillon, Sanborn, Hajij, Miolane. Equations of topological neural networks (2023). awesome-tnns/awesome-tnns

Examples

Here we provide an example of pseudocode for SCCNN layer in an SC of order two input X_0: [n_nodes, in_channels] input X_1: [n_edges, in_channels] input X_2: [n_faces, in_channels]

graph Laplacian L_0: [n_nodes, n_nodes] 1-Lap_down L_1_down: [n_edges, n_edges] 1-Lap_up L_1_up: [n_edges, n_edges] 2-Lap L_2: [n_faces,n_faces] 1-incidence B_1: [n_nodes, n_edges] 2-incidence B_2: [n_edges, n_faces]

conv_order: int, e.g., 2

output Y_0: [n_nodes, out_channels] output Y_1: [n_edges, out_channels] output Y_2: [n_faces, out_channels]

SCCNN layer looks like:

Y_0 = torch.einsum( concat(

X_0, L_0@X_0, L_0@L_0@X_0 || B_1@X_1, B_1@L_1_down@X_1, B_1@L_1_down@L_1_down@X_1

), weight_0) Y_1 = torch.einsum( concat(

B_1.T@X_1, B_1.T@L_0@X_0, B_1.T@L_0@L_0@X_0 || X_1, L_1_down@X_1, L_1_down@L_1_down@X_1,

L_1_up@X_1, L_1_up@L_1_up@X_1 ||

B_2@X_2, B_2@L_2@X_2, B_2@L_2@L_2@X_2

), weight_1) Y_2 = torch.einsum( concat(

X_2, L_2@X_2, L_2@L_2@X_2 || B_2.T@X_1, B_2.T@L_1_up@X_1, B_2.T@L_1_up@L_1_up@X_1

), weight_2)

where

• weight_0, weight_2, weight_2 are the trainable parameters

• weight_0: [out_channels, in_channels, total_order_0]

  • total_order_0 = 1+conv_order + 1+conv_order

• weight_1: [out_channels, in_channels, total_order_1]

  • total_order_1 = 1+conv_order +

    1+conv_order+conv_order + 1+conv_order

• weight_2: [out_channels, in_channels, total_order_2]

  • total_order_2 = 1+conv_order + 1+conv_order

• to implement Lap_down@Lap_down@X, we consider chebyshev method

  to avoid matrix@matrix computation

aggr_norm_func(conv_operator, x)

Perform aggregation normalization.

chebyshev_conv(conv_operator, conv_order, x)

Perform Chebyshev convolution.

Parameters:

conv_operatortorch.sparse, shape = (n_simplices,n_simplices)

Convolution operator e.g., the adjacency matrix, or the Hodge Laplacians.

conv_orderint

The order of the convolution.

xtorch.Tensor, shape = (n_simplices,num_channels)

Feature tensor.

Returns:

torch.Tensor

Output tensor. x[:, :, k] = (conv_operator@….@conv_operator) @ x.

forward(x_all, laplacian_all, incidence_all)

Forward computation (see [1]_).

\[\begin{split}\begin{align*} &🟥 \quad m_{y \rightarrow z}^{(0\rightarrow1)} = B_1^T \cdot h_y^{t,(0)} \cdot \Theta^{t,(0 \rightarrow 1)}\\ &🟧 $\quad m_{z}^{(0\rightarrow1)} = \frac{1}\sum_{y \in \mathcal{B}(z)} m_{y \rightarrow z}^{(0\rightarrow1)} \qquad \text{where} \sum \text{represents a mean.}\\ &🟥 $\quad m_{z \rightarrow x}^{(1 \rightarrow 0)} = B_1\odot att(m_{z \in \mathcal{C}(x)}^{(0\rightarrow1)}, h_x^{t,(0)}) \cdot m_z^{(0\rightarrow1)} \cdot \Theta^{t,(1 \rightarrow 0)}\\ &🟧 $\quad m_x^{(1\rightarrow0)} = \sum_{z \in \mathcal{C}(x)} m_{z \rightarrow x}^{(1\rightarrow0)} \qquad \text{where} \sum \text{represents a mean.}\\ &🟩 \quad m_x^{(0)} = m_x^{(1\rightarrow0)}\\ &🟦 \quad h_x^{t+1, (0)} = \Theta^{t, \text{update}} \cdot (h_x^{t,(0)}||m_x^{(0)})+b^{t, \text{update}}\\ \end{align*}\end{split}\]

Parameters:

x_alltuple of tensors, shape = (x_0,x_1,x_2)

Tuple of input feature tensors:

• x_0: torch.Tensor, shape = (n_nodes,in_channels_0),

• x_1: torch.Tensor, shape = (n_edges,in_channels_1),

• x_2: torch.Tensor, shape = (n_triangles,in_channels_2).

laplacian_all: tuple of tensors, shape = (laplacian_0,laplacian_down_1,laplacian_up_1,laplacian_2)

Tuple of laplacian tensors:

• laplacian_0: torch.sparse, graph Laplacian,

• laplacian_down_1: torch.sparse, the 1-Hodge laplacian (lower part),

• laplacian_up_1: torch.sparse, the 1-hodge laplacian (upper part),

• laplacian_2: torch.sparse, the 2-hodge laplacian.

incidence_alltuple of tensors, shape = (b1,b2)

Tuple of incidence tensors:

• b1: torch.sparse, shape = (n_nodes,n_edges), node-to-edge incidence matrix,

• b2: torch.sparse, shape = (n_edges,n_triangles), edge-to-face incidence matrix.

Returns:

y_0torch.Tensor

Output features on nodes.

y_1torch.Tensor

Output features on edges.

y_2torch.Tensor

Output features on triangles.

reset_parameters(gain: float = 1.414)

Reset learnable parameters.

Parameters:

gainfloat

Gain for the weight initialization.

Notes

This function will be called by subclasses of MessagePassing that have trainable weights.

update(x)

Update embeddings on each cell (step 4).

Parameters:

xtorch.Tensor, shape = (n_target_cells, out_channels)

Feature tensor.

Returns:

torch.Tensor, shape = (n_target_cells, out_channels)

Updated output features on target cells.