HMPNN_Layer

HMPNN (Hypergraph Message Passing Neural Network) Layer introduced in Heydari et Livi 2022.

class topomodelx.nn.hypergraph.hmpnn_layer.HMPNNLayer(in_channels, node_to_hyperedge_messaging_func=None, hyperedge_to_node_messaging_func=None, adjacency_dropout: float = 0.7, aggr_func: Literal['sum', 'mean', 'add'] = 'sum', updating_dropout: float = 0.5, updating_func=None, **kwargs)

HMPNN Layer [1].

The layer is a hypergraph comprised of nodes and hyperedges that makes their new reprsentation using the input representation and the messages passed between them. In this layer, the message passed from a node to its neighboring hyperedges is only a function of its input representation, but the message from a hyperedge to its neighboring nodes is also a function of the messages recieved from them beforehand. This way, a node could have a more explicit effect on its upper adjacent neighbors i.e. the nodes that it share a hyperedge with.

\[\begin{split}\begin{align*} &🟥 \quad m_{{y \rightarrow z}}^{(0 \rightarrow 1)} = M_\mathcal{C} (h_y^{t,(0)}, h_z^{t, (1)})\\ &🟧 \quad m_{z'}^{(0 \rightarrow 1)} = AGG'{y \in \mathcal{B}(z)} m_{y \rightarrow z}^{(0\rightarrow1)}\\ &🟧 \quad m_{z}^{(0 \rightarrow 1)} = AGG_{y \in \mathcal{B}(z)} m_{y \rightarrow z}^{(0 \rightarrow 1)}\\ &🟥 \quad m_{z \rightarrow x}^{(1 \rightarrow0)} = M_\mathcal{B}(h_z^{t,(1)}, m_z^{(1)})\\ &🟧 \quad m_x^{(1 \rightarrow0)} = AGG_{z \in \mathcal{C}(x)} m_{z \rightarrow x}^{(1 \rightarrow0)}\\ &🟩 \quad m_x^{(0)} = m_x^{(1 \rightarrow 0)}\\ &🟩 \quad m_z^{(1)} = m_{z'}^{(0 \rightarrow 1)}\\ &🟦 \quad h_x^{t+1, (0)} = U^{(0)}(h_x^{t,(0)}, m_x^{(0)})\\ &🟦 \quad h_z^{t+1,(1)} = U^{(1)}(h_z^{t,(1)}, m_{z}^{(1)}) \end{align*}\end{split}\]

Parameters:

in_channelsint

Dimension of input features.

node_to_hyperedge_messaging_funcNone

Node messaging function as a callable or nn.Module object. If not given, a linear plus sigmoid function is used, according to the paper.

hyperedge_to_node_messaging_funcNone

Hyperedge messaging function as a callable or nn.Module object. It gets hyperedge input features and aggregated messages of nodes as input and returns hyperedge messages. If not given, two inputs are concatenated and a linear layer reducing back to in_channels plus sigmoid is applied, according to the paper.

adjacency_dropoutint, default = 0.7

Adjacency dropout rate.

aggr_funcLiteral[“sum”, “mean”, “add”], default=”sum”

Message aggregation function.

updating_dropoutint, default = 0.5

Regular dropout rate applied to node and hyperedge features.

updating_funccallable or None, default = None

The final function or nn.Module object to be called on node and hyperedge features to retrieve their new representation. If not given, a linear layer is applied, received message is added and sigmoid is called.

**kwargsoptional

Additional arguments for the layer modules.

Methods

add_module(name, module)	Adds a child module to the current module.
apply(fn)	Applies fn recursively to every submodule (as returned by .children()) as well as self.
apply_regular_dropout(x)	Apply regular dropout according to the paper.
bfloat16()	Casts all floating point parameters and buffers to bfloat16 datatype.
buffers([recurse])	Returns an iterator over module buffers.
children()	Returns an iterator over immediate children modules.
cpu()	Moves all model parameters and buffers to the CPU.
cuda([device])	Moves all model parameters and buffers to the GPU.
double()	Casts all floating point parameters and buffers to double datatype.
eval()	Sets the module in evaluation mode.
extra_repr()	Set the extra representation of the module
float()	Casts all floating point parameters and buffers to float datatype.
forward(x_0, x_1, incidence_1)	Forward computation.
get_buffer(target)	Returns the buffer given by target if it exists, otherwise throws an error.
get_extra_state()	Returns any extra state to include in the module's state_dict.
get_parameter(target)	Returns the parameter given by target if it exists, otherwise throws an error.
get_submodule(target)	Returns the submodule given by target if it exists, otherwise throws an error.
half()	Casts all floating point parameters and buffers to half datatype.
ipu([device])	Moves all model parameters and buffers to the IPU.
load_state_dict(state_dict[, strict])	Copies parameters and buffers from state_dict into this module and its descendants.
modules()	Returns an iterator over all modules in the network.
named_buffers([prefix, recurse, ...])	Returns an iterator over module buffers, yielding both the name of the buffer as well as the buffer itself.
named_children()	Returns an iterator over immediate children modules, yielding both the name of the module as well as the module itself.
named_modules([memo, prefix, remove_duplicate])	Returns an iterator over all modules in the network, yielding both the name of the module as well as the module itself.
named_parameters([prefix, recurse, ...])	Returns an iterator over module parameters, yielding both the name of the parameter as well as the parameter itself.
parameters([recurse])	Returns an iterator over module parameters.
register_backward_hook(hook)	Registers a backward hook on the module.
register_buffer(name, tensor[, persistent])	Adds a buffer to the module.
register_forward_hook(hook, *[, prepend, ...])	Registers a forward hook on the module.
register_forward_pre_hook(hook, *[, ...])	Registers a forward pre-hook on the module.
register_full_backward_hook(hook[, prepend])	Registers a backward hook on the module.
register_full_backward_pre_hook(hook[, prepend])	Registers a backward pre-hook on the module.
register_load_state_dict_post_hook(hook)	Registers a post hook to be run after module's load_state_dict is called.
register_module(name, module)	Alias for add_module().
register_parameter(name, param)	Adds a parameter to the module.
register_state_dict_pre_hook(hook)	These hooks will be called with arguments: self, prefix, and keep_vars before calling state_dict on self.
requires_grad_([requires_grad])	Change if autograd should record operations on parameters in this module.
set_extra_state(state)	This function is called from load_state_dict() to handle any extra state found within the state_dict.
share_memory()	See torch.Tensor.share_memory_()
state_dict(*args[, destination, prefix, ...])	Returns a dictionary containing references to the whole state of the module.
to(*args, **kwargs)	Moves and/or casts the parameters and buffers.
to_empty(*, device)	Moves the parameters and buffers to the specified device without copying storage.
train([mode])	Sets the module in training mode.
type(dst_type)	Casts all parameters and buffers to dst_type.
xpu([device])	Moves all model parameters and buffers to the XPU.
zero_grad([set_to_none])	Sets gradients of all model parameters to zero.

__call__

References

[1]

Heydari S, Livi L. Message passing neural networks for hypergraphs. ICANN 2022. https://arxiv.org/abs/2203.16995

apply_regular_dropout(x)

Apply regular dropout according to the paper.

Unmasked features in a vector are scaled by d+k / d in which k is the number of masked features in the vector and d is the total number of features.

Parameters:

xtorch.Tensor

Input features.

Returns:

torch.Tensor

Output features.

forward(x_0, x_1, incidence_1)

Forward computation.

Parameters:

x_0torch.Tensor, shape = (n_nodes, node_in_channels)

Input features of the nodes.

x_1torch.Tensor, shape = (n_edges, hyperedge_in_channels)

Input features of the hyperedges.

incidence_1torch.sparse.Tensor, shape = (n_nodes, n_edges)

Incidence matrix mapping hyperedges to nodes (B_1).

Returns:

x_0torch.Tensor, shape = (n_nodes, node_in_channels)

Output features of the nodes.

x_1torch.Tensor, shape = (n_edges, hyperedge_in_channels)

Output features of the hyperedges.