ArxivDataset(root, name, transform=None, pre_transform=None, pre_filter=None)

Bases: InMemoryDataset

ArXiv citation network dataset loader for graph-based analysis.

Parameters:
  • root (str) –

    Root directory where the dataset should be saved

  • name (str) –

    Name of the arXiv dataset

  • transform (callable, default: None ) –

    Function/transform that takes in a Data object and returns a transformed version. Default: None

  • pre_transform (callable, default: None ) –

    Function/transform to be applied to the data object before saving. Default: None

  • pre_filter (callable, default: None ) –

    Function that takes in a Data object and returns a boolean value, indicating whether the data object should be included. Default: None
Notes

Dataset Structure:

  • Nodes represent arXiv papers
  • Edges represent citations between papers
  • Node features from paper content
  • Labels indicate paper categories
  • Includes train/val/test splits (80/10/10)
processed_file_names property

Names of processed data files.

Returns:
  • list[str] –

    List of processed file names
Notes

Processed files:

  • data.pt: Contains processed PyTorch Geometric data object
raw_file_names property

Names of required raw files.

Returns:
  • list[str] –

    List of required raw file names
Notes

Required files:

  • *.pkl: Pickle file containing graph data, features, and labels
download()

Download raw data files.

Notes

Empty implementation - data should be manually placed in raw directory

process()

Process raw data into PyTorch Geometric Data format.

Notes

Processing Steps:

  • Load pickle file containing:

    • Edge indices (citations)
    • Node features (paper content)
    • Labels (paper categories)

  • Convert to PyTorch tensors

  • Create Data object with:

    • Edge indices
    • Node features
    • Node labels
    • Train/val/test masks

  • Apply pre-transform if specified

  • Save processed data

Data Split:

  • Training: 80%
  • Validation: 10%
  • Testing: 10%

Features:

  • Random split generation
  • Feature type conversion
  • Optional pre-transform support
  • Efficient data storage