MAGDataset(root, name, transform=None, pre_transform=None, pre_filter=None)

Bases: InMemoryDataset

Microsoft Academic Graph (MAG) dataset loader for graph-based analysis.

Parameters:
  • root (str) –

    Root directory where the dataset should be saved

  • name (str) –

    Name of the MAG dataset

  • transform (callable, default: None ) –

    Function/transform that takes in a Data object and returns a transformed version. Default: None

  • pre_transform (callable, default: None ) –

    Function/transform to be applied to the data object before saving. Default: None

  • pre_filter (callable, default: None ) –

    Function that takes in a Data object and returns a boolean value, indicating whether the data object should be included. Default: None
Notes

Dataset Structure:

  • Nodes represent academic papers
  • Edges represent citation relationships
  • Node features from paper content
  • Labels indicate paper fields (top 20)
  • Includes train/val/test splits (80/10/10)
processed_file_names property

Names of processed data files.

Returns:
  • list[str] –

    List of processed file names
Notes

Processed files:

  • data.pt: Contains processed PyTorch Geometric data object
raw_file_names property

Names of required raw files.

Returns:
  • list[str] –

    List of required raw file names
Notes

Required files:

  • labels_20.pt: PyTorch file containing graph data with top 20 fields
download()

Download raw data files.

Notes

Empty implementation - data should be manually placed in raw directory

process()

Process raw data into PyTorch Geometric Data format.

Notes

Processing Steps:

  • Load PyTorch data:

    • Node features (paper content)
    • Edge indices (citations)
    • Labels (paper fields)

  • Create Data object with:

    • Edge indices
    • Node features
    • Node labels
    • Train/val/test masks

  • Apply pre-transform if specified

  • Save processed data

Data Split:

  • Training: 80%
  • Validation: 10%
  • Testing: 10%

Features:

  • Direct tensor loading
  • Random split generation
  • Optional pre-transform support
  • Efficient data storage