sad.callback package

Submodules

sad.callback.base module

class CallbackBase(config: Dict, caller: sad.callback.caller.CallerProtocol)

Bases: abc.ABC

A callback base class that every concrete callback subclass will inherit from.

Instance of this class will be managed by a caller instance that is compliant with CallerProtocol. Currently instances of sad.trainer.TrainerBase classes could be such callers. Callback instances will be created during caller’s initialization. Configurations for this callback is provided under caller:spec:callbacks:. An example is shown below:

trainer:
  name: SGDTrainer
  spec:
    n_iters: 20
    w_l1: 0.1
    w_l2: 0.0
    u_idxs: [0, 1, 2, 3]
    callbacks:
    - name: "CheckpointingCallback"
      spec:
        start: 10
        every: 1

property caller: sad.callback.caller.CallerProtocol

Reference to an instance of a caller class that is compliant with CallerProtocol. Could be an instance of sad.trainer.TrainerBase.

property config: Dict

Configuration dictionary that is used to initialize the instance.

abstract on_iter_begin(iter_idx: int, **kwargs)

Will be called from caller when an iteration begins. An iteration could be an epoch during training loop.

Parameters

iter_idx (int) – The index of iteration, 0-based.

abstract on_iter_end(iter_idx: int, **kwargs)

Will be called from caller when an iteration ends.

Parameters

iter_idx (int) – The index of iteration. 0-based.

abstract on_loop_begin(**kwargs)

Will be called from caller when main loop begins. The main loop could be training loop in sad.trainer.TrainerBase.

abstract on_loop_end(**kwargs)

Will be called from caller when main loop ends.

abstract on_step_begin(iter_idx: int, step_idx: int, **kwargs)

Will be called from caller when a step begins. A step could be one gradient updates from a minibatch during training.

Parameters

• iter_idx (int) – The index of iteration. 0-based.

• step_idx (int) – The index of step. 0-based.

abstract on_step_end(iter_idx: int, step_idx: int, **kwargs)

Will be called from caller when a step finishes.

Parameters

• iter_idx (int) – The index of iteration. 0-based.

• step_idx (int) – The index of step. 0-based.

property spec: Dict

A reference to "spec" field in self.config. When no such field exists or the value is None, an empty dictionary will be set.

class CallbackFactory

Bases: object

A factory class that is responsible to create callback instances.

logger = <Logger callback.CallbackFactory (INFO)>

Class attribute for logging.

Type

logging.Logger

classmethod produce(config: Dict, caller: sad.callback.caller.CallerProtocol) → sad.callback.base.CallbackBase

A class method to initialize instances of sad.callback.CallbackBase.

Parameters

• config (config) –

  Configuration used to initialize an instance object. An example is given below:

  name: "EarlyStoppingCallback"
  spec:
      allow_incomplete_epoch: False
  

• caller (sad.callback.CallerProtocol) – An instance of a class that is compliant with CallerProtocol. Currently sad.trainer.TrainerBase is of this class type. A callback instance will be created with its caller. During caller’s loop, callback methods will be invoked.

classmethod register(wrapped_class: sad.callback.base.CallbackBase) → sad.callback.base.CallbackBase

A class level decorator responsible to decorate sad.callback.CallbackBase classes and register them into CallbackFactory._registry.

sad.callback.caller module

class CallerProtocol

Bases: abc.ABC

A caller protocol that defines a set of interfaces that will be used to interact with instances of sad.callback.CallbackBase. Currently sad.trainer.TrainerBase is respecting this protocol.

property callbacks: List[sad.callback.CallbackBase]

A list of callback instances this caller owns.

abstract property config: Dict

Configuration dictionary that is used to initialize instances of classes compliant with CallerProtocal.

abstract property generator: sad.generator.base.GeneratorBase

An instance of sad.model.GeneratorBase. A reference to such an instance that will be used to generate data to train self.model.

initialize_callback()

Initialize callbacks. Callback configurations are supplied under trainer:spec:callbacks field in self.config. self.spec holds a reference to self.config["spec"].

Initialized instances of sad.callback.CallbackBase will be pushed to self.callbacks, with the same order as their appear in configuration caller:spec:callbacks.

abstract property model: sad.model.base.ModelBase

An instance of sad.model.ModelBase. A reference to such an instance that will be trained by the caller.

abstract property n_iters: int

An integer suggests how many iterations the caller will perform.

on_iter_begin(iter_idx: int, **kwargs)

Will be called when an iteration begins. An iteration could be an epoch during training.

Parameters

iter_idx (int) – The index of iteration, 0-based.

on_iter_end(iter_idx: int, **kwargs)

Will be called when an iteration finishes.

Parameters

iter_idx (int) – The index of an iteration. 0-based.

on_loop_begin(**kwargs)

Will be called when main loop begins.

on_loop_end(**kwargs)

Will be called when main loop finishes.

on_step_begin(iter_idx: int, step_idx: int, **kwargs)

Will be called when step begins. A step could be a gradient update from a minibatch during training loop.

Parameters

• iter_idx (int) – The index of iteration. 0-based.

• step_idx (int) – The index of step. 0-based.

on_step_end(iter_idx: int, step_idx: int, **kwargs)

Will be called when a step finishes.

Parameters

• iter_idx (int) – The index of iteration. 0-based.

• step_idx (int) – The index of step. 0-based.

register_callback(callback: CallbackBase)

Callback registration. The actual place where a callback instance is pushed to self.callbacks list. This function will be called when a callback instance is initialized - newly created callback instances will register themselves to their caller.

Parameters

callback (CallbackBase) – An instance of sad.callback.CallbackBase. It is at the initialization of callback argument when this method is called.

abstract property spec: Dict

A reference to "spec" field in self.config.

abstract property stop: bool

A flag to indicate caller if early stop is needed.

abstract property task: sad.task.base.TaskBase

An instance of sad.task.TaskBase. It is a reference to a task instance in which current caller is initialized.

sad.callback.metrics_logging module

class MetricsLoggingCallback(config: Dict, caller: sad.callback.caller.CallerProtocol)

Bases: sad.callback.base.CallbackBase

A callback class that is responsible to log metrics during caller’s main loop.

Instance of this class will be managed by caller instances that is compliant with sad.caller.CallerProtocol, during caller’s initialization. Configurations for this callback is provided under trainer:spec:callbacks:. An example is shown below:

trainer:
  name: SGDTrainer
  spec:
    n_iters: 20
    w_l1: 0.1
    w_l2: 0.0
    u_idxs: [0, 1, 2, 3]
    callbacks:
    - name: "MetricsLoggingCallback"
      spec:
        every_iter: 1
        every_step: 2

property every_iter: int

Number of iterations every logging event happens. Read directly from "every_iter" field in self.spec. A negative number suggests no metrics logging will happen at iteration ends.

property every_step: int

Number of steps every logging event happens. Read directly from "every_step" field in self.spec. A negative number suggests this callback will not log metrics at step ends.

property history: Dict[str, List[Tuple]]

A dictionary that holds metrics history. It has following fields:

history = {
    "step_end": [(iter_idx, step_idx, metrics), ... ],
    "iter_end": [(iter_idx, metrics), ... ]
}

The metrics in the list is a dictionary by itself, with following fields:

metrics = {
    "ll": float,  // log likelihood of trained model
    "t_sparsity": float,  // sparsity of right item matrix
    "mse": float,  // MSE wrt true parameter X, available in simulation
    "ll0": float   // True log likelihood, available in simulation
}

This information will be saved to metricsloggingcallback_history.json.

on_iter_begin(iter_idx: int)

Will be called from caller when an iteration begins. An iteration could be an epoch during training loop.

Parameters

iter_idx (int) – The index of iteration, 0-based.

on_iter_end(iter_idx: int, ll: float = - 1, t_sparsity: float = - 1, mse: float = - 1, ll0: float = - 1, **kwargs)

Will be called to determine whether to log metrics at the end of an iteration. After confirming, it will organize metrics to a dictionary and push the dictionary into a history queue. The format of the dictionary is shown below:

metrics = {
    "ll": float,  // log likelihood of trained model
    "t_sparsity": float,  // sparsity of right item matrix
    "mse": float,  // MSE wrt true parameter X, available in simulation
    "ll0": float   // True log likelihood, available in simulation
}

Parameters

• iter_idx (int) – The index of iteration, 0-based.

• ll (float) – Log likelihood at current iteration.

• t_sparsity (float) – The proportion of elements that are close to 1 in T matrix.

• mse (float) – The mean squared error between estimated item preference tensor and true tensor. Only available in simulation.

• ll0 (float) – Log likelihood under true parameter values. Only available in simulation.

on_loop_begin(**kwargs)

Will be called from caller when main loop begins. The main loop could be training loop in sad.trainer.TrainerBase.

on_loop_end(**kwargs)

Will be called when caller’s main loop finishes. When this method is triggered, the metrics history will be saved to a Json file with name metricsloggingcallback_history.json in the model folder.

on_step_begin(iter_idx: int, step_idx: int, **kwargs)

Will be called from caller when a step begins. A step could be one gradient updates from a minibatch during training.

Parameters

• iter_idx (int) – The index of iteration. 0-based.

• step_idx (int) – The index of step. 0-based.

on_step_end(iter_idx: int, step_idx: int, ll: float = - 1, t_sparsity: float = - 1, mse: float = - 1, ll0: float = - 1, **kwargs)

Will be called to determine whether to log metrics at end of a step. After confirming, it will organize metrics to a dictionary and push the dictionary into history queue. The format of the dictionary is shown below:

metrics = {
    "ll": float,  // log likelihood of trained model
    "t_sparsity": float,  // sparsity of right item matrix
    "mse": float,  // MSE wrt true parameter X, available in simulation
    "ll0": float   // True log likelihood, available in simulation
}

Parameters

• iter_idx (int) – The index of iteration, 0-based.

• step_idx (int) – The index of step, 0-based.

• ll (float) – Log likelihood at current step.

• t_sparsity (float) – The proportion of elements that are close to 1 in T matrix.

• mse (float) – The mean squared error between estimated item preference tensor and true tensor. Only available in simulation.

• ll0 (float) – Log likelihood under true parameter values. Only available in simulation.

sad.callback.w_l1_scheduler module

class WeightL1SchedulerCallback(config: Dict, caller: sad.callback.caller.CallerProtocol)

Bases: sad.callback.base.CallbackBase

A callback class that is responsible to update weight of L1 regularization during training.

Instance of this class will be managed by instances compliant with sad.caller.CallerProtocol instances, during caller’s’ initialization. Configurations for this callback is provided under trainer:spec:callbacks:. An example is shown below:

trainer:
  name: SGDTrainer
  spec:
    n_iters: 20
    w_l1: 0.1
    w_l2: 0.0
    u_idxs: [0, 1, 2, 3]
    callbacks:
    - name: "WeightL1SchedulerCallback"
      spec:
        scheme: "exp_rise"
        rate: -0.1
        start: 0.5

property every: int

Number of iterations every update is performed. 1 means weight of L1 regularization is subject to change for every iteration. Will read directly from "every" field in self.spec.

load(folder: str)

property new_w_l1: float

The new weight of L1 regularization. Effective when self.scheme is set to "step". When activated, w_l1 will be changed to self.new_w_l1. Will read directly from "new_w_l1" field under self.spec.

on_iter_begin(iter_idx: int, **kwargs)

Will be called to determine whether to attempt to update the weight of L1 regulation when an iteration begins.

Parameters

iter_idx (int) – The index of iteration, 0-based.

on_iter_end(iter_idx: int, **kwargs)

Not applicable to this class.

on_loop_begin(**kwargs)

Not applicable to this class.

on_loop_end(**kwargs)

Not applicable to this class.

on_step_begin(iter_idx: int, step_idx: int, **kwargs)

Not applicable to this class.

on_step_end(iter_idx: int, step_idx: int, **kwargs)

To be determined.

property rate: float

The rate of rise. Effective when self.scheme is set to "exp_rise". When activated, weight of L1 regularization will be changed by multiplying its value by exp(rate). Will read directly from "rate" field in self.spec.

save(folder: str)

property scheme: str

The scheme of how weight of L1 regularization will be changed. Currently can take "exp_rise"|"step". Will read directly from "scheme" field from self.spec.

property start: int

A positive number suggesting when to start to apply changes to weight of L1 regularization. When start < 1, it will be treated as a proportion, suggesting w_l1 will subject to change when iter_idx >= int(n_iters * start). Otherwise, iter_idx >= int(start) will be the condition.

exp_rise(w_l1: float, rate: float) → float

A scheduling function to calculate new weight of L1 regularization with exponential rise.

Parameters

• w_l1 (float) – Current weight of L1 regularization.

• rate (float) – The rate of rise. When activated, w_l1 will be changed by multiplying exp(rate).

Returns

Updated weight of L1 regularization.

Return type

float

step(w_l1: float, new_w_l1: float) → float

A scheduling function to update learning rate..

Parameters

• w_l1 (float) – Current weight of L1 regularization.

• new_w_l1 (float) – New weight of L1 regularization.

Returns

Updated weight of L1 regularization.

Return type

float

Module contents