im trying to write a custom HuberLoss function but...
# neural-forecast
n
im trying to write a custom HuberLoss function but Im having trouble following this guide due to the fact that HuberLoss references multiple other pieces of source code. is there a more convenient way to do this other than editing the source code
k
Hey @nickeleres, You can code your own loss function, but you need to keep the input output contracts of the NeuralForecast library as well as provide the necessary methods that all the pytorch networks use the BasePointLoss has those methods, Here is the code that you can modify to have your own loss function:
Copy code
class BasePointLoss(torch.nn.Module):
    def __init__(self, horizon_weight, outputsize_multiplier, output_names):
        super(BasePointLoss, self).__init__()
        if horizon_weight is not None:
            horizon_weight = torch.Tensor(horizon_weight.flatten())
        self.horizon_weight = horizon_weight
        self.outputsize_multiplier = outputsize_multiplier
        self.output_names = output_names
        self.is_distribution_output = False

    def domain_map(self, y_hat: torch.Tensor):
        """
        Univariate loss operates in dimension [B,T,H]/[B,H]
        This changes the network's output from [B,H,1]->[B,H]
        """
        return y_hat.squeeze(-1)

    def _compute_weights(self, y, mask):
        """
        Compute final weights for each datapoint (based on all weights and all masks)
        Set horizon_weight to a ones[H] tensor if not set.
        If set, check that it has the same length as the horizon in x.
        """
        if mask is None:
            mask = torch.ones_like(y, device=y.device)

        if self.horizon_weight is None:
            self.horizon_weight = torch.ones(mask.shape[-1])
        else:
            assert mask.shape[-1] == len(
                self.horizon_weight
            ), "horizon_weight must have same length as Y"

        weights = self.horizon_weight.clone()
        weights = torch.ones_like(mask, device=mask.device) * <http://weights.to|weights.to>(mask.device)
        return weights * mask

class HuberLoss(BasePointLoss):
    def __init__(self, delta: float = 1.0, horizon_weight=None):
        super(HuberLoss, self).__init__(
            horizon_weight=horizon_weight, outputsize_multiplier=1, output_names=[""]
        )
        self.delta = delta

    def __call__(
        self,
        y: torch.Tensor,
        y_hat: torch.Tensor,
        mask: Union[torch.Tensor, None] = None,
    ):
        """
        **Parameters:**<br>
        `y`: tensor, Actual values.<br>
        `y_hat`: tensor, Predicted values.<br>
        `mask`: tensor, Specifies date stamps per serie to consider in loss.<br>

        **Returns:**<br>
        `huber_loss`: tensor (single value).
        """
        losses = F.huber_loss(y, y_hat, reduction="none", delta=self.delta)
        weights = self._compute_weights(y=y, mask=mask)
        return _weighted_mean(losses=losses, weights=weights)
n
hey thanks. i tried that but there are objects
F
and
_weighted_mean
which are out of scope
how do i work around that @Kin Gtz. Olivares?
Copy code
NameError: name 'F' is not defined
k
Copy code
import torch.nn.functional as F

def _divide_no_nan(a: torch.Tensor, b: torch.Tensor) -> torch.Tensor:
    """
    Auxiliary funtion to handle divide by 0
    """
    div = a / b
    div[div != div] = 0.0
    div[div == float("inf")] = 0.0
    return div

# %% ../../nbs/losses.pytorch.ipynb 7
def _weighted_mean(losses, weights):
    """
    Compute weighted mean of losses per datapoint.
    """
    return _divide_no_nan(torch.sum(losses * weights), torch.sum(weights))
Check the imports ^