Can I add time series specific features to NN configurations, like with ML models, or do NNs mostly depend on their own architecture and hyperparameters to learn those patterns?

In theory they don’t rely on it. But I think from a practical perspective it makes sense to add some smart features. There is also research h where people do target transformation for NN for ts. And it always depends on your data. If it works, it works :)