Does "using checkpointing" tell me anything useful about why generation is stuck?
Trying a local image-to-video workflow on a 3060 12 GB. The last messages are a new image size of (768, 1024), then a pile of "VideoTransformerBlock is using checkpointing" lines. No traceback. GPU memory sits at about 11.6 GB and the progress bar hasn't moved in nine minutes.
I understand checkpointing trades extra computation for lower memory use. What I don't understand is whether these lines mean it's actually doing work or just announcing how the model was configured.
What's the first thing you'd check to distinguish very slow inference from a stalled run? I'd rather collect something useful before killing it and changing six settings at once.
0
Does "using checkpointing" tell me anything useful about why generation is stuck?
Trying a local image-to-video workflow on a 3060 12 GB. The last messages are a new image size of (768, 1024), then a pile of "VideoTransformerBlock is using checkpointing" lines. No traceback. GPU memory sits at about 11.6 GB and the progress bar hasn't moved in nine minutes.
I understand checkpointing trades extra computation for lower memory use. What I don't understand is whether these lines mean it's actually doing work or just announcing how the model was configured.
What's the first thing you'd check to distinguish very slow inference from a stalled run? I'd rather collect something useful before killing it and changing six settings at once.