Repository navigation
Frequent Hang Issue in test_vnet and test_ahnet during daily tests #8581
Description
[2025-09-20T21:06:34.990Z] Starting test: test_ahnet_shape_2d_0 (tests.networks.nets.test_ahnet.TestAHNET)...
[2025-09-20T22:20:44.737Z] Sending interrupt signal to process
[2025-09-20T22:20:44.737Z] Killing processes
[2025-09-20T22:20:45.018Z] kill finished with exit code 0
[2025-09-20T22:20:58.768Z] script returned exit code 143
[2025-09-19T20:39:29.985Z] Starting test: test_vnet_shape_0 (tests.networks.nets.test_vnet.TestVNet)...
[2025-09-19T22:20:43.981Z] Sending interrupt signal to process
[2025-09-19T22:20:43.988Z] Killing processes
[2025-09-19T22:20:44.300Z] kill finished with exit code 0
[2025-09-19T22:20:54.367Z] script returned exit code 143
Activity
KumoLiu commentedon Sep 22, 2025
I found the issue only happen when pytorch==2.8.0.
I fixed the pytorch version to 2.7.1 in blossom pip pipeline.
ericspod commentedon Apr 19, 2026
#8561 will want to test with PyTorch 2.8.0. The tests there seem to be completing successfully so it's possible this issue is resolved.
ericspod commentedon May 15, 2026
I'm going to close this as resolved now with other changes that have been made since raising.