"./operations.test -test.v -test.timeout 2h0m0s -remote TestInternxt: -list-retries 5 -verbose -test.run '^(TestMaxDelete|TestMoveFileBackupDir|TestPurge|TestRmdirsLeaveRoot|TestRmdirsNoLeaveRoot|TestRmdirsWithFilter)$|^TestMultithreadCopy$/^(upload=true,size=62914560,streams=2|upload=true,size=62914561,streams=2)$'" - Starting (try 2/5) 2026/08/19 02:12:56 DEBUG : Creating backend with remote "TestInternxt:rclone-test-laduxiw0juno" 2026/08/19 02:12:56 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2026/08/19 02:12:56 DEBUG : User info: rootFolderId=2661cf64-e104-478b-b737-48c3ac8984d2, bucket=697358bf16ceabcf39af85f2 2026/08/19 02:12:56 DEBUG : Creating backend with remote "/tmp/rclone2875482400" === RUN TestMultithreadCopy run.go:198: Remote "Internxt root 'rclone-test-laduxiw0juno'", Local "Local file system at /tmp/rclone2875482400", Modify Window "876000h0m0s" FileLimitsResponse { "maxUploadFileSize": 107374182400, "versioning": { "enabled": true, "maxFileSize": 20971520, "retentionDays": 30, "maxVersions": 20 } } 2026/08/19 02:13:08 NOTICE: Internxt root 'rclone-test-laduxiw0juno': Multipart upload aborted for chunksize-probe === RUN TestMultithreadCopy/upload=true,size=62914560,streams=2 2026/08/19 02:13:14 DEBUG : test-multithread-copy-true-62914560-2: multi-thread copy: disabling buffering because source is local disk multithread_test.go:225: file too small for multipart upload: multi-thread copy: failed to open chunk writer: file too small for multipart upload (minimum 104857600 bytes) === RUN TestMultithreadCopy/upload=true,size=62914561,streams=2 fstest.go:298: Sleeping for 1s for list eventual consistency: 1/5 fstest.go:298: Sleeping for 2s for list eventual consistency: 2/5 fstest.go:298: Sleeping for 4s for list eventual consistency: 3/5 fstest.go:298: Sleeping for 8s for list eventual consistency: 4/5 fstest.go:298: Sleeping for 16s for list eventual consistency: 5/5 fstest.go:305: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:305 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:352 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/multithread_test.go:193 Error: Should be true Test: TestMultithreadCopy/upload=true,size=62914561,streams=2 Messages: listing wrong, want test-multithread-copy-true-62914561-2 (62914561) got test-multithread-copy-true-62914560-2 (62914560), test-multithread-copy-true-62914561-2 (62914561) fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:352 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/multithread_test.go:193 /usr/local/go/src/runtime/asm_amd64.s:1771 Error: Should be true Test: TestMultithreadCopy/upload=true,size=62914561,streams=2 Messages: Unexpected file "test-multithread-copy-true-62914560-2" 2026/08/19 02:13:52 DEBUG : test-multithread-copy-true-62914561-2: multi-thread copy: disabling buffering because source is local disk multithread_test.go:225: file too small for multipart upload: multi-thread copy: failed to open chunk writer: file too small for multipart upload (minimum 104857600 bytes) --- FAIL: TestMultithreadCopy (58.57s) --- SKIP: TestMultithreadCopy/upload=true,size=62914560,streams=2 (6.34s) --- FAIL: TestMultithreadCopy/upload=true,size=62914561,streams=2 (37.45s) === RUN TestMaxDelete run.go:198: Remote "Internxt root 'rclone-test-laduxiw0juno'", Local "Local file system at /tmp/rclone2875482400", Modify Window "876000h0m0s" 2026/08/19 02:14:00 DEBUG : Waiting for deletions to finish 2026/08/19 02:14:02 ERROR : large: Got fatal error on delete: --max-delete threshold reached 2026/08/19 02:14:02 INFO : medium: Deleted 2026/08/19 02:14:02 INFO : small: Deleted fstest.go:298: Sleeping for 1s for list eventual consistency: 1/5 fstest.go:301: Flushing the directory cache fstest.go:292: Sleeping for 6s just to make sure --- PASS: TestMaxDelete (17.19s) === RUN TestPurge 2026/08/19 02:14:12 DEBUG : Creating backend with remote "TestInternxt:rclone-test-kibuqoh9gefa" 2026/08/19 02:14:12 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2026/08/19 02:14:12 DEBUG : User info: rootFolderId=2661cf64-e104-478b-b737-48c3ac8984d2, bucket=697358bf16ceabcf39af85f2 2026/08/19 02:14:13 DEBUG : Creating backend with remote "/tmp/rclone1343446347" run.go:198: Remote "Internxt root 'rclone-test-kibuqoh9gefa'", Local "Local file system at /tmp/rclone1343446347", Modify Window "876000h0m0s" FileLimitsResponse { "maxUploadFileSize": 107374182400, "versioning": { "enabled": true, "maxFileSize": 20971520, "retentionDays": 30, "maxVersions": 20 } } 2026/08/19 02:14:26 INFO : A2: Making directory 2026/08/19 02:14:26 INFO : A1/B2: Making directory 2026/08/19 02:14:26 INFO : A1/B2/C2: Making directory 2026/08/19 02:14:27 INFO : A1/B1/C3: Making directory 2026/08/19 02:14:27 INFO : A3: Making directory 2026/08/19 02:14:28 INFO : A3/B3: Making directory 2026/08/19 02:14:28 INFO : A3/B3/C4: Making directory 2026/08/19 02:14:32 DEBUG : Waiting for deletions to finish 2026/08/19 02:14:32 INFO : A1/B1/C1/one: Deleted 2026/08/19 02:14:33 DEBUG : removing 2 level 3 directories 2026/08/19 02:14:33 INFO : A1/B1/C3: Removing directory 2026/08/19 02:14:33 INFO : A1/B1/C1: Removing directory 2026/08/19 02:14:34 DEBUG : removing 1 level 2 directories 2026/08/19 02:14:34 INFO : A1/B1: Removing directory 2026/08/19 02:14:35 DEBUG : Waiting for deletions to finish 2026/08/19 02:14:45 INFO : A1/two: Deleted 2026/08/19 02:14:48 DEBUG : removing 2 level 3 directories 2026/08/19 02:14:48 INFO : A3/B3/C4: Removing directory 2026/08/19 02:14:48 INFO : A1/B2/C2: Removing directory 2026/08/19 02:14:49 DEBUG : removing 2 level 2 directories 2026/08/19 02:14:49 INFO : A3/B3: Removing directory 2026/08/19 02:14:49 INFO : A1/B2: Removing directory 2026/08/19 02:14:49 ERROR : A3/B3: Failed to rmdir: directory not empty 2026/08/19 02:14:49 ERROR : A1/B2: Failed to rmdir: directory not empty 2026/08/19 02:14:49 DEBUG : removing 3 level 1 directories 2026/08/19 02:14:49 INFO : A3: Removing directory 2026/08/19 02:14:49 INFO : A1: Removing directory 2026/08/19 02:14:49 INFO : A2: Removing directory 2026/08/19 02:14:49 ERROR : A3: Failed to rmdir: directory not empty 2026/08/19 02:14:49 ERROR : A1: Failed to rmdir: directory not empty 2026/08/19 02:14:49 DEBUG : removing 1 level 0 directories 2026/08/19 02:14:49 INFO : Internxt root 'rclone-test-kibuqoh9gefa': Removing directory 2026/08/19 02:14:49 ERROR : Failed to rmdir: directory not empty operations_test.go:632: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/operations_test.go:632 Error: Received unexpected error: failed to remove directories: 5 errors: last error: directory not empty Test: TestPurge 2026/08/19 02:14:55 DEBUG : Internxt root 'rclone-test-kibuqoh9gefa': Purge dir "A3/B3" 2026/08/19 02:14:56 DEBUG : Internxt root 'rclone-test-kibuqoh9gefa': Purge dir "A3" 2026/08/19 02:14:56 NOTICE: purge failed to rmdir "A3": directory not empty 2026/08/19 02:14:56 DEBUG : Internxt root 'rclone-test-kibuqoh9gefa': Purge dir "A1/B2" 2026/08/19 02:14:57 DEBUG : Internxt root 'rclone-test-kibuqoh9gefa': Purge dir "A1" 2026/08/19 02:14:57 NOTICE: purge failed to rmdir "A1": directory not empty 2026/08/19 02:14:57 DEBUG : Internxt root 'rclone-test-kibuqoh9gefa': Purge dir "" 2026/08/19 02:14:57 NOTICE: purge failed to rmdir "": directory not empty --- FAIL: TestPurge (44.64s) === RUN TestRmdirsNoLeaveRoot run.go:198: Remote "Internxt root 'rclone-test-laduxiw0juno'", Local "Local file system at /tmp/rclone2875482400", Modify Window "876000h0m0s" 2026/08/19 02:15:00 INFO : A2: Making directory 2026/08/19 02:15:01 INFO : A1/B2: Making directory 2026/08/19 02:15:01 INFO : A1/B2/C2: Making directory 2026/08/19 02:15:02 INFO : A1/B1/C3: Making directory 2026/08/19 02:15:02 INFO : A3: Making directory 2026/08/19 02:15:02 INFO : A3/B3: Making directory 2026/08/19 02:15:03 INFO : A3/B3/C4: Making directory 2026/08/19 02:15:11 DEBUG : pacer: low level retry 1/10 (error failed to list all folders at offset 0: list folders: error code: 502 (status 502)) 2026/08/19 02:15:11 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2026/08/19 02:15:11 DEBUG : pacer: low level retry 1/10 (error failed to list all folders at offset 0: list folders: error code: 502 (status 502)) 2026/08/19 02:15:11 DEBUG : pacer: Rate limited, increasing sleep to 40ms 2026/08/19 02:15:11 DEBUG : pacer: Reducing sleep to 30ms 2026/08/19 02:15:11 DEBUG : pacer: Reducing sleep to 22.5ms 2026/08/19 02:15:11 DEBUG : pacer: Reducing sleep to 16.875ms 2026/08/19 02:15:11 DEBUG : pacer: Reducing sleep to 12.65625ms 2026/08/19 02:15:12 DEBUG : pacer: Reducing sleep to 10ms 2026/08/19 02:15:12 DEBUG : removing 1 level 3 directories 2026/08/19 02:15:12 INFO : A3/B3/C4: Removing directory 2026/08/19 02:15:21 DEBUG : removing 2 level 3 directories 2026/08/19 02:15:21 INFO : A1/B2/C2: Removing directory 2026/08/19 02:15:21 INFO : A1/B1/C3: Removing directory 2026/08/19 02:15:21 DEBUG : removing 2 level 2 directories 2026/08/19 02:15:21 INFO : A3/B3: Removing directory 2026/08/19 02:15:21 INFO : A1/B2: Removing directory 2026/08/19 02:15:22 DEBUG : removing 2 level 1 directories 2026/08/19 02:15:22 INFO : A3: Removing directory 2026/08/19 02:15:22 INFO : A2: Removing directory 2026/08/19 02:15:22 ERROR : A3: error listing: directory not found 2026/08/19 02:15:22 ERROR : A2: error listing: directory not found fstest.go:298: Sleeping for 1s for list eventual consistency: 1/5 fstest.go:301: Flushing the directory cache fstest.go:292: Sleeping for 6s just to make sure 2026/08/19 02:15:34 DEBUG : removing 1 level 3 directories 2026/08/19 02:15:34 INFO : A1/B1/C1: Removing directory 2026/08/19 02:15:35 DEBUG : removing 1 level 2 directories 2026/08/19 02:15:35 INFO : A1/B1: Removing directory 2026/08/19 02:15:35 ERROR : A1/B1: Failed to rmdir: directory not empty 2026/08/19 02:15:35 DEBUG : removing 1 level 1 directories 2026/08/19 02:15:35 INFO : A1: Removing directory 2026/08/19 02:15:35 ERROR : A1: Failed to rmdir: directory not empty 2026/08/19 02:15:35 DEBUG : removing 1 level 0 directories 2026/08/19 02:15:35 INFO : Internxt root 'rclone-test-laduxiw0juno': Removing directory 2026/08/19 02:15:36 ERROR : Failed to rmdir: directory not empty operations_test.go:729: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/operations_test.go:729 Error: Received unexpected error: failed to remove directories: 3 errors: last error: directory not empty Test: TestRmdirsNoLeaveRoot run.go:186: removing dir "A1" failed - try 1/5: sleeping 1s: directory not empty 2026/08/19 02:15:40 DEBUG : pacer: low level retry 1/10 (error failed to list all files at offset 0: list files: error code: 502 (status 502)) 2026/08/19 02:15:40 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2026/08/19 02:15:40 DEBUG : pacer: Reducing sleep to 15ms --- FAIL: TestRmdirsNoLeaveRoot (43.31s) === RUN TestRmdirsLeaveRoot run.go:198: Remote "Internxt root 'rclone-test-laduxiw0juno'", Local "Local file system at /tmp/rclone2875482400", Modify Window "876000h0m0s" 2026/08/19 02:15:40 INFO : A1: Making directory 2026/08/19 02:15:40 DEBUG : pacer: Reducing sleep to 11.25ms 2026/08/19 02:15:41 DEBUG : pacer: Reducing sleep to 10ms 2026/08/19 02:15:41 INFO : A1/B1: Making directory 2026/08/19 02:15:41 INFO : A1/B1/C1: Making directory 2026/08/19 02:15:45 DEBUG : removing 1 level 3 directories 2026/08/19 02:15:45 INFO : A1/B1/C1: Removing directory 2026/08/19 02:15:46 DEBUG : removing 1 level 2 directories 2026/08/19 02:15:46 INFO : A1/B1: Removing directory 2026/08/19 02:15:46 ERROR : A1/B1: Failed to rmdir: directory not empty operations_test.go:763: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/operations_test.go:763 Error: Received unexpected error: failed to remove directories: directory not empty Test: TestRmdirsLeaveRoot run.go:186: removing dir "A1" failed - try 1/5: sleeping 1s: directory not empty --- FAIL: TestRmdirsLeaveRoot (12.21s) === RUN TestRmdirsWithFilter run.go:198: Remote "Internxt root 'rclone-test-laduxiw0juno'", Local "Local file system at /tmp/rclone2875482400", Modify Window "876000h0m0s" 2026/08/19 02:15:52 INFO : A1: Making directory 2026/08/19 02:15:53 INFO : A1/B1: Making directory 2026/08/19 02:15:53 INFO : A1/B1/C1: Making directory 2026/08/19 02:15:56 DEBUG : removing 1 level 3 directories 2026/08/19 02:15:56 INFO : A1/B1/C1: Removing directory 2026/08/19 02:15:56 DEBUG : removing 1 level 2 directories 2026/08/19 02:15:56 INFO : A1/B1: Removing directory 2026/08/19 02:15:57 ERROR : A1/B1: Failed to rmdir: directory not empty operations_test.go:802: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/operations_test.go:802 Error: Received unexpected error: failed to remove directories: directory not empty Test: TestRmdirsWithFilter --- FAIL: TestRmdirsWithFilter (5.80s) === RUN TestMoveFileBackupDir run.go:198: Remote "Internxt root 'rclone-test-laduxiw0juno'", Local "Local file system at /tmp/rclone2875482400", Modify Window "876000h0m0s" 2026/08/19 02:16:01 DEBUG : Creating backend with remote "TestInternxt:rclone-test-laduxiw0juno/backup" 2026/08/19 02:16:01 DEBUG : User info: rootFolderId=2661cf64-e104-478b-b737-48c3ac8984d2, bucket=697358bf16ceabcf39af85f2 2026/08/19 02:16:01 DEBUG : dst/file1: size = 14 (Local file system at /tmp/rclone2875482400) 2026/08/19 02:16:01 DEBUG : dst/file1: size = 18 (Internxt root 'rclone-test-laduxiw0juno') 2026/08/19 02:16:01 DEBUG : dst/file1: Sizes differ 2026/08/19 02:16:04 INFO : dst/file1: Moved (server-side) 2026/08/19 02:16:05 DEBUG : Internxt root 'rclone-test-laduxiw0juno': Renamed existing file dst/file1 to backup file1.rclone-backup-dogudeg9. (UUID: d3467604-6dff-42f4-aaed-d8c018a9702f) 2026/08/19 02:16:05 DEBUG : Internxt root 'rclone-test-laduxiw0juno': Upload succeeded, deleting backup file file1.rclone-backup-dogudeg9. (UUID: d3467604-6dff-42f4-aaed-d8c018a9702f) 2026/08/19 02:16:06 DEBUG : Internxt root 'rclone-test-laduxiw0juno': Successfully deleted backup file 2026/08/19 02:16:06 DEBUG : dst/file1: size = 14 OK 2026/08/19 02:16:06 INFO : dst/file1: Copied (new) 2026/08/19 02:16:06 INFO : dst/file1: Deleted fstest.go:298: Sleeping for 1s for list eventual consistency: 1/5 fstest.go:301: Flushing the directory cache fstest.go:298: Sleeping for 2s for list eventual consistency: 2/5 fstest.go:301: Flushing the directory cache fstest.go:298: Sleeping for 4s for list eventual consistency: 3/5 fstest.go:301: Flushing the directory cache fstest.go:298: Sleeping for 8s for list eventual consistency: 4/5 fstest.go:301: Flushing the directory cache fstest.go:298: Sleeping for 16s for list eventual consistency: 5/5 fstest.go:301: Flushing the directory cache fstest.go:305: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:305 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:358 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/operations_test.go:1079 Error: Should be true Test: TestMoveFileBackupDir Messages: listing wrong, want backup/dst/file1 (18), dst/file1 (14) got dst/file1 (14) fstest.go:203: Not found "backup/dst/file1" fstest.go:206: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:206 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:310 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:358 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/operations_test.go:1079 /usr/local/go/src/runtime/asm_amd64.s:1771 Error: Not equal: expected: 0 actual : 1 Test: TestMoveFileBackupDir Messages: 1 objects not found 2026/08/19 02:16:50 ERROR : backup: error listing: directory not found fstest.go:298: Sleeping for 1s for list eventual consistency: 1/5 fstest.go:301: Flushing the directory cache fstest.go:292: Sleeping for 6s just to make sure --- FAIL: TestMoveFileBackupDir (59.04s) FAIL 2026/08/19 02:16:57 DEBUG : Internxt root 'rclone-test-laduxiw0juno': Purge dir "" "./operations.test -test.v -test.timeout 2h0m0s -remote TestInternxt: -list-retries 5 -verbose -test.run '^(TestMaxDelete|TestMoveFileBackupDir|TestPurge|TestRmdirsLeaveRoot|TestRmdirsNoLeaveRoot|TestRmdirsWithFilter)$|^TestMultithreadCopy$/^(upload=true,size=62914560,streams=2|upload=true,size=62914561,streams=2)$'" - Finished ERROR in 4m2.050113154s (try 2/5): exit status 1: Failed [TestMultithreadCopy/upload=true,size=62914561,streams=2 TestPurge TestRmdirsNoLeaveRoot TestRmdirsLeaveRoot TestRmdirsWithFilter TestMoveFileBackupDir]