"./vfs.test -test.v -test.timeout 1h0m0s -remote TestGoFile: -verbose -test.run '^TestFileRename$/^(minimal,forceCache=true|off,forceCache=false)$'" - Starting (try 2/5) 2024/10/29 03:56:00 DEBUG : Creating backend with remote "TestGoFile:rclone-test-bixazow8fise" 2024/10/29 03:56:00 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2024/10/29 03:56:01 DEBUG : Creating backend with remote "/tmp/rclone695606446" === RUN TestFileRename === RUN TestFileRename/off,forceCache=false run.go:180: Remote "gofile root 'rclone-test-bixazow8fise'", Local "Local file system at /tmp/rclone695606446", Modify Window "1s" 2024/10/29 03:56:01 INFO : gofile root 'rclone-test-bixazow8fise': poll-interval is not supported by this remote 2024/10/29 03:56:02 DEBUG : Upload server store10 (eu) responded in 132.959277ms 2024/10/29 03:56:02 DEBUG : Upload server store6 (eu) responded in 131.282892ms 2024/10/29 03:56:02 DEBUG : dir/file1: Using upload server store6 (eu) 2024/10/29 03:56:03 DEBUG : WaitForWriters: timeout=30s 2024/10/29 03:56:03 DEBUG : dir: Looking for writers 2024/10/29 03:56:03 DEBUG : file1: reading active writers 2024/10/29 03:56:03 DEBUG : : Looking for writers 2024/10/29 03:56:03 DEBUG : dir: reading active writers 2024/10/29 03:56:03 DEBUG : >WaitForWriters: 2024/10/29 03:56:03 INFO : dir/file1: Moved (server-side) to: newLeaf 2024/10/29 03:56:03 DEBUG : newLeaf: Updating file with newLeaf 0xc0002dea80 2024/10/29 03:56:03 DEBUG : dir: Added virtual directory entry vDel: "file1" 2024/10/29 03:56:03 DEBUG : : Added virtual directory entry vAddFile: "newLeaf" 2024/10/29 03:56:04 INFO : newLeaf: Moved (server-side) to: dir/file1 2024/10/29 03:56:04 DEBUG : dir/file1: Updating file with dir/file1 0xc0002dea80 2024/10/29 03:56:04 DEBUG : : Added virtual directory entry vDel: "newLeaf" 2024/10/29 03:56:04 DEBUG : dir: Added virtual directory entry vAddFile: "file1" 2024/10/29 03:56:04 DEBUG : dir/file1: Open: flags=O_WRONLY|O_TRUNC 2024/10/29 03:56:04 DEBUG : dir/file1: >Open: fd=dir/file1 (w), err= 2024/10/29 03:56:04 DEBUG : dir: Added virtual directory entry vAddFile: "file1" 2024/10/29 03:56:04 DEBUG : dir/file1: File is currently open, delaying rename 0xc0002dea80 2024/10/29 03:56:04 DEBUG : dir: Added virtual directory entry vDel: "file1" 2024/10/29 03:56:04 DEBUG : : Added virtual directory entry vAddFile: "newLeaf" 2024/10/29 03:56:04 DEBUG : gofile root 'rclone-test-bixazow8fise': File to upload is small (25 bytes), uploading instead of streaming 2024/10/29 03:56:04 DEBUG : dir/file1: Using upload server store10 (eu) 2024/10/29 03:56:05 DEBUG : dir/file1: Removing old object on successful upload 2024/10/29 03:56:05 DEBUG : dir/file1: md5 = c7247bead226dfd7bd970c17229f4044 OK 2024/10/29 03:56:05 DEBUG : dir/file1: Size and md5 of src and dst objects identical 2024/10/29 03:56:05 DEBUG : : Added virtual directory entry vAddFile: "newLeaf" 2024/10/29 03:56:05 DEBUG : newLeaf: Running delayed rename now 2024/10/29 03:56:05 INFO : dir/file1: Moved (server-side) to: newLeaf 2024/10/29 03:56:05 DEBUG : newLeaf: Updating file with newLeaf 0xc0002dea80 2024/10/29 03:56:05 DEBUG : WaitForWriters: timeout=30s 2024/10/29 03:56:05 DEBUG : dir: Looking for writers 2024/10/29 03:56:05 DEBUG : : Looking for writers 2024/10/29 03:56:05 DEBUG : newLeaf: reading active writers 2024/10/29 03:56:05 DEBUG : dir: reading active writers 2024/10/29 03:56:05 DEBUG : >WaitForWriters: 2024/10/29 03:56:05 DEBUG : WaitForWriters: timeout=30s 2024/10/29 03:56:05 DEBUG : dir: Looking for writers 2024/10/29 03:56:05 DEBUG : : Looking for writers 2024/10/29 03:56:05 DEBUG : dir: reading active writers 2024/10/29 03:56:05 DEBUG : newLeaf: reading active writers 2024/10/29 03:56:05 DEBUG : >WaitForWriters: === RUN TestFileRename/minimal,forceCache=true run.go:180: Remote "gofile root 'rclone-test-bixazow8fise'", Local "Local file system at /tmp/rclone695606446", Modify Window "1s" 2024/10/29 03:56:06 INFO : gofile root 'rclone-test-bixazow8fise': poll-interval is not supported by this remote 2024/10/29 03:56:06 DEBUG : vfs cache: root is "/home/rclone/.cache/rclone" 2024/10/29 03:56:06 DEBUG : vfs cache: data root is "/home/rclone/.cache/rclone/vfs/TestGoFile/rclone-test-bixazow8fise" 2024/10/29 03:56:06 DEBUG : vfs cache: metadata root is "/home/rclone/.cache/rclone/vfsMeta/TestGoFile/rclone-test-bixazow8fise" 2024/10/29 03:56:06 DEBUG : Creating backend with remote ":local,encoding='Slash,Dot':/home/rclone/.cache/rclone/vfs/TestGoFile/rclone-test-bixazow8fise" 2024/10/29 03:56:06 DEBUG : :local: detected overridden config - adding "{bxYPm}" suffix to name 2024/10/29 03:56:06 DEBUG : fs cache: renaming cache item ":local,encoding='Slash,Dot':/home/rclone/.cache/rclone/vfs/TestGoFile/rclone-test-bixazow8fise" to be canonical ":local{bxYPm}:/home/rclone/.cache/rclone/vfs/TestGoFile/rclone-test-bixazow8fise" 2024/10/29 03:56:06 DEBUG : Creating backend with remote ":local,encoding='Slash,Dot':/home/rclone/.cache/rclone/vfsMeta/TestGoFile/rclone-test-bixazow8fise" 2024/10/29 03:56:06 DEBUG : :local: detected overridden config - adding "{bxYPm}" suffix to name 2024/10/29 03:56:06 DEBUG : fs cache: renaming cache item ":local,encoding='Slash,Dot':/home/rclone/.cache/rclone/vfsMeta/TestGoFile/rclone-test-bixazow8fise" to be canonical ":local{bxYPm}:/home/rclone/.cache/rclone/vfsMeta/TestGoFile/rclone-test-bixazow8fise" 2024/10/29 03:56:06 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2024/10/29 03:56:07 DEBUG : dir/file1: Using upload server store10 (eu) 2024/10/29 03:56:07 DEBUG : dir/file1: Open: flags=O_RDWR|O_CREATE|O_TRUNC 2024/10/29 03:56:07 DEBUG : dir/file1: newRWFileHandle: 2024/10/29 03:56:07 DEBUG : dir/file1(0xc000b10b40): openPending: 2024/10/29 03:56:07 DEBUG : dir/file1: vfs cache: checking remote fingerprint "14,2001-02-03 04:05:06 +0000 UTC,0ef726ce9b1a7692357ff70dd321d595" against cached fingerprint "" 2024/10/29 03:56:07 DEBUG : dir/file1: vfs cache: truncate to size=14 2024/10/29 03:56:07 DEBUG : dir: Added virtual directory entry vAddFile: "file1" 2024/10/29 03:56:07 DEBUG : dir/file1(0xc000b10b40): >openPending: err= 2024/10/29 03:56:07 DEBUG : dir/file1: vfs cache: truncate to size=0 2024/10/29 03:56:07 DEBUG : dir/file1: >newRWFileHandle: err= 2024/10/29 03:56:07 DEBUG : dir: Added virtual directory entry vAddFile: "file1" 2024/10/29 03:56:07 DEBUG : dir/file1: >Open: fd=dir/file1 (rw), err= 2024/10/29 03:56:07 DEBUG : dir/file1(0xc000b10b40): _writeAt: size=14, off=0 2024/10/29 03:56:07 DEBUG : dir/file1(0xc000b10b40): >_writeAt: n=14, err= 2024/10/29 03:56:07 DEBUG : dir/file1: vfs cache: setting modification time to 2001-02-03 04:05:06.499999999 +0000 UTC 2024/10/29 03:56:07 DEBUG : dir/file1(0xc000b10b40): close: 2024/10/29 03:56:07 DEBUG : vfs cache: looking for range={Pos:0 Size:14} in [{Pos:0 Size:14}] - present true 2024/10/29 03:56:07 DEBUG : dir/file1: vfs cache: setting modification time to 2001-02-03 04:05:06.499999999 +0000 UTC 2024/10/29 03:56:07 INFO : dir/file1: vfs cache: queuing for upload in 100ms 2024/10/29 03:56:07 DEBUG : dir/file1(0xc000b10b40): >close: err= 2024/10/29 03:56:07 DEBUG : WaitForWriters: timeout=30s 2024/10/29 03:56:07 DEBUG : dir: Looking for writers 2024/10/29 03:56:07 DEBUG : file1: reading active writers 2024/10/29 03:56:07 DEBUG : : Looking for writers 2024/10/29 03:56:07 DEBUG : dir: reading active writers 2024/10/29 03:56:07 DEBUG : Still 0 writers active and 1 cache items in use, waiting 10ms 2024/10/29 03:56:07 DEBUG : dir: Looking for writers 2024/10/29 03:56:07 DEBUG : file1: reading active writers 2024/10/29 03:56:07 DEBUG : : Looking for writers 2024/10/29 03:56:07 DEBUG : dir: reading active writers 2024/10/29 03:56:07 DEBUG : Still 0 writers active and 1 cache items in use, waiting 20ms 2024/10/29 03:56:07 DEBUG : dir: Looking for writers 2024/10/29 03:56:07 DEBUG : file1: reading active writers 2024/10/29 03:56:07 DEBUG : : Looking for writers 2024/10/29 03:56:07 DEBUG : dir: reading active writers 2024/10/29 03:56:07 DEBUG : Still 0 writers active and 1 cache items in use, waiting 40ms 2024/10/29 03:56:07 DEBUG : dir: Looking for writers 2024/10/29 03:56:07 DEBUG : file1: reading active writers 2024/10/29 03:56:07 DEBUG : : Looking for writers 2024/10/29 03:56:07 DEBUG : dir: reading active writers 2024/10/29 03:56:07 DEBUG : Still 0 writers active and 1 cache items in use, waiting 80ms 2024/10/29 03:56:07 DEBUG : dir/file1: vfs cache: starting upload 2024/10/29 03:56:07 DEBUG : dir/file1: Using upload server store10 (eu) 2024/10/29 03:56:07 DEBUG : dir: Looking for writers 2024/10/29 03:56:07 DEBUG : file1: reading active writers 2024/10/29 03:56:07 DEBUG : : Looking for writers 2024/10/29 03:56:07 DEBUG : dir: reading active writers 2024/10/29 03:56:07 DEBUG : Still 0 writers active and 1 cache items in use, waiting 160ms 2024/10/29 03:56:07 DEBUG : dir/file1: Removing old object on successful upload 2024/10/29 03:56:08 DEBUG : dir: Looking for writers 2024/10/29 03:56:08 DEBUG : file1: reading active writers 2024/10/29 03:56:08 DEBUG : : Looking for writers 2024/10/29 03:56:08 DEBUG : dir: reading active writers 2024/10/29 03:56:08 DEBUG : Still 0 writers active and 1 cache items in use, waiting 320ms 2024/10/29 03:56:08 DEBUG : dir/file1: md5 = 0ef726ce9b1a7692357ff70dd321d595 OK 2024/10/29 03:56:08 INFO : dir/file1: Copied (replaced existing) 2024/10/29 03:56:08 DEBUG : dir/file1: vfs cache: writeback object to VFS layer 2024/10/29 03:56:08 DEBUG : dir/file1: Not setting pending mod time 2001-02-03 04:05:06.499999999 +0000 UTC as it is already set 2024/10/29 03:56:08 DEBUG : dir: Added virtual directory entry vAddFile: "file1" 2024/10/29 03:56:08 INFO : dir/file1: vfs cache: upload succeeded try #1 2024/10/29 03:56:08 DEBUG : dir: Looking for writers 2024/10/29 03:56:08 DEBUG : file1: reading active writers 2024/10/29 03:56:08 DEBUG : : Looking for writers 2024/10/29 03:56:08 DEBUG : dir: reading active writers 2024/10/29 03:56:08 DEBUG : >WaitForWriters: 2024/10/29 03:56:08 DEBUG : dir/file1: Open: flags=O_RDONLY 2024/10/29 03:56:08 DEBUG : dir/file1: newRWFileHandle: 2024/10/29 03:56:08 DEBUG : dir/file1: >newRWFileHandle: err= 2024/10/29 03:56:08 DEBUG : dir/file1: >Open: fd=dir/file1 (rw), err= 2024/10/29 03:56:08 DEBUG : dir/file1(0xc00090c0c0): _readAt: size=512, off=0 2024/10/29 03:56:08 DEBUG : dir/file1(0xc00090c0c0): openPending: 2024/10/29 03:56:08 DEBUG : dir/file1: vfs cache: checking remote fingerprint "14,2001-02-03 04:05:06 +0000 UTC,0ef726ce9b1a7692357ff70dd321d595" against cached fingerprint "14,2001-02-03 04:05:06 +0000 UTC,0ef726ce9b1a7692357ff70dd321d595" 2024/10/29 03:56:08 DEBUG : dir/file1: vfs cache: truncate to size=14 (not needed as size correct) 2024/10/29 03:56:08 DEBUG : dir: Added virtual directory entry vAddFile: "file1" 2024/10/29 03:56:08 DEBUG : dir/file1(0xc00090c0c0): >openPending: err= 2024/10/29 03:56:08 DEBUG : vfs cache: looking for range={Pos:0 Size:14} in [{Pos:0 Size:14}] - present true 2024/10/29 03:56:08 DEBUG : dir/file1(0xc00090c0c0): >_readAt: n=14, err=EOF 2024/10/29 03:56:08 DEBUG : dir/file1(0xc00090c0c0): close: 2024/10/29 03:56:08 DEBUG : dir/file1: vfs cache: setting modification time to 2001-02-03 04:05:06 +0000 UTC 2024/10/29 03:56:08 DEBUG : dir/file1(0xc00090c0c0): >close: err= 2024/10/29 03:56:09 DEBUG : Rate limited, sleep for 5s 2024/10/29 03:56:14 DEBUG : pacer: low level retry 1/10 (error Error "error-rateLimit") 2024/10/29 03:56:14 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2024/10/29 03:56:14 DEBUG : Rate limited, sleep for 5s 2024/10/29 03:56:19 DEBUG : pacer: low level retry 2/10 (error Error "error-rateLimit") 2024/10/29 03:56:19 DEBUG : pacer: Rate limited, increasing sleep to 40ms 2024/10/29 03:56:19 DEBUG : pacer: Reducing sleep to 20ms 2024/10/29 03:56:19 INFO : dir/file1: Moved (server-side) to: newLeaf 2024/10/29 03:56:19 INFO : dir/file1: vfs cache: renamed in cache to "newLeaf" 2024/10/29 03:56:19 DEBUG : newLeaf: Updating file with newLeaf 0xc0002defc0 2024/10/29 03:56:19 DEBUG : dir: Added virtual directory entry vDel: "file1" 2024/10/29 03:56:19 DEBUG : : Added virtual directory entry vAddFile: "newLeaf" 2024/10/29 03:56:19 DEBUG : pacer: Reducing sleep to 10ms 2024/10/29 03:56:19 DEBUG : Rate limited, sleep for 5s 2024/10/29 03:56:24 DEBUG : pacer: low level retry 1/10 (error Error "error-rateLimit") 2024/10/29 03:56:24 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2024/10/29 03:56:24 DEBUG : Rate limited, sleep for 5s 2024/10/29 03:56:29 DEBUG : pacer: low level retry 2/10 (error Error "error-rateLimit") 2024/10/29 03:56:29 DEBUG : pacer: Rate limited, increasing sleep to 40ms 2024/10/29 03:56:29 DEBUG : Rate limited, sleep for 5s 2024/10/29 03:56:34 DEBUG : pacer: low level retry 3/10 (error Error "error-rateLimit") 2024/10/29 03:56:34 DEBUG : pacer: Rate limited, increasing sleep to 80ms 2024/10/29 03:56:34 DEBUG : Rate limited, sleep for 5s 2024/10/29 03:56:39 DEBUG : pacer: low level retry 4/10 (error Error "error-rateLimit") 2024/10/29 03:56:39 DEBUG : pacer: Rate limited, increasing sleep to 160ms 2024/10/29 03:56:39 DEBUG : Rate limited, sleep for 5s 2024/10/29 03:56:44 DEBUG : pacer: low level retry 5/10 (error Error "error-rateLimit") 2024/10/29 03:56:44 DEBUG : pacer: Rate limited, increasing sleep to 320ms 2024/10/29 03:56:44 DEBUG : Rate limited, sleep for 5s 2024/10/29 03:56:49 DEBUG : pacer: low level retry 6/10 (error Error "error-rateLimit") 2024/10/29 03:56:49 DEBUG : pacer: Rate limited, increasing sleep to 640ms 2024/10/29 03:56:49 DEBUG : Rate limited, sleep for 5s 2024/10/29 03:56:54 DEBUG : pacer: low level retry 7/10 (error Error "error-rateLimit") 2024/10/29 03:56:54 DEBUG : pacer: Rate limited, increasing sleep to 1.28s 2024/10/29 03:56:55 DEBUG : Rate limited, sleep for 5s 2024/10/29 03:57:00 DEBUG : pacer: low level retry 8/10 (error Error "error-rateLimit") 2024/10/29 03:57:00 DEBUG : pacer: Rate limited, increasing sleep to 2.56s 2024/10/29 03:57:00 DEBUG : Rate limited, sleep for 5s 2024/10/29 03:57:05 DEBUG : pacer: low level retry 9/10 (error Error "error-rateLimit") 2024/10/29 03:57:05 DEBUG : pacer: Rate limited, increasing sleep to 5.12s 2024/10/29 03:57:05 DEBUG : Rate limited, sleep for 5s 2024/10/29 03:57:06 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item newLeaf not removed, freed 0 bytes 2024/10/29 03:57:06 INFO : vfs cache: cleaned: objects 1 (was 1) in use 0, to upload 0, uploading 0, total size 14 (was 14) 2024/10/29 03:57:10 DEBUG : pacer: low level retry 10/10 (error Error "error-rateLimit") 2024/10/29 03:57:10 DEBUG : pacer: Rate limited, increasing sleep to 10.24s 2024/10/29 03:57:10 ERROR : newLeaf: Couldn't move: failed to move item: Error "error-rateLimit" 2024/10/29 03:57:10 ERROR : dir/file1: File.Rename error: failed to move item: Error "error-rateLimit" 2024/10/29 03:57:10 ERROR : newLeaf: Dir.Rename error: failed to move item: Error "error-rateLimit" file_test.go:358: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/vfs/file_test.go:358 /home/rclone/go/src/github.com/rclone/rclone/vfs/file_test.go:413 Error: Received unexpected error: failed to move item: Error "error-rateLimit" Test: TestFileRename/minimal,forceCache=true 2024/10/29 03:57:10 DEBUG : WaitForWriters: timeout=30s 2024/10/29 03:57:10 DEBUG : dir: Looking for writers 2024/10/29 03:57:10 DEBUG : : Looking for writers 2024/10/29 03:57:10 DEBUG : newLeaf: reading active writers 2024/10/29 03:57:10 DEBUG : dir: reading active writers 2024/10/29 03:57:10 DEBUG : >WaitForWriters: 2024/10/29 03:57:10 DEBUG : vfs cache: cleaner exiting 2024/10/29 03:57:10 DEBUG : pacer: Reducing sleep to 5.12s 2024/10/29 03:57:20 DEBUG : pacer: Reducing sleep to 2.56s 2024/10/29 03:57:25 DEBUG : pacer: Reducing sleep to 1.28s 2024/10/29 03:57:28 DEBUG : pacer: Reducing sleep to 640ms 2024/10/29 03:57:29 DEBUG : pacer: Reducing sleep to 320ms --- FAIL: TestFileRename (88.47s) --- PASS: TestFileRename/off,forceCache=false (5.79s) --- FAIL: TestFileRename/minimal,forceCache=true (82.69s) FAIL 2024/10/29 03:57:29 DEBUG : gofile root 'rclone-test-bixazow8fise': Purge remote 2024/10/29 03:57:30 DEBUG : pacer: Reducing sleep to 160ms "./vfs.test -test.v -test.timeout 1h0m0s -remote TestGoFile: -verbose -test.run '^TestFileRename$/^(minimal,forceCache=true|off,forceCache=false)$'" - Finished ERROR in 1m29.789318741s (try 2/5): exit status 1: Failed [TestFileRename/minimal,forceCache=true]