"./vfs.test -test.v -test.timeout 1h0m0s -remote TestChunkerChunk50bMD5HashS3: -verbose -size-limit 1024 -test.run '^(TestRWFileHandleMethodsWrite|TestRWFileHandleWriteAt|TestRWFileHandleWriteNoWrite|TestWriteFileHandleFlush|TestWriteFileHandleMethods|TestWriteFileHandleWriteAt)$'" - Starting (try 5/5) === RUN TestRWFileHandleMethodsWrite 2020/07/24 10:41:37 INFO : Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7': poll-interval is not supported by this remote 2020/07/24 10:41:37 DEBUG : vfs cache: root is "/home/rclone/.cache/rclone/vfs/TestChunkerChunk50bMD5HashS3/rclone-test-lubaxip2wawolek7yiruram7" 2020/07/24 10:41:37 DEBUG : vfs cache: metadata root is "/home/rclone/.cache/rclone/vfs/TestChunkerChunk50bMD5HashS3/rclone-test-lubaxip2wawolek7yiruram7" 2020/07/24 10:41:37 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2020/07/24 10:41:37 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2020/07/24 10:41:37 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2020/07/24 10:41:37 DEBUG : file1: newRWFileHandle: 2020/07/24 10:41:37 DEBUG : file1(0xc0000add40): openPending: 2020/07/24 10:41:37 DEBUG : file1: vfs cache: truncate to size=0 2020/07/24 10:41:37 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:41:37 DEBUG : file1(0xc0000add40): >openPending: err= 2020/07/24 10:41:37 DEBUG : file1: >newRWFileHandle: err= 2020/07/24 10:41:37 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:41:37 DEBUG : file1: >Open: fd=file1 (rw), err= 2020/07/24 10:41:37 DEBUG : file1: >OpenFile: fd=file1 (rw), err= 2020/07/24 10:41:37 DEBUG : file1(0xc0000add40): _writeAt: size=5, off=0 2020/07/24 10:41:37 DEBUG : file1(0xc0000add40): >_writeAt: n=5, err= 2020/07/24 10:41:37 DEBUG : file1(0xc0000add40): _writeAt: size=7, off=5 2020/07/24 10:41:37 DEBUG : file1(0xc0000add40): >_writeAt: n=7, err= 2020/07/24 10:41:37 DEBUG : file1: vfs cache: truncate to size=11 2020/07/24 10:41:37 DEBUG : file1(0xc0000add40): close: 2020/07/24 10:41:37 DEBUG : file1: vfs cache: setting modification time to 2020-07-24 10:41:37.189942994 +0000 UTC m=+0.138836686 2020/07/24 10:41:37 INFO : file1: vfs cache: queuing for upload in 100ms 2020/07/24 10:41:37 DEBUG : file1(0xc0000add40): >close: err= 2020/07/24 10:41:37 DEBUG : file1(0xc0000add40): close: 2020/07/24 10:41:37 DEBUG : file1(0xc0000add40): >close: err=file already closed 2020/07/24 10:41:37 DEBUG : WaitForWriters: timeout=10s 2020/07/24 10:41:37 DEBUG : : Looking for writers 2020/07/24 10:41:37 DEBUG : file1: reading active writers 2020/07/24 10:41:37 DEBUG : Still 0 writers active and 1 cache items in use, waiting 10ms 2020/07/24 10:41:37 DEBUG : : Looking for writers 2020/07/24 10:41:37 DEBUG : file1: reading active writers 2020/07/24 10:41:37 DEBUG : Still 0 writers active and 1 cache items in use, waiting 20ms 2020/07/24 10:41:37 DEBUG : : Looking for writers 2020/07/24 10:41:37 DEBUG : file1: reading active writers 2020/07/24 10:41:37 DEBUG : Still 0 writers active and 1 cache items in use, waiting 40ms 2020/07/24 10:41:37 DEBUG : : Looking for writers 2020/07/24 10:41:37 DEBUG : file1: reading active writers 2020/07/24 10:41:37 DEBUG : Still 0 writers active and 1 cache items in use, waiting 80ms 2020/07/24 10:41:37 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:41:37 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram72FA0FEFCEBF7905D3HNy1ouXiXrjq8C3hC5DVlh1mTJ+xTIPEzfUcEZJRg22CuoCbcA1mD5+jCHs+EYc+/m7gCSJoSE= 2020/07/24 10:41:37 ERROR : file1: vfs cache: failed to upload try #1, will retry in 200ms: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram72FA0FEFCEBF7905D3HNy1ouXiXrjq8C3hC5DVlh1mTJ+xTIPEzfUcEZJRg22CuoCbcA1mD5+jCHs+EYc+/m7gCSJoSE= 2020/07/24 10:41:37 DEBUG : : Looking for writers 2020/07/24 10:41:37 DEBUG : file1: reading active writers 2020/07/24 10:41:37 DEBUG : Still 0 writers active and 1 cache items in use, waiting 160ms 2020/07/24 10:41:37 DEBUG : : Looking for writers 2020/07/24 10:41:37 DEBUG : file1: reading active writers 2020/07/24 10:41:37 DEBUG : Still 0 writers active and 1 cache items in use, waiting 320ms 2020/07/24 10:41:37 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:41:37 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram76Z4M6Q1M9T6J2T7WjekFAUD1SdXbZrQypPwq1k+Sj3F1y3WPzW/CcWpPgoXauBI5mGmwO+kd4Br6anYGXQObKCOTajo= 2020/07/24 10:41:37 ERROR : file1: vfs cache: failed to upload try #2, will retry in 400ms: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram76Z4M6Q1M9T6J2T7WjekFAUD1SdXbZrQypPwq1k+Sj3F1y3WPzW/CcWpPgoXauBI5mGmwO+kd4Br6anYGXQObKCOTajo= 2020/07/24 10:41:37 DEBUG : : Looking for writers 2020/07/24 10:41:37 DEBUG : file1: reading active writers 2020/07/24 10:41:37 DEBUG : Still 0 writers active and 1 cache items in use, waiting 640ms 2020/07/24 10:41:38 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:41:38 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram748944974732D2BC9YapDD1nNPIP4kAvjmKtGK3xLMBIvq+ajIIWPz9XmKPhkSXZ163J5wI5BXh/E8F5nDy9KV6isLWU= 2020/07/24 10:41:38 ERROR : file1: vfs cache: failed to upload try #3, will retry in 800ms: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram748944974732D2BC9YapDD1nNPIP4kAvjmKtGK3xLMBIvq+ajIIWPz9XmKPhkSXZ163J5wI5BXh/E8F5nDy9KV6isLWU= 2020/07/24 10:41:38 DEBUG : : Looking for writers 2020/07/24 10:41:38 DEBUG : file1: reading active writers 2020/07/24 10:41:38 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:41:38 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:41:38 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram71154FBBA41FAFBD9fRfynSHE5Sxx0bfrxKkE1X0MvcbndhAW3A0K+4nrPfWpNwglN/o1j+5aijpjmDd2Pf8CO5LX7zQ= 2020/07/24 10:41:38 ERROR : file1: vfs cache: failed to upload try #4, will retry in 1.6s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram71154FBBA41FAFBD9fRfynSHE5Sxx0bfrxKkE1X0MvcbndhAW3A0K+4nrPfWpNwglN/o1j+5aijpjmDd2Pf8CO5LX7zQ= 2020/07/24 10:41:39 DEBUG : : Looking for writers 2020/07/24 10:41:39 DEBUG : file1: reading active writers 2020/07/24 10:41:39 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:41:40 DEBUG : : Looking for writers 2020/07/24 10:41:40 DEBUG : file1: reading active writers 2020/07/24 10:41:40 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:41:40 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:41:40 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram70BCD76A66795CAEFK5KIN3gVsCpxXMYyw8Rz1zrc6cgBpe4ETqo7vfw08NFdWzhsKlIzt/1r0E7SqGk7m4hCTTWZFmE= 2020/07/24 10:41:40 ERROR : file1: vfs cache: failed to upload try #5, will retry in 3.2s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram70BCD76A66795CAEFK5KIN3gVsCpxXMYyw8Rz1zrc6cgBpe4ETqo7vfw08NFdWzhsKlIzt/1r0E7SqGk7m4hCTTWZFmE= 2020/07/24 10:41:41 DEBUG : : Looking for writers 2020/07/24 10:41:41 DEBUG : file1: reading active writers 2020/07/24 10:41:41 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:41:42 DEBUG : : Looking for writers 2020/07/24 10:41:42 DEBUG : file1: reading active writers 2020/07/24 10:41:42 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:41:43 DEBUG : : Looking for writers 2020/07/24 10:41:43 DEBUG : file1: reading active writers 2020/07/24 10:41:43 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:41:43 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:41:43 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram77F5113F74200FED5Jdybi9lUwg+Hhx4srPzBBAQjd3a4XGOML+L905fL4o08XSXMYpBJxUDe/8JzsJF7tl6tZo1RCsw= 2020/07/24 10:41:43 ERROR : file1: vfs cache: failed to upload try #6, will retry in 6.4s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram77F5113F74200FED5Jdybi9lUwg+Hhx4srPzBBAQjd3a4XGOML+L905fL4o08XSXMYpBJxUDe/8JzsJF7tl6tZo1RCsw= 2020/07/24 10:41:44 DEBUG : : Looking for writers 2020/07/24 10:41:44 DEBUG : file1: reading active writers 2020/07/24 10:41:44 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:41:45 DEBUG : : Looking for writers 2020/07/24 10:41:45 DEBUG : file1: reading active writers 2020/07/24 10:41:45 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:41:46 DEBUG : : Looking for writers 2020/07/24 10:41:46 DEBUG : file1: reading active writers 2020/07/24 10:41:46 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:41:47 ERROR : Exiting even though 0 writers active and 1 cache items in use after 10s Cache{ "file1": &{c:0xc0001c61b0 mu:{state:0 sema:0} name:file1 opens:0 downloaders: o: fd: metaDirty:false modified:false info:{ModTime:{wall:13816705661290040530 ext:138836686 loc:0x28e63c0} ATime:{wall:13816705661290053047 ext:138849170 loc:0x28e63c0} Size:11 Rs:[{Pos:0 Size:11}] Fingerprint: Dirty:true} writeBackID:1}, } 2020/07/24 10:41:47 DEBUG : >WaitForWriters: 2020/07/24 10:41:50 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:41:50 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7897CFBEAB7FE4AFBrDo25gZ/JBpWOIRHcTYa47xSAvat67P9l7hgtMVSygmKkpZE/r/+GNxpow0L9O6h2i0raKcixBg= 2020/07/24 10:41:50 ERROR : file1: vfs cache: failed to upload try #7, will retry in 12.8s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7897CFBEAB7FE4AFBrDo25gZ/JBpWOIRHcTYa47xSAvat67P9l7hgtMVSygmKkpZE/r/+GNxpow0L9O6h2i0raKcixBg= 2020/07/24 10:41:54 DEBUG : WaitForWriters: timeout=10s 2020/07/24 10:41:54 DEBUG : : Looking for writers 2020/07/24 10:41:54 DEBUG : file1: reading active writers 2020/07/24 10:41:54 DEBUG : Still 0 writers active and 1 cache items in use, waiting 10ms 2020/07/24 10:41:54 DEBUG : : Looking for writers 2020/07/24 10:41:54 DEBUG : file1: reading active writers 2020/07/24 10:41:54 DEBUG : Still 0 writers active and 1 cache items in use, waiting 20ms 2020/07/24 10:41:54 DEBUG : : Looking for writers 2020/07/24 10:41:54 DEBUG : file1: reading active writers 2020/07/24 10:41:54 DEBUG : Still 0 writers active and 1 cache items in use, waiting 40ms 2020/07/24 10:41:54 DEBUG : : Looking for writers 2020/07/24 10:41:54 DEBUG : file1: reading active writers 2020/07/24 10:41:54 DEBUG : Still 0 writers active and 1 cache items in use, waiting 80ms 2020/07/24 10:41:54 DEBUG : : Looking for writers 2020/07/24 10:41:54 DEBUG : file1: reading active writers 2020/07/24 10:41:54 DEBUG : Still 0 writers active and 1 cache items in use, waiting 160ms 2020/07/24 10:41:54 DEBUG : : Looking for writers 2020/07/24 10:41:54 DEBUG : file1: reading active writers 2020/07/24 10:41:54 DEBUG : Still 0 writers active and 1 cache items in use, waiting 320ms 2020/07/24 10:41:54 DEBUG : : Looking for writers 2020/07/24 10:41:54 DEBUG : file1: reading active writers 2020/07/24 10:41:54 DEBUG : Still 0 writers active and 1 cache items in use, waiting 640ms 2020/07/24 10:41:55 DEBUG : : Looking for writers 2020/07/24 10:41:55 DEBUG : file1: reading active writers 2020/07/24 10:41:55 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:41:56 DEBUG : : Looking for writers 2020/07/24 10:41:56 DEBUG : file1: reading active writers 2020/07/24 10:41:56 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:41:57 DEBUG : : Looking for writers 2020/07/24 10:41:57 DEBUG : file1: reading active writers 2020/07/24 10:41:57 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:41:58 DEBUG : : Looking for writers 2020/07/24 10:41:58 DEBUG : file1: reading active writers 2020/07/24 10:41:58 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:41:59 DEBUG : : Looking for writers 2020/07/24 10:41:59 DEBUG : file1: reading active writers 2020/07/24 10:41:59 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:00 DEBUG : : Looking for writers 2020/07/24 10:42:00 DEBUG : file1: reading active writers 2020/07/24 10:42:00 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:01 DEBUG : : Looking for writers 2020/07/24 10:42:01 DEBUG : file1: reading active writers 2020/07/24 10:42:01 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:02 DEBUG : : Looking for writers 2020/07/24 10:42:02 DEBUG : file1: reading active writers 2020/07/24 10:42:02 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:03 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:03 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram71KBQFSAG1VAG0T3MltLqS9KgcrMJ0mAXL5c6fHJbHNXQUI+JzBYjqv9L/niAjBs5xjgAkizQ6bDjl7O6nE0GSE+rVFI= 2020/07/24 10:42:03 ERROR : file1: vfs cache: failed to upload try #8, will retry in 25.6s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram71KBQFSAG1VAG0T3MltLqS9KgcrMJ0mAXL5c6fHJbHNXQUI+JzBYjqv9L/niAjBs5xjgAkizQ6bDjl7O6nE0GSE+rVFI= 2020/07/24 10:42:03 DEBUG : : Looking for writers 2020/07/24 10:42:03 DEBUG : file1: reading active writers 2020/07/24 10:42:03 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:04 ERROR : Exiting even though 0 writers active and 1 cache items in use after 10s Cache{ "file1": &{c:0xc0001c61b0 mu:{state:0 sema:0} name:file1 opens:0 downloaders: o: fd: metaDirty:false modified:false info:{ModTime:{wall:13816705661290040530 ext:138836686 loc:0x28e63c0} ATime:{wall:13816705661290053047 ext:138849170 loc:0x28e63c0} Size:11 Rs:[{Pos:0 Size:11}] Fingerprint: Dirty:true} writeBackID:1}, } 2020/07/24 10:42:04 DEBUG : >WaitForWriters: 2020/07/24 10:42:04 DEBUG : vfs cache: cleaner exiting --- FAIL: TestRWFileHandleMethodsWrite (27.19s) run.go:176: Remote "Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7'", Local "Local file system at /tmp/rclone577212835", Modify Window "1ns" fstest.go:295: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:295: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:295: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:302: Error Trace: fstest.go:302 fstest.go:335 read_write_test.go:344 Error: Should be true Test: TestRWFileHandleMethodsWrite Messages: listing wrong, want file1 (11) got fstest.go:200: Not found "file1" fstest.go:203: Error Trace: fstest.go:203 fstest.go:307 fstest.go:335 read_write_test.go:344 Error: Not equal: expected: 0 actual : 1 Test: TestRWFileHandleMethodsWrite Messages: 1 objects not found === RUN TestRWFileHandleWriteAt 2020/07/24 10:42:04 INFO : Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7': poll-interval is not supported by this remote 2020/07/24 10:42:04 DEBUG : vfs cache: root is "/home/rclone/.cache/rclone/vfs/TestChunkerChunk50bMD5HashS3/rclone-test-lubaxip2wawolek7yiruram7" 2020/07/24 10:42:04 DEBUG : vfs cache: metadata root is "/home/rclone/.cache/rclone/vfs/TestChunkerChunk50bMD5HashS3/rclone-test-lubaxip2wawolek7yiruram7" 2020/07/24 10:42:04 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2020/07/24 10:42:04 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2020/07/24 10:42:04 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2020/07/24 10:42:04 DEBUG : file1: newRWFileHandle: 2020/07/24 10:42:04 DEBUG : file1(0xc0007523c0): openPending: 2020/07/24 10:42:04 DEBUG : file1: vfs cache: truncate to size=0 2020/07/24 10:42:04 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:42:04 DEBUG : file1(0xc0007523c0): >openPending: err= 2020/07/24 10:42:04 DEBUG : file1: >newRWFileHandle: err= 2020/07/24 10:42:04 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:42:04 DEBUG : file1: >Open: fd=file1 (rw), err= 2020/07/24 10:42:04 DEBUG : file1: >OpenFile: fd=file1 (rw), err= 2020/07/24 10:42:04 DEBUG : file1(0xc0007523c0): _writeAt: size=7, off=0 2020/07/24 10:42:04 DEBUG : file1(0xc0007523c0): >_writeAt: n=7, err= 2020/07/24 10:42:04 DEBUG : file1(0xc0007523c0): _writeAt: size=6, off=5 2020/07/24 10:42:04 DEBUG : file1(0xc0007523c0): >_writeAt: n=6, err= 2020/07/24 10:42:04 DEBUG : file1(0xc0007523c0): close: 2020/07/24 10:42:04 DEBUG : file1: vfs cache: setting modification time to 2020-07-24 10:42:04.273167319 +0000 UTC m=+27.222060991 2020/07/24 10:42:04 INFO : file1: vfs cache: queuing for upload in 100ms 2020/07/24 10:42:04 DEBUG : file1(0xc0007523c0): >close: err= 2020/07/24 10:42:04 DEBUG : file1(0xc0007523c0): _writeAt: size=5, off=0 2020/07/24 10:42:04 DEBUG : file1(0xc0007523c0): >_writeAt: n=0, err=file already closed 2020/07/24 10:42:04 DEBUG : WaitForWriters: timeout=10s 2020/07/24 10:42:04 DEBUG : : Looking for writers 2020/07/24 10:42:04 DEBUG : file1: reading active writers 2020/07/24 10:42:04 DEBUG : Still 0 writers active and 1 cache items in use, waiting 10ms 2020/07/24 10:42:04 DEBUG : : Looking for writers 2020/07/24 10:42:04 DEBUG : file1: reading active writers 2020/07/24 10:42:04 DEBUG : Still 0 writers active and 1 cache items in use, waiting 20ms 2020/07/24 10:42:04 DEBUG : : Looking for writers 2020/07/24 10:42:04 DEBUG : file1: reading active writers 2020/07/24 10:42:04 DEBUG : Still 0 writers active and 1 cache items in use, waiting 40ms 2020/07/24 10:42:04 DEBUG : : Looking for writers 2020/07/24 10:42:04 DEBUG : file1: reading active writers 2020/07/24 10:42:04 DEBUG : Still 0 writers active and 1 cache items in use, waiting 80ms 2020/07/24 10:42:04 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:04 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram77E4499989CE40BBFPNP3n+KdC7Ri9wE/5FYz5GoMRp61tx4Y1MSujn54oMIgcGShdW1WtH/X8e88pWqkeFU1lgDL/eY= 2020/07/24 10:42:04 ERROR : file1: vfs cache: failed to upload try #1, will retry in 200ms: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram77E4499989CE40BBFPNP3n+KdC7Ri9wE/5FYz5GoMRp61tx4Y1MSujn54oMIgcGShdW1WtH/X8e88pWqkeFU1lgDL/eY= 2020/07/24 10:42:04 DEBUG : : Looking for writers 2020/07/24 10:42:04 DEBUG : file1: reading active writers 2020/07/24 10:42:04 DEBUG : Still 0 writers active and 1 cache items in use, waiting 160ms 2020/07/24 10:42:04 DEBUG : : Looking for writers 2020/07/24 10:42:04 DEBUG : file1: reading active writers 2020/07/24 10:42:04 DEBUG : Still 0 writers active and 1 cache items in use, waiting 320ms 2020/07/24 10:42:04 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:04 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7EE1A5E02F7EEC2EFYjSuVejzSUaeN0bjDLcZjwnq3skcVa6kHsJRbuvzjFZVvZxcJGjQfjSDb38uWFLiJ6RUj47RayM= 2020/07/24 10:42:04 ERROR : file1: vfs cache: failed to upload try #2, will retry in 400ms: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7EE1A5E02F7EEC2EFYjSuVejzSUaeN0bjDLcZjwnq3skcVa6kHsJRbuvzjFZVvZxcJGjQfjSDb38uWFLiJ6RUj47RayM= 2020/07/24 10:42:04 DEBUG : : Looking for writers 2020/07/24 10:42:04 DEBUG : file1: reading active writers 2020/07/24 10:42:04 DEBUG : Still 0 writers active and 1 cache items in use, waiting 640ms 2020/07/24 10:42:05 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:05 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram73C11061F556FFD03xxN+l6m40IkR3XN6EdLI9zNKg0PWZXE5gfOOyWHt4RAZnxi6YeEmQIL8fjgr3i9DJQgqQjgrUa8= 2020/07/24 10:42:05 ERROR : file1: vfs cache: failed to upload try #3, will retry in 800ms: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram73C11061F556FFD03xxN+l6m40IkR3XN6EdLI9zNKg0PWZXE5gfOOyWHt4RAZnxi6YeEmQIL8fjgr3i9DJQgqQjgrUa8= 2020/07/24 10:42:05 DEBUG : : Looking for writers 2020/07/24 10:42:05 DEBUG : file1: reading active writers 2020/07/24 10:42:05 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:05 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:05 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7AJ0KCN4S5T6W5RFTHA9CQD60OfgxfraEEsA4OYlLvHSFdwe84RF7/2oVeTaviF27Bq/02+8D0WX1ilvKlaSrZ4FetdI= 2020/07/24 10:42:05 ERROR : file1: vfs cache: failed to upload try #4, will retry in 1.6s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7AJ0KCN4S5T6W5RFTHA9CQD60OfgxfraEEsA4OYlLvHSFdwe84RF7/2oVeTaviF27Bq/02+8D0WX1ilvKlaSrZ4FetdI= 2020/07/24 10:42:06 DEBUG : : Looking for writers 2020/07/24 10:42:06 DEBUG : file1: reading active writers 2020/07/24 10:42:06 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:07 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:07 DEBUG : : Looking for writers 2020/07/24 10:42:07 DEBUG : file1: reading active writers 2020/07/24 10:42:07 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:07 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram794B05470F13DBFC8CaaII6WX7Fswst6neBTjSORHLSifCFVcJZbOGJAAI2vBJLLlfnHr2iDCSnWcft0ny3/B0AT2dFY= 2020/07/24 10:42:07 ERROR : file1: vfs cache: failed to upload try #5, will retry in 3.2s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram794B05470F13DBFC8CaaII6WX7Fswst6neBTjSORHLSifCFVcJZbOGJAAI2vBJLLlfnHr2iDCSnWcft0ny3/B0AT2dFY= 2020/07/24 10:42:08 DEBUG : : Looking for writers 2020/07/24 10:42:08 DEBUG : file1: reading active writers 2020/07/24 10:42:08 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:09 DEBUG : : Looking for writers 2020/07/24 10:42:09 DEBUG : file1: reading active writers 2020/07/24 10:42:09 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:10 DEBUG : : Looking for writers 2020/07/24 10:42:10 DEBUG : file1: reading active writers 2020/07/24 10:42:10 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:10 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:10 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7007BE99A5DE61F5AvVUKpjsxCSjKf2JUQscThOeVimlEk3So6xUyxeKmG28xVHbBES1T8Qk1GrVOoTyosj9hBB+IC7U= 2020/07/24 10:42:10 ERROR : file1: vfs cache: failed to upload try #6, will retry in 6.4s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7007BE99A5DE61F5AvVUKpjsxCSjKf2JUQscThOeVimlEk3So6xUyxeKmG28xVHbBES1T8Qk1GrVOoTyosj9hBB+IC7U= 2020/07/24 10:42:11 DEBUG : : Looking for writers 2020/07/24 10:42:11 DEBUG : file1: reading active writers 2020/07/24 10:42:11 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:12 DEBUG : : Looking for writers 2020/07/24 10:42:12 DEBUG : file1: reading active writers 2020/07/24 10:42:12 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:13 DEBUG : : Looking for writers 2020/07/24 10:42:13 DEBUG : file1: reading active writers 2020/07/24 10:42:13 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:14 ERROR : Exiting even though 0 writers active and 1 cache items in use after 10s Cache{ "file1": &{c:0xc00077c990 mu:{state:0 sema:0} name:file1 opens:0 downloaders: o: fd: metaDirty:false modified:false info:{ModTime:{wall:13816705690364294103 ext:27222060991 loc:0x28e63c0} ATime:{wall:13816705690364311082 ext:27222077955 loc:0x28e63c0} Size:11 Rs:[{Pos:0 Size:11}] Fingerprint: Dirty:true} writeBackID:1}, } 2020/07/24 10:42:14 DEBUG : >WaitForWriters: 2020/07/24 10:42:17 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:17 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram77B10F3EA01FE3686xAScdpcjDa1XjhXzk65Jodfl/3o/iuOLP/fnRVkttsle7gV0W65sVH0tBnHCP5McDATBDXOP7E4= 2020/07/24 10:42:17 ERROR : file1: vfs cache: failed to upload try #7, will retry in 12.8s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram77B10F3EA01FE3686xAScdpcjDa1XjhXzk65Jodfl/3o/iuOLP/fnRVkttsle7gV0W65sVH0tBnHCP5McDATBDXOP7E4= 2020/07/24 10:42:21 DEBUG : WaitForWriters: timeout=10s 2020/07/24 10:42:21 DEBUG : : Looking for writers 2020/07/24 10:42:21 DEBUG : file1: reading active writers 2020/07/24 10:42:21 DEBUG : Still 0 writers active and 1 cache items in use, waiting 10ms 2020/07/24 10:42:21 DEBUG : : Looking for writers 2020/07/24 10:42:21 DEBUG : file1: reading active writers 2020/07/24 10:42:21 DEBUG : Still 0 writers active and 1 cache items in use, waiting 20ms 2020/07/24 10:42:21 DEBUG : : Looking for writers 2020/07/24 10:42:21 DEBUG : file1: reading active writers 2020/07/24 10:42:21 DEBUG : Still 0 writers active and 1 cache items in use, waiting 40ms 2020/07/24 10:42:21 DEBUG : : Looking for writers 2020/07/24 10:42:21 DEBUG : file1: reading active writers 2020/07/24 10:42:21 DEBUG : Still 0 writers active and 1 cache items in use, waiting 80ms 2020/07/24 10:42:21 DEBUG : : Looking for writers 2020/07/24 10:42:21 DEBUG : file1: reading active writers 2020/07/24 10:42:21 DEBUG : Still 0 writers active and 1 cache items in use, waiting 160ms 2020/07/24 10:42:21 DEBUG : : Looking for writers 2020/07/24 10:42:21 DEBUG : file1: reading active writers 2020/07/24 10:42:21 DEBUG : Still 0 writers active and 1 cache items in use, waiting 320ms 2020/07/24 10:42:21 DEBUG : : Looking for writers 2020/07/24 10:42:21 DEBUG : file1: reading active writers 2020/07/24 10:42:21 DEBUG : Still 0 writers active and 1 cache items in use, waiting 640ms 2020/07/24 10:42:22 DEBUG : : Looking for writers 2020/07/24 10:42:22 DEBUG : file1: reading active writers 2020/07/24 10:42:22 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:23 DEBUG : : Looking for writers 2020/07/24 10:42:23 DEBUG : file1: reading active writers 2020/07/24 10:42:23 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:24 DEBUG : : Looking for writers 2020/07/24 10:42:24 DEBUG : file1: reading active writers 2020/07/24 10:42:24 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:25 DEBUG : : Looking for writers 2020/07/24 10:42:25 DEBUG : file1: reading active writers 2020/07/24 10:42:25 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:26 DEBUG : : Looking for writers 2020/07/24 10:42:26 DEBUG : file1: reading active writers 2020/07/24 10:42:26 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:27 DEBUG : : Looking for writers 2020/07/24 10:42:27 DEBUG : file1: reading active writers 2020/07/24 10:42:27 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:28 DEBUG : : Looking for writers 2020/07/24 10:42:28 DEBUG : file1: reading active writers 2020/07/24 10:42:28 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:29 DEBUG : : Looking for writers 2020/07/24 10:42:29 DEBUG : file1: reading active writers 2020/07/24 10:42:29 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:30 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:30 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7FF2CAAAE9D87B7ADL6BrEv7o7Kajh+TbMX4F8GC/ICJ6BiXcPX7DDLgnJzu2EPjPX6hC+b5OCUO1ojbmgm4qgpANudY= 2020/07/24 10:42:30 ERROR : file1: vfs cache: failed to upload try #8, will retry in 25.6s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7FF2CAAAE9D87B7ADL6BrEv7o7Kajh+TbMX4F8GC/ICJ6BiXcPX7DDLgnJzu2EPjPX6hC+b5OCUO1ojbmgm4qgpANudY= 2020/07/24 10:42:30 DEBUG : : Looking for writers 2020/07/24 10:42:30 DEBUG : file1: reading active writers 2020/07/24 10:42:30 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2020/07/24 10:42:31 ERROR : Exiting even though 0 writers active and 1 cache items in use after 10s Cache{ "file1": &{c:0xc00077c990 mu:{state:0 sema:0} name:file1 opens:0 downloaders: o: fd: metaDirty:false modified:false info:{ModTime:{wall:13816705690364294103 ext:27222060991 loc:0x28e63c0} ATime:{wall:13816705690364311082 ext:27222077955 loc:0x28e63c0} Size:11 Rs:[{Pos:0 Size:11}] Fingerprint: Dirty:true} writeBackID:1}, } 2020/07/24 10:42:31 DEBUG : >WaitForWriters: 2020/07/24 10:42:31 DEBUG : vfs cache: cleaner exiting --- FAIL: TestRWFileHandleWriteAt (27.12s) run.go:176: Remote "Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7'", Local "Local file system at /tmp/rclone577212835", Modify Window "1ns" fstest.go:295: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:295: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:295: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:302: Error Trace: fstest.go:302 fstest.go:335 read_write_test.go:392 Error: Should be true Test: TestRWFileHandleWriteAt Messages: listing wrong, want file1 (11) got fstest.go:200: Not found "file1" fstest.go:203: Error Trace: fstest.go:203 fstest.go:307 fstest.go:335 read_write_test.go:392 Error: Not equal: expected: 0 actual : 1 Test: TestRWFileHandleWriteAt Messages: 1 objects not found === RUN TestRWFileHandleWriteNoWrite 2020/07/24 10:42:31 INFO : Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7': poll-interval is not supported by this remote 2020/07/24 10:42:31 DEBUG : vfs cache: root is "/home/rclone/.cache/rclone/vfs/TestChunkerChunk50bMD5HashS3/rclone-test-lubaxip2wawolek7yiruram7" 2020/07/24 10:42:31 DEBUG : vfs cache: metadata root is "/home/rclone/.cache/rclone/vfs/TestChunkerChunk50bMD5HashS3/rclone-test-lubaxip2wawolek7yiruram7" 2020/07/24 10:42:31 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2020/07/24 10:42:31 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2020/07/24 10:42:31 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2020/07/24 10:42:31 DEBUG : file1: newRWFileHandle: 2020/07/24 10:42:31 DEBUG : file1(0xc000150d00): openPending: 2020/07/24 10:42:31 DEBUG : file1: vfs cache: truncate to size=0 2020/07/24 10:42:31 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:42:31 DEBUG : file1(0xc000150d00): >openPending: err= 2020/07/24 10:42:31 DEBUG : file1: >newRWFileHandle: err= 2020/07/24 10:42:31 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:42:31 DEBUG : file1: >Open: fd=file1 (rw), err= 2020/07/24 10:42:31 DEBUG : file1: >OpenFile: fd=file1 (rw), err= 2020/07/24 10:42:31 DEBUG : file1(0xc000150d00): close: 2020/07/24 10:42:31 DEBUG : file1: vfs cache: setting modification time to 2020-07-24 10:42:31.389178876 +0000 UTC m=+54.338072535 2020/07/24 10:42:31 INFO : file1: vfs cache: queuing for upload in 100ms 2020/07/24 10:42:31 DEBUG : file1(0xc000150d00): >close: err= 2020/07/24 10:42:31 DEBUG : file2: OpenFile: flags=O_WRONLY|O_CREATE|O_TRUNC, perm=-rwxrwxrwx 2020/07/24 10:42:31 DEBUG : file2: Open: flags=O_WRONLY|O_CREATE|O_TRUNC 2020/07/24 10:42:31 DEBUG : file2: newRWFileHandle: 2020/07/24 10:42:31 DEBUG : file2(0xc000151000): openPending: 2020/07/24 10:42:31 DEBUG : file2: vfs cache: truncate to size=0 2020/07/24 10:42:31 DEBUG : : Added virtual directory entry vAdd: "file2" 2020/07/24 10:42:31 DEBUG : file2(0xc000151000): >openPending: err= 2020/07/24 10:42:31 DEBUG : file2: >newRWFileHandle: err= 2020/07/24 10:42:31 DEBUG : : Added virtual directory entry vAdd: "file2" 2020/07/24 10:42:31 DEBUG : file2: >Open: fd=file2 (rw), err= 2020/07/24 10:42:31 DEBUG : file2: >OpenFile: fd=file2 (rw), err= 2020/07/24 10:42:31 DEBUG : file2(0xc000151000): RWFileHandle.Flush 2020/07/24 10:42:31 DEBUG : file2(0xc000151000): RWFileHandle.Release 2020/07/24 10:42:31 DEBUG : file2(0xc000151000): close: 2020/07/24 10:42:31 DEBUG : file2: vfs cache: setting modification time to 2020-07-24 10:42:31.389943822 +0000 UTC m=+54.338837479 2020/07/24 10:42:31 INFO : file2: vfs cache: queuing for upload in 100ms 2020/07/24 10:42:31 DEBUG : file2(0xc000151000): >close: err= 2020/07/24 10:42:31 DEBUG : WaitForWriters: timeout=10s 2020/07/24 10:42:31 DEBUG : : Looking for writers 2020/07/24 10:42:31 DEBUG : file1: reading active writers 2020/07/24 10:42:31 DEBUG : file2: reading active writers 2020/07/24 10:42:31 DEBUG : Still 0 writers active and 2 cache items in use, waiting 10ms 2020/07/24 10:42:31 DEBUG : : Looking for writers 2020/07/24 10:42:31 DEBUG : file1: reading active writers 2020/07/24 10:42:31 DEBUG : file2: reading active writers 2020/07/24 10:42:31 DEBUG : Still 0 writers active and 2 cache items in use, waiting 20ms 2020/07/24 10:42:31 DEBUG : : Looking for writers 2020/07/24 10:42:31 DEBUG : file1: reading active writers 2020/07/24 10:42:31 DEBUG : file2: reading active writers 2020/07/24 10:42:31 DEBUG : Still 0 writers active and 2 cache items in use, waiting 40ms 2020/07/24 10:42:31 DEBUG : : Looking for writers 2020/07/24 10:42:31 DEBUG : file1: reading active writers 2020/07/24 10:42:31 DEBUG : file2: reading active writers 2020/07/24 10:42:31 DEBUG : Still 0 writers active and 2 cache items in use, waiting 80ms 2020/07/24 10:42:31 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:31 DEBUG : file2: vfs cache: starting upload 2020/07/24 10:42:31 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7DDF7F333E09DC179Swh1r2WFHsAJjOF0KuCS6awktgxKCS/yuktyLxWcnHWy18mLtwFVQ0zGgQiaeWcshI1R/a/8/e4= 2020/07/24 10:42:31 ERROR : file1: vfs cache: failed to upload try #1, will retry in 200ms: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7DDF7F333E09DC179Swh1r2WFHsAJjOF0KuCS6awktgxKCS/yuktyLxWcnHWy18mLtwFVQ0zGgQiaeWcshI1R/a/8/e4= 2020/07/24 10:42:31 DEBUG : : Looking for writers 2020/07/24 10:42:31 DEBUG : file1: reading active writers 2020/07/24 10:42:31 DEBUG : file2: reading active writers 2020/07/24 10:42:31 DEBUG : Still 0 writers active and 2 cache items in use, waiting 160ms 2020/07/24 10:42:31 ERROR : file2: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram78112FE2052CDBCEDXLthR5M5fcoiC0bul5Y+7NY1ZkkkT5qZRMhDf1FhPKoyliMGUbZYEspqxYGE9Hg1AEklqcmCEcg= 2020/07/24 10:42:31 ERROR : file2: vfs cache: failed to upload try #1, will retry in 200ms: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram78112FE2052CDBCEDXLthR5M5fcoiC0bul5Y+7NY1ZkkkT5qZRMhDf1FhPKoyliMGUbZYEspqxYGE9Hg1AEklqcmCEcg= 2020/07/24 10:42:31 DEBUG : : Looking for writers 2020/07/24 10:42:31 DEBUG : file1: reading active writers 2020/07/24 10:42:31 DEBUG : file2: reading active writers 2020/07/24 10:42:31 DEBUG : Still 0 writers active and 2 cache items in use, waiting 320ms 2020/07/24 10:42:31 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:31 DEBUG : file2: vfs cache: starting upload 2020/07/24 10:42:31 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram72E553E39AA3F27E7LpeIqSacfIQ0BPKqEUT36HA9egFiSqFOPtnUpJSD5Umg6cRMzAUV3LVEQ/CAMapEvjAqI2uvqW8= 2020/07/24 10:42:31 ERROR : file1: vfs cache: failed to upload try #2, will retry in 400ms: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram72E553E39AA3F27E7LpeIqSacfIQ0BPKqEUT36HA9egFiSqFOPtnUpJSD5Umg6cRMzAUV3LVEQ/CAMapEvjAqI2uvqW8= 2020/07/24 10:42:31 ERROR : file2: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7C912919766FF5491NPhYv0x715lO+1sSvuk5wTy/nPNkKy74WdJgeW5JRRj/e1F8kVoHDmH+AYX0oJUAH4GoAfaXQnI= 2020/07/24 10:42:31 ERROR : file2: vfs cache: failed to upload try #2, will retry in 400ms: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7C912919766FF5491NPhYv0x715lO+1sSvuk5wTy/nPNkKy74WdJgeW5JRRj/e1F8kVoHDmH+AYX0oJUAH4GoAfaXQnI= 2020/07/24 10:42:32 DEBUG : : Looking for writers 2020/07/24 10:42:32 DEBUG : file1: reading active writers 2020/07/24 10:42:32 DEBUG : file2: reading active writers 2020/07/24 10:42:32 DEBUG : Still 0 writers active and 2 cache items in use, waiting 640ms 2020/07/24 10:42:32 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:32 DEBUG : file2: vfs cache: starting upload 2020/07/24 10:42:32 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7CDD77EC894C9CF9Blud/+Muhov6dZ4iHhzs3DRgl+fgQauy7Tl6WrzTF7oqJ9WiFlTWsF4xWnXHzpnaJAN2IHqQoYJ8= 2020/07/24 10:42:32 ERROR : file1: vfs cache: failed to upload try #3, will retry in 800ms: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7CDD77EC894C9CF9Blud/+Muhov6dZ4iHhzs3DRgl+fgQauy7Tl6WrzTF7oqJ9WiFlTWsF4xWnXHzpnaJAN2IHqQoYJ8= 2020/07/24 10:42:32 ERROR : file2: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram755B5E198511C09BC4TM3yFc3lvdC32lGRNrJb/tpbH2BXDPD8/SOXrANawEN3Vq+cDLLBXx4Xb4tDQlAurlArExPsFc= 2020/07/24 10:42:32 ERROR : file2: vfs cache: failed to upload try #3, will retry in 800ms: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram755B5E198511C09BC4TM3yFc3lvdC32lGRNrJb/tpbH2BXDPD8/SOXrANawEN3Vq+cDLLBXx4Xb4tDQlAurlArExPsFc= 2020/07/24 10:42:32 DEBUG : : Looking for writers 2020/07/24 10:42:32 DEBUG : file1: reading active writers 2020/07/24 10:42:32 DEBUG : file2: reading active writers 2020/07/24 10:42:32 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:32 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:33 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7F19A9E821FA243B3QwdbywjhUmV/oAkI+/55uDk7HfMqFbEx37ao7hScVbrlGwfPVUMqlzosHFpPy9fLB9VfuTonLS8= 2020/07/24 10:42:33 ERROR : file1: vfs cache: failed to upload try #4, will retry in 1.6s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7F19A9E821FA243B3QwdbywjhUmV/oAkI+/55uDk7HfMqFbEx37ao7hScVbrlGwfPVUMqlzosHFpPy9fLB9VfuTonLS8= 2020/07/24 10:42:33 DEBUG : file2: vfs cache: starting upload 2020/07/24 10:42:33 ERROR : file2: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7FR7Q2GCN9S1ZCQ2GpB/xxdYf9bp+Vi0CvfIJXi1qxNvJJFNxOlQ8YSwedL89V+3UvEhk3HYLpkGoEtcUOcg1laccULI= 2020/07/24 10:42:33 ERROR : file2: vfs cache: failed to upload try #4, will retry in 1.6s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7FR7Q2GCN9S1ZCQ2GpB/xxdYf9bp+Vi0CvfIJXi1qxNvJJFNxOlQ8YSwedL89V+3UvEhk3HYLpkGoEtcUOcg1laccULI= 2020/07/24 10:42:33 DEBUG : : Looking for writers 2020/07/24 10:42:33 DEBUG : file1: reading active writers 2020/07/24 10:42:33 DEBUG : file2: reading active writers 2020/07/24 10:42:33 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:34 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:34 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram74839E99A1F1C5C51ceLrXG/WRbP7x3mqetGG3NgLtykpMGZcwpurHSbGw1mn4nQUZYEhBoAcsGch9phJz8pTt2Q3xsE= 2020/07/24 10:42:34 ERROR : file1: vfs cache: failed to upload try #5, will retry in 3.2s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram74839E99A1F1C5C51ceLrXG/WRbP7x3mqetGG3NgLtykpMGZcwpurHSbGw1mn4nQUZYEhBoAcsGch9phJz8pTt2Q3xsE= 2020/07/24 10:42:34 DEBUG : : Looking for writers 2020/07/24 10:42:34 DEBUG : file2: reading active writers 2020/07/24 10:42:34 DEBUG : file1: reading active writers 2020/07/24 10:42:34 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:34 DEBUG : file2: vfs cache: starting upload 2020/07/24 10:42:34 ERROR : file2: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7CT3P3K8XAQ9T7GDWiL/KdjXLWu6WK1BTlEl5gRm4uTLFbQvwmZPD38IfTRE4MFn3Ow1xQ93KzbWSgSXJfVjUv4/WT+Y= 2020/07/24 10:42:34 ERROR : file2: vfs cache: failed to upload try #5, will retry in 3.2s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7CT3P3K8XAQ9T7GDWiL/KdjXLWu6WK1BTlEl5gRm4uTLFbQvwmZPD38IfTRE4MFn3Ow1xQ93KzbWSgSXJfVjUv4/WT+Y= 2020/07/24 10:42:35 DEBUG : : Looking for writers 2020/07/24 10:42:35 DEBUG : file2: reading active writers 2020/07/24 10:42:35 DEBUG : file1: reading active writers 2020/07/24 10:42:35 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:36 DEBUG : : Looking for writers 2020/07/24 10:42:36 DEBUG : file1: reading active writers 2020/07/24 10:42:36 DEBUG : file2: reading active writers 2020/07/24 10:42:36 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:37 DEBUG : : Looking for writers 2020/07/24 10:42:37 DEBUG : file1: reading active writers 2020/07/24 10:42:37 DEBUG : file2: reading active writers 2020/07/24 10:42:37 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:37 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:37 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram762C6947C68C75470qxrAiCAa7n3e/aDdznvSV/JQzpubyaHy5cKn5lYaq0YLOHFqIdXfUJGPrT/oAN22Ba1teCG9ggI= 2020/07/24 10:42:37 ERROR : file1: vfs cache: failed to upload try #6, will retry in 6.4s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram762C6947C68C75470qxrAiCAa7n3e/aDdznvSV/JQzpubyaHy5cKn5lYaq0YLOHFqIdXfUJGPrT/oAN22Ba1teCG9ggI= 2020/07/24 10:42:38 DEBUG : file2: vfs cache: starting upload 2020/07/24 10:42:38 ERROR : file2: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7BCF18CDE2DFDE550Mjz5pWe/pgRxxOGQgsxa9JamJomAtzEzMs5vf7ogXzeERptzhSd8tJxUjvw9sI897gzKXhfbjbg= 2020/07/24 10:42:38 ERROR : file2: vfs cache: failed to upload try #6, will retry in 6.4s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7BCF18CDE2DFDE550Mjz5pWe/pgRxxOGQgsxa9JamJomAtzEzMs5vf7ogXzeERptzhSd8tJxUjvw9sI897gzKXhfbjbg= 2020/07/24 10:42:38 DEBUG : : Looking for writers 2020/07/24 10:42:38 DEBUG : file1: reading active writers 2020/07/24 10:42:38 DEBUG : file2: reading active writers 2020/07/24 10:42:38 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:39 DEBUG : : Looking for writers 2020/07/24 10:42:39 DEBUG : file1: reading active writers 2020/07/24 10:42:39 DEBUG : file2: reading active writers 2020/07/24 10:42:39 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:40 DEBUG : : Looking for writers 2020/07/24 10:42:40 DEBUG : file1: reading active writers 2020/07/24 10:42:40 DEBUG : file2: reading active writers 2020/07/24 10:42:40 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:41 ERROR : Exiting even though 0 writers active and 2 cache items in use after 10s Cache{ "file2": &{c:0xc0001c6fc0 mu:{state:0 sema:0} name:file2 opens:0 downloaders: o: fd: metaDirty:false modified:false info:{ModTime:{wall:13816705719472099854 ext:54338837479 loc:0x28e63c0} ATime:{wall:13816705719472223974 ext:54338961607 loc:0x28e63c0} Size:0 Rs:[] Fingerprint: Dirty:true} writeBackID:2}, "file1": &{c:0xc0001c6fc0 mu:{state:0 sema:0} name:file1 opens:0 downloaders: o: fd: metaDirty:false modified:false info:{ModTime:{wall:13816705719471334908 ext:54338072535 loc:0x28e63c0} ATime:{wall:13816705719471521830 ext:54338259461 loc:0x28e63c0} Size:0 Rs:[] Fingerprint: Dirty:true} writeBackID:1}, } 2020/07/24 10:42:41 DEBUG : >WaitForWriters: 2020/07/24 10:42:44 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:44 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram76A3C34878F715D7EZkeaE5K062Nqw5mh2pidn7gBIvXt3HUJUcdupBvYhP6MV/8ZtbldDnKMiR5qmIZCFJzdVrYRRp8= 2020/07/24 10:42:44 ERROR : file1: vfs cache: failed to upload try #7, will retry in 12.8s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram76A3C34878F715D7EZkeaE5K062Nqw5mh2pidn7gBIvXt3HUJUcdupBvYhP6MV/8ZtbldDnKMiR5qmIZCFJzdVrYRRp8= 2020/07/24 10:42:44 DEBUG : file2: vfs cache: starting upload 2020/07/24 10:42:44 ERROR : file2: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7346960AF37D9A8F8U0B2LZLTT2QQ4adv76aERRzMzgg4Hy4klM9Vqia8rUKyEQysEqJGzAFWebPRxx6L3e07E3PJwtM= 2020/07/24 10:42:44 ERROR : file2: vfs cache: failed to upload try #7, will retry in 12.8s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7346960AF37D9A8F8U0B2LZLTT2QQ4adv76aERRzMzgg4Hy4klM9Vqia8rUKyEQysEqJGzAFWebPRxx6L3e07E3PJwtM= 2020/07/24 10:42:48 DEBUG : WaitForWriters: timeout=10s 2020/07/24 10:42:48 DEBUG : : Looking for writers 2020/07/24 10:42:48 DEBUG : file1: reading active writers 2020/07/24 10:42:48 DEBUG : file2: reading active writers 2020/07/24 10:42:48 DEBUG : Still 0 writers active and 2 cache items in use, waiting 10ms 2020/07/24 10:42:48 DEBUG : : Looking for writers 2020/07/24 10:42:48 DEBUG : file1: reading active writers 2020/07/24 10:42:48 DEBUG : file2: reading active writers 2020/07/24 10:42:48 DEBUG : Still 0 writers active and 2 cache items in use, waiting 20ms 2020/07/24 10:42:48 DEBUG : : Looking for writers 2020/07/24 10:42:48 DEBUG : file1: reading active writers 2020/07/24 10:42:48 DEBUG : file2: reading active writers 2020/07/24 10:42:48 DEBUG : Still 0 writers active and 2 cache items in use, waiting 40ms 2020/07/24 10:42:48 DEBUG : : Looking for writers 2020/07/24 10:42:48 DEBUG : file1: reading active writers 2020/07/24 10:42:48 DEBUG : file2: reading active writers 2020/07/24 10:42:48 DEBUG : Still 0 writers active and 2 cache items in use, waiting 80ms 2020/07/24 10:42:48 DEBUG : : Looking for writers 2020/07/24 10:42:48 DEBUG : file1: reading active writers 2020/07/24 10:42:48 DEBUG : file2: reading active writers 2020/07/24 10:42:48 DEBUG : Still 0 writers active and 2 cache items in use, waiting 160ms 2020/07/24 10:42:48 DEBUG : : Looking for writers 2020/07/24 10:42:48 DEBUG : file1: reading active writers 2020/07/24 10:42:48 DEBUG : file2: reading active writers 2020/07/24 10:42:48 DEBUG : Still 0 writers active and 2 cache items in use, waiting 320ms 2020/07/24 10:42:49 DEBUG : : Looking for writers 2020/07/24 10:42:49 DEBUG : file1: reading active writers 2020/07/24 10:42:49 DEBUG : file2: reading active writers 2020/07/24 10:42:49 DEBUG : Still 0 writers active and 2 cache items in use, waiting 640ms 2020/07/24 10:42:49 DEBUG : : Looking for writers 2020/07/24 10:42:49 DEBUG : file1: reading active writers 2020/07/24 10:42:49 DEBUG : file2: reading active writers 2020/07/24 10:42:49 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:50 DEBUG : : Looking for writers 2020/07/24 10:42:50 DEBUG : file1: reading active writers 2020/07/24 10:42:50 DEBUG : file2: reading active writers 2020/07/24 10:42:50 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:51 DEBUG : : Looking for writers 2020/07/24 10:42:51 DEBUG : file2: reading active writers 2020/07/24 10:42:51 DEBUG : file1: reading active writers 2020/07/24 10:42:51 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:52 DEBUG : : Looking for writers 2020/07/24 10:42:52 DEBUG : file1: reading active writers 2020/07/24 10:42:52 DEBUG : file2: reading active writers 2020/07/24 10:42:52 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:53 DEBUG : : Looking for writers 2020/07/24 10:42:53 DEBUG : file1: reading active writers 2020/07/24 10:42:53 DEBUG : file2: reading active writers 2020/07/24 10:42:53 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:54 DEBUG : : Looking for writers 2020/07/24 10:42:54 DEBUG : file1: reading active writers 2020/07/24 10:42:54 DEBUG : file2: reading active writers 2020/07/24 10:42:54 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:55 DEBUG : : Looking for writers 2020/07/24 10:42:55 DEBUG : file1: reading active writers 2020/07/24 10:42:55 DEBUG : file2: reading active writers 2020/07/24 10:42:55 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:56 DEBUG : : Looking for writers 2020/07/24 10:42:56 DEBUG : file1: reading active writers 2020/07/24 10:42:56 DEBUG : file2: reading active writers 2020/07/24 10:42:56 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:57 DEBUG : file1: vfs cache: starting upload 2020/07/24 10:42:57 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7A5C13D82CE9036B1aZwalQPvleWCmeBCCOjUyTMe4ZGeS5201QamEMAYwTo8foQ/o8NoPt13h89hVMbIcjchVkuZDOY= 2020/07/24 10:42:57 ERROR : file1: vfs cache: failed to upload try #8, will retry in 25.6s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7A5C13D82CE9036B1aZwalQPvleWCmeBCCOjUyTMe4ZGeS5201QamEMAYwTo8foQ/o8NoPt13h89hVMbIcjchVkuZDOY= 2020/07/24 10:42:57 DEBUG : file2: vfs cache: starting upload 2020/07/24 10:42:57 ERROR : file2: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram75EB3682CC38BF22An8aViJH/pSEQx/W2y+pqE/3frhJ5kIcITVVhykjt6dOIHRlyDzwXR6iGt9un2G66yjkI97G6C3o= 2020/07/24 10:42:57 ERROR : file2: vfs cache: failed to upload try #8, will retry in 25.6s: vfs cache: failed to transfer file from cache to remote: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram75EB3682CC38BF22An8aViJH/pSEQx/W2y+pqE/3frhJ5kIcITVVhykjt6dOIHRlyDzwXR6iGt9un2G66yjkI97G6C3o= 2020/07/24 10:42:57 DEBUG : : Looking for writers 2020/07/24 10:42:57 DEBUG : file1: reading active writers 2020/07/24 10:42:57 DEBUG : file2: reading active writers 2020/07/24 10:42:57 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2020/07/24 10:42:58 ERROR : Exiting even though 0 writers active and 2 cache items in use after 10s Cache{ "file1": &{c:0xc0001c6fc0 mu:{state:0 sema:0} name:file1 opens:0 downloaders: o: fd: metaDirty:false modified:false info:{ModTime:{wall:13816705719471334908 ext:54338072535 loc:0x28e63c0} ATime:{wall:13816705719471521830 ext:54338259461 loc:0x28e63c0} Size:0 Rs:[] Fingerprint: Dirty:true} writeBackID:1}, "file2": &{c:0xc0001c6fc0 mu:{state:0 sema:0} name:file2 opens:0 downloaders: o: fd: metaDirty:false modified:false info:{ModTime:{wall:13816705719472099854 ext:54338837479 loc:0x28e63c0} ATime:{wall:13816705719472223974 ext:54338961607 loc:0x28e63c0} Size:0 Rs:[] Fingerprint: Dirty:true} writeBackID:2}, } 2020/07/24 10:42:58 DEBUG : >WaitForWriters: 2020/07/24 10:42:58 DEBUG : vfs cache: cleaner exiting --- FAIL: TestRWFileHandleWriteNoWrite (27.03s) run.go:176: Remote "Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7'", Local "Local file system at /tmp/rclone577212835", Modify Window "1ns" fstest.go:295: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:295: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:295: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:302: Error Trace: fstest.go:302 fstest.go:335 read_write_test.go:426 Error: Should be true Test: TestRWFileHandleWriteNoWrite Messages: listing wrong, want file1 (0), file2 (0) got fstest.go:200: Not found "file2" fstest.go:200: Not found "file1" fstest.go:203: Error Trace: fstest.go:203 fstest.go:307 fstest.go:335 read_write_test.go:426 Error: Not equal: expected: 0 actual : 2 Test: TestRWFileHandleWriteNoWrite Messages: 2 objects not found === RUN TestWriteFileHandleMethods 2020/07/24 10:42:58 INFO : Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7': poll-interval is not supported by this remote 2020/07/24 10:42:58 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2020/07/24 10:42:58 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2020/07/24 10:42:58 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:42:58 DEBUG : file1: >Open: fd=file1 (w), err= 2020/07/24 10:42:58 DEBUG : file1: >OpenFile: fd=file1 (w), err= 2020/07/24 10:42:58 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:42:58 ERROR : file1: WriteFileHandle: Read: Can't read and write to file without --vfs-cache-mode >= minimal 2020/07/24 10:42:58 ERROR : file1: WriteFileHandle: ReadAt: Can't read and write to file without --vfs-cache-mode >= minimal 2020/07/24 10:42:58 ERROR : file1: WriteFileHandle: Truncate: Can't change size without --vfs-cache-mode >= writes 2020/07/24 10:42:58 DEBUG : Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7': File to upload is small (5 bytes), uploading instead of streaming 2020/07/24 10:42:58 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram719EE4773F6660770x21i4wPYZxavC4K2SZPW+ThqzXoSd6VsuXIa3Ml8cG6EZvrMePZWfEDHiAqYKPTurKgpJ3pZSMc= 2020/07/24 10:42:58 ERROR : file1: WriteFileHandle.New Rcat failed: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram719EE4773F6660770x21i4wPYZxavC4K2SZPW+ThqzXoSd6VsuXIa3Ml8cG6EZvrMePZWfEDHiAqYKPTurKgpJ3pZSMc= 2020/07/24 10:43:05 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2020/07/24 10:43:05 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2020/07/24 10:43:05 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:43:05 DEBUG : file1: >Open: fd=file1 (w), err= 2020/07/24 10:43:05 DEBUG : file1: >OpenFile: fd=file1 (w), err= 2020/07/24 10:43:05 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:43:05 DEBUG : Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7': File to upload is small (0 bytes), uploading instead of streaming 2020/07/24 10:43:05 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7E5AA7042AC881E03ujcHG7PIo06vA/MEI7GJPYNmtIFNcvIfGk4bnJK+WTXKQ1gILHshaVpQ9GS5zwSFtxz3RmR91j8= 2020/07/24 10:43:05 ERROR : file1: WriteFileHandle.New Rcat failed: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7E5AA7042AC881E03ujcHG7PIo06vA/MEI7GJPYNmtIFNcvIfGk4bnJK+WTXKQ1gILHshaVpQ9GS5zwSFtxz3RmR91j8= 2020/07/24 10:43:05 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2020/07/24 10:43:05 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2020/07/24 10:43:05 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:43:05 DEBUG : file1: >Open: fd=file1 (w), err= 2020/07/24 10:43:05 DEBUG : file1: >OpenFile: fd=file1 (w), err= 2020/07/24 10:43:05 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:43:05 DEBUG : WaitForWriters: timeout=10s 2020/07/24 10:43:05 DEBUG : : Looking for writers 2020/07/24 10:43:05 DEBUG : file1: reading active writers 2020/07/24 10:43:05 DEBUG : file1: active writers 1 2020/07/24 10:43:05 DEBUG : Still 1 writers active and 0 cache items in use, waiting 10ms 2020/07/24 10:43:05 DEBUG : : Looking for writers 2020/07/24 10:43:05 DEBUG : file1: reading active writers 2020/07/24 10:43:05 DEBUG : file1: active writers 1 2020/07/24 10:43:05 DEBUG : Still 1 writers active and 0 cache items in use, waiting 20ms 2020/07/24 10:43:05 DEBUG : : Looking for writers 2020/07/24 10:43:05 DEBUG : file1: reading active writers 2020/07/24 10:43:05 DEBUG : file1: active writers 1 2020/07/24 10:43:05 DEBUG : Still 1 writers active and 0 cache items in use, waiting 40ms 2020/07/24 10:43:05 DEBUG : : Looking for writers 2020/07/24 10:43:05 DEBUG : file1: reading active writers 2020/07/24 10:43:05 DEBUG : file1: active writers 1 2020/07/24 10:43:05 DEBUG : Still 1 writers active and 0 cache items in use, waiting 80ms 2020/07/24 10:43:05 DEBUG : : Looking for writers 2020/07/24 10:43:05 DEBUG : file1: reading active writers 2020/07/24 10:43:05 DEBUG : file1: active writers 1 2020/07/24 10:43:05 DEBUG : Still 1 writers active and 0 cache items in use, waiting 160ms 2020/07/24 10:43:05 DEBUG : : Looking for writers 2020/07/24 10:43:05 DEBUG : file1: reading active writers 2020/07/24 10:43:05 DEBUG : file1: active writers 1 2020/07/24 10:43:05 DEBUG : Still 1 writers active and 0 cache items in use, waiting 320ms 2020/07/24 10:43:06 DEBUG : : Looking for writers 2020/07/24 10:43:06 DEBUG : file1: reading active writers 2020/07/24 10:43:06 DEBUG : file1: active writers 1 2020/07/24 10:43:06 DEBUG : Still 1 writers active and 0 cache items in use, waiting 640ms 2020/07/24 10:43:06 DEBUG : : Looking for writers 2020/07/24 10:43:06 DEBUG : file1: reading active writers 2020/07/24 10:43:06 DEBUG : file1: active writers 1 2020/07/24 10:43:06 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2020/07/24 10:43:07 DEBUG : : Looking for writers 2020/07/24 10:43:07 DEBUG : file1: reading active writers 2020/07/24 10:43:07 DEBUG : file1: active writers 1 2020/07/24 10:43:07 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2020/07/24 10:43:08 DEBUG : : Looking for writers 2020/07/24 10:43:08 DEBUG : file1: reading active writers 2020/07/24 10:43:08 DEBUG : file1: active writers 1 2020/07/24 10:43:08 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2020/07/24 10:43:09 DEBUG : : Looking for writers 2020/07/24 10:43:09 DEBUG : file1: reading active writers 2020/07/24 10:43:09 DEBUG : file1: active writers 1 2020/07/24 10:43:09 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2020/07/24 10:43:10 DEBUG : : Looking for writers 2020/07/24 10:43:10 DEBUG : file1: reading active writers 2020/07/24 10:43:10 DEBUG : file1: active writers 1 2020/07/24 10:43:10 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2020/07/24 10:43:11 DEBUG : : Looking for writers 2020/07/24 10:43:11 DEBUG : file1: reading active writers 2020/07/24 10:43:11 DEBUG : file1: active writers 1 2020/07/24 10:43:11 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2020/07/24 10:43:12 DEBUG : : Looking for writers 2020/07/24 10:43:12 DEBUG : file1: reading active writers 2020/07/24 10:43:12 DEBUG : file1: active writers 1 2020/07/24 10:43:12 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2020/07/24 10:43:13 DEBUG : : Looking for writers 2020/07/24 10:43:13 DEBUG : file1: reading active writers 2020/07/24 10:43:13 DEBUG : file1: active writers 1 2020/07/24 10:43:13 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2020/07/24 10:43:14 DEBUG : : Looking for writers 2020/07/24 10:43:14 DEBUG : file1: reading active writers 2020/07/24 10:43:14 DEBUG : file1: active writers 1 2020/07/24 10:43:14 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2020/07/24 10:43:15 ERROR : Exiting even though 1 writers active and 0 cache items in use after 10s Cache: 2020/07/24 10:43:15 DEBUG : >WaitForWriters: --- FAIL: TestWriteFileHandleMethods (17.15s) run.go:176: Remote "Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7'", Local "Local file system at /tmp/rclone577212835", Modify Window "1ns" write_test.go:84: Error Trace: write_test.go:84 Error: Received unexpected error: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram719EE4773F6660770x21i4wPYZxavC4K2SZPW+ThqzXoSd6VsuXIa3Ml8cG6EZvrMePZWfEDHiAqYKPTurKgpJ3pZSMc= Test: TestWriteFileHandleMethods fstest.go:295: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:295: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:295: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:302: Error Trace: fstest.go:302 fstest.go:335 write_test.go:97 Error: Should be true Test: TestWriteFileHandleMethods Messages: listing wrong, want file1 (5) got fstest.go:200: Not found "file1" fstest.go:203: Error Trace: fstest.go:203 fstest.go:307 fstest.go:335 write_test.go:97 Error: Not equal: expected: 0 actual : 1 Test: TestWriteFileHandleMethods Messages: 1 objects not found write_test.go:103: Error Trace: write_test.go:103 Error: Received unexpected error: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7E5AA7042AC881E03ujcHG7PIo06vA/MEI7GJPYNmtIFNcvIfGk4bnJK+WTXKQ1gILHshaVpQ9GS5zwSFtxz3RmR91j8= Test: TestWriteFileHandleMethods dir_test.go:252: Error Trace: dir_test.go:252 write_test.go:104 Error: Not equal: expected: []string{"file1,5,false"} actual : []string{"file1,0,false"} Diff: --- Expected +++ Actual @@ -1,3 +1,3 @@ ([]string) (len=1) { - (string) (len=13) "file1,5,false" + (string) (len=13) "file1,0,false" } Test: TestWriteFileHandleMethods write_test.go:111: Error Trace: write_test.go:111 Error: Not equal: expected: *errors.errorString(&errors.errorString{s:"permission denied"}) actual : () Test: TestWriteFileHandleMethods === RUN TestWriteFileHandleWriteAt 2020/07/24 10:43:15 INFO : Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7': poll-interval is not supported by this remote 2020/07/24 10:43:15 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2020/07/24 10:43:15 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2020/07/24 10:43:15 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:43:15 DEBUG : file1: >Open: fd=file1 (w), err= 2020/07/24 10:43:15 DEBUG : file1: >OpenFile: fd=file1 (w), err= 2020/07/24 10:43:15 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:43:15 DEBUG : file1: waiting for in-sequence write to 100 for 1s 2020/07/24 10:43:16 DEBUG : file1: aborting in-sequence write wait, off=100 2020/07/24 10:43:16 DEBUG : file1: failed to wait for in-sequence write to 100 2020/07/24 10:43:16 ERROR : file1: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes 2020/07/24 10:43:16 DEBUG : Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7': File to upload is small (11 bytes), uploading instead of streaming 2020/07/24 10:43:16 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram71717499D661E0268wOsgpbDrnrXggRqIEMgkKmnnv13gCsOT5LS8H0eJMSDATtHug3OEps1uaVHGnrGcAxM2XsphVTg= 2020/07/24 10:43:16 ERROR : file1: WriteFileHandle.New Rcat failed: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram71717499D661E0268wOsgpbDrnrXggRqIEMgkKmnnv13gCsOT5LS8H0eJMSDATtHug3OEps1uaVHGnrGcAxM2XsphVTg= 2020/07/24 10:43:16 ERROR : file1: WriteFileHandle.Write: error: Bad file descriptor 2020/07/24 10:43:23 DEBUG : WaitForWriters: timeout=10s 2020/07/24 10:43:23 DEBUG : : Looking for writers 2020/07/24 10:43:23 DEBUG : file1: reading active writers 2020/07/24 10:43:23 DEBUG : >WaitForWriters: --- FAIL: TestWriteFileHandleWriteAt (8.09s) run.go:176: Remote "Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7'", Local "Local file system at /tmp/rclone577212835", Modify Window "1ns" write_test.go:162: Error Trace: write_test.go:162 Error: Received unexpected error: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram71717499D661E0268wOsgpbDrnrXggRqIEMgkKmnnv13gCsOT5LS8H0eJMSDATtHug3OEps1uaVHGnrGcAxM2XsphVTg= Test: TestWriteFileHandleWriteAt fstest.go:295: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:295: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:295: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:302: Error Trace: fstest.go:302 fstest.go:335 write_test.go:176 Error: Should be true Test: TestWriteFileHandleWriteAt Messages: listing wrong, want file1 (11) got fstest.go:200: Not found "file1" fstest.go:203: Error Trace: fstest.go:203 fstest.go:307 fstest.go:335 write_test.go:176 Error: Not equal: expected: 0 actual : 1 Test: TestWriteFileHandleWriteAt Messages: 1 objects not found === RUN TestWriteFileHandleFlush 2020/07/24 10:43:23 INFO : Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7': poll-interval is not supported by this remote 2020/07/24 10:43:23 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2020/07/24 10:43:23 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2020/07/24 10:43:23 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:43:23 DEBUG : file1: >Open: fd=file1 (w), err= 2020/07/24 10:43:23 DEBUG : file1: >OpenFile: fd=file1 (w), err= 2020/07/24 10:43:23 DEBUG : file1: WriteFileHandle.Flush unwritten handle, writing 0 bytes to avoid race conditions 2020/07/24 10:43:23 DEBUG : : Added virtual directory entry vAdd: "file1" 2020/07/24 10:43:23 DEBUG : Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7': File to upload is small (5 bytes), uploading instead of streaming 2020/07/24 10:43:23 ERROR : file1: Failed to copy: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7B392576C37882F29/D5EjD2qO0rR+63FQXME3UdA2XlyE0iEMyfE2yeQwTffxJIp/yZgyOkJFzx7UVjyzfa8gHmgjvE= 2020/07/24 10:43:23 ERROR : file1: WriteFileHandle.New Rcat failed: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7B392576C37882F29/D5EjD2qO0rR+63FQXME3UdA2XlyE0iEMyfE2yeQwTffxJIp/yZgyOkJFzx7UVjyzfa8gHmgjvE= 2020/07/24 10:43:23 ERROR : file1: WriteFileHandle.Flush error: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7B392576C37882F29/D5EjD2qO0rR+63FQXME3UdA2XlyE0iEMyfE2yeQwTffxJIp/yZgyOkJFzx7UVjyzfa8gHmgjvE= 2020/07/24 10:43:23 DEBUG : file1: WriteFileHandle.Flush nothing to do 2020/07/24 10:43:23 DEBUG : WaitForWriters: timeout=10s 2020/07/24 10:43:23 DEBUG : : Looking for writers 2020/07/24 10:43:23 DEBUG : file1: reading active writers 2020/07/24 10:43:23 DEBUG : >WaitForWriters: --- FAIL: TestWriteFileHandleFlush (0.07s) run.go:176: Remote "Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7'", Local "Local file system at /tmp/rclone577212835", Modify Window "1ns" write_test.go:198: Error Trace: write_test.go:198 Error: Received unexpected error: s3 upload: 404 Not Found: NoSuchBucketThe specified bucket does not existrclone-test-lubaxip2wawolek7yiruram7B392576C37882F29/D5EjD2qO0rR+63FQXME3UdA2XlyE0iEMyfE2yeQwTffxJIp/yZgyOkJFzx7UVjyzfa8gHmgjvE= Test: TestWriteFileHandleFlush FAIL 2020/07/24 10:43:23 DEBUG : Chunked 'TestChunkerChunk50bMD5HashS3:rclone-test-lubaxip2wawolek7yiruram7': Purge dir "" 2020/07/24 10:43:23 purge failed to rmdir "": NoSuchBucket: The specified bucket does not exist status code: 404, request id: 1F8A9E1723C3CCB4, host id: pJX3Q6RIOT+sT5xdELJf+hJHBX28iq/hAWVTnRodpTbngLMxLtEw8asWE0l0nJMZufuxmytryso= 2020/07/24 10:43:23 purge failed: directory not found "./vfs.test -test.v -test.timeout 1h0m0s -remote TestChunkerChunk50bMD5HashS3: -verbose -size-limit 1024 -test.run '^(TestRWFileHandleMethodsWrite|TestRWFileHandleWriteAt|TestRWFileHandleWriteNoWrite|TestWriteFileHandleFlush|TestWriteFileHandleMethods|TestWriteFileHandleWriteAt)$'" - Finished ERROR in 1m46.70562637s (try 5/5): exit status 1: Failed [TestRWFileHandleMethodsWrite TestRWFileHandleWriteAt TestRWFileHandleWriteNoWrite TestWriteFileHandleMethods TestWriteFileHandleWriteAt TestWriteFileHandleFlush]