"./vfs.test -test.v -test.timeout 2h0m0s -remote TestFileFabric: -verbose -test.run '^(TestDirCreate|TestDirForgetAll|TestDirForgetPath|TestDirHandleMethods|TestDirHandleReaddir|TestDirHandleReaddirnames|TestDirMethods|TestDirMkdir|TestDirMkdirSub|TestDirOpen|TestDirReadDirAll|TestDirRemove|TestDirRemoveAll|TestDirRemoveName|TestDirRename|TestDirSetModTime|TestDirStat|TestDirWalk|TestFileMethods|TestFileOpen|TestFileOpenRead|TestFileOpenWrite|TestFileReadAtNonZeroLength|TestFileReadAtZeroLength|TestFileRemove|TestFileRemoveAll|TestFileRename|TestFileSetModTime|TestRWFileHandleFlushRead|TestRWFileHandleMethodsRead|TestRWFileHandleMethodsWrite|TestRWFileHandleReadAt|TestRWFileHandleReleaseRead|TestRWFileHandleSeek|TestRWFileHandleSizeCreateExisting|TestRWFileHandleSizeTruncateExisting|TestRWFileHandleWriteAt|TestRWFileHandleWriteNoWrite|TestRWFileModTimeWithOpenWriters|TestRcGetVFS|TestReadFileHandleFlush|TestReadFileHandleMethods|TestReadFileHandleReadAt|TestReadFileHandleRelease|TestReadFileHandleSeek|TestVFSNew|TestVFSOpenFile|TestVFSRename|TestVFSStat|TestVFSStatParent|TestWriteFileHandleFlush|TestWriteFileHandleMethods|TestWriteFileHandleRelease|TestWriteFileHandleWriteAt|TestWriteFileModTimeWithOpenWriters)$/^(cache=full,open=false,write=false|cache=full,open=true,write=false|cache=full,open=true,write=true|cache=off,open=false,write=false|cache=off,open=true,write=false|cache=off,open=true,write=true|full,forceCache=false|minimal,forceCache=false|minimal,forceCache=true|off,forceCache=false|writes,forceCache=false|writes,forceCache=true)$'" - Starting (try 2/5) 2022/05/18 06:11:11 DEBUG : Creating backend with remote "TestFileFabric:rclone-test-guyumar1gucuroj1gejusov5" 2022/05/18 06:11:11 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2022/05/18 06:11:11 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: checkPathExists(map[path:rclone-test-guyumar1gucuroj1gejusov5 pid:120673762]) options=[] 2022/05/18 06:11:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:12 DEBUG : Creating backend with remote "/tmp/rclone24145073" === RUN TestDirHandleMethods run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:12 INFO : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': poll-interval is not supported by this remote 2022/05/18 06:11:12 NOTICE: filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': --vfs-cache-mode writes or full is recommended for this remote as it can't stream 2022/05/18 06:11:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:14 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirHandleMethods (2.18s) === RUN TestDirHandleReaddir run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:14 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:14 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:14 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:14 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:16 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:11:16 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:11:16 DEBUG : : Looking for writers 2022/05/18 06:11:16 DEBUG : >WaitForWriters: 2022/05/18 06:11:16 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:17 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:17 ERROR : : error listing: directory not found --- FAIL: TestDirHandleReaddir (2.70s) === RUN TestDirHandleReaddirnames run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:17 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:17 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:17 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:17 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:19 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirHandleReaddirnames (2.07s) === RUN TestDirMethods run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:19 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:19 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:19 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:19 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:21 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirMethods (2.07s) === RUN TestDirForgetAll run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:21 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:21 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:21 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:21 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:23 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirForgetAll (2.07s) === RUN TestDirForgetPath run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:23 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:23 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:23 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:23 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirForgetPath (2.10s) === RUN TestDirWalk run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:27 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirWalk (2.10s) === RUN TestDirSetModTime run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:27 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:27 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:28 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:28 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:29 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirSetModTime (2.05s) === RUN TestDirStat run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:29 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:29 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:30 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:30 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:31 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirStat (2.14s) === RUN TestDirReadDirAll run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:31 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:31 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:32 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:32 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:33 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:11:33 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:11:33 DEBUG : : Looking for writers 2022/05/18 06:11:33 DEBUG : >WaitForWriters: 2022/05/18 06:11:33 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:34 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:34 ERROR : : error listing: directory not found --- FAIL: TestDirReadDirAll (2.70s) === RUN TestDirOpen run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:34 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:34 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:34 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:34 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:36 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirOpen (2.11s) === RUN TestDirCreate run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:36 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:36 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:37 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:37 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:38 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirCreate (2.03s) === RUN TestDirMkdir run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:38 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:38 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:39 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:39 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:40 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirMkdir (2.08s) === RUN TestDirMkdirSub run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:40 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:40 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:41 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:41 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:42 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirMkdirSub (2.10s) === RUN TestDirRemove run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:42 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:42 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:43 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:43 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirRemove (2.07s) === RUN TestDirRemoveAll run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:45 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:45 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:46 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirRemoveAll (2.09s) === RUN TestDirRemoveName run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:46 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:46 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:47 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:47 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:48 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirRemoveName (2.13s) === RUN TestDirRename run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:48 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:48 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:49 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:49 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:50 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestDirRename (2.04s) === RUN TestFileMethods run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:50 INFO : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': poll-interval is not supported by this remote 2022/05/18 06:11:50 NOTICE: filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': --vfs-cache-mode writes or full is recommended for this remote as it can't stream 2022/05/18 06:11:50 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:51 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:51 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:53 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestFileMethods (2.13s) === RUN TestFileSetModTime === RUN TestFileSetModTime/cache=off,open=false,write=false run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:53 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:53 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:53 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:53 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:55 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) === RUN TestFileSetModTime/cache=off,open=true,write=false run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:55 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:55 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:55 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:55 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:57 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) === RUN TestFileSetModTime/cache=off,open=true,write=true run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:57 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:11:57 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:57 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:57 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:11:59 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) === RUN TestFileSetModTime/cache=full,open=false,write=false run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:11:59 INFO : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': poll-interval is not supported by this remote 2022/05/18 06:11:59 DEBUG : vfs cache: root is "/home/rclone/.cache/rclone" 2022/05/18 06:11:59 DEBUG : vfs cache: data root is "/home/rclone/.cache/rclone/vfs/TestFileFabric/rclone-test-guyumar1gucuroj1gejusov5" 2022/05/18 06:11:59 DEBUG : vfs cache: metadata root is "/home/rclone/.cache/rclone/vfsMeta/TestFileFabric/rclone-test-guyumar1gucuroj1gejusov5" 2022/05/18 06:11:59 DEBUG : Creating backend with remote "/home/rclone/.cache/rclone/vfs/TestFileFabric/rclone-test-guyumar1gucuroj1gejusov5" 2022/05/18 06:11:59 DEBUG : Config file has changed externaly - reloading 2022/05/18 06:11:59 DEBUG : Creating backend with remote "/home/rclone/.cache/rclone/vfsMeta/TestFileFabric/rclone-test-guyumar1gucuroj1gejusov5" 2022/05/18 06:11:59 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:11:59 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2022/05/18 06:11:59 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:11:59 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:01 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) === RUN TestFileSetModTime/cache=full,open=true,write=false run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:01 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:01 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:02 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:02 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:03 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) === RUN TestFileSetModTime/cache=full,open=true,write=true run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:03 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:03 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:04 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:04 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:05 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestFileSetModTime (12.76s) --- FAIL: TestFileSetModTime/cache=off,open=false,write=false (2.08s) --- FAIL: TestFileSetModTime/cache=off,open=true,write=false (2.13s) --- FAIL: TestFileSetModTime/cache=off,open=true,write=true (2.08s) --- FAIL: TestFileSetModTime/cache=full,open=false,write=false (2.15s) --- FAIL: TestFileSetModTime/cache=full,open=true,write=false (2.24s) --- FAIL: TestFileSetModTime/cache=full,open=true,write=true (2.07s) === RUN TestFileOpenRead run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:05 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:05 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:06 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:06 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:07 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestFileOpenRead (2.06s) === RUN TestFileOpenWrite run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:07 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:07 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:08 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:08 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:09 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestFileOpenWrite (2.07s) === RUN TestFileRemove run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:09 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:09 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:10 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:10 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestFileRemove (2.04s) === RUN TestFileRemoveAll run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:14 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestFileRemoveAll (2.13s) === RUN TestFileOpen run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:14 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:14 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:14 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:14 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:16 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestFileOpen (2.03s) === RUN TestFileRename === RUN TestFileRename/off,forceCache=false run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:16 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:16 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:16 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:16 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:18 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) === RUN TestFileRename/minimal,forceCache=false run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:18 INFO : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': poll-interval is not supported by this remote 2022/05/18 06:12:18 NOTICE: filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': --vfs-cache-mode writes or full is recommended for this remote as it can't stream 2022/05/18 06:12:18 DEBUG : vfs cache: root is "/home/rclone/.cache/rclone" 2022/05/18 06:12:18 DEBUG : vfs cache: data root is "/home/rclone/.cache/rclone/vfs/TestFileFabric/rclone-test-guyumar1gucuroj1gejusov5" 2022/05/18 06:12:18 DEBUG : vfs cache: metadata root is "/home/rclone/.cache/rclone/vfsMeta/TestFileFabric/rclone-test-guyumar1gucuroj1gejusov5" 2022/05/18 06:12:18 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:18 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2022/05/18 06:12:18 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:18 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:20 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) === RUN TestFileRename/minimal,forceCache=true run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:20 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:20 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:20 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:20 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:22 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) === RUN TestFileRename/writes,forceCache=false run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:22 INFO : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': poll-interval is not supported by this remote 2022/05/18 06:12:22 DEBUG : vfs cache: root is "/home/rclone/.cache/rclone" 2022/05/18 06:12:22 DEBUG : vfs cache: data root is "/home/rclone/.cache/rclone/vfs/TestFileFabric/rclone-test-guyumar1gucuroj1gejusov5" 2022/05/18 06:12:22 DEBUG : vfs cache: metadata root is "/home/rclone/.cache/rclone/vfsMeta/TestFileFabric/rclone-test-guyumar1gucuroj1gejusov5" 2022/05/18 06:12:22 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:22 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2022/05/18 06:12:23 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:23 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:24 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) === RUN TestFileRename/writes,forceCache=true run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:24 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:24 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:26 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) === RUN TestFileRename/full,forceCache=false run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:26 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:26 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:27 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:27 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:28 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestFileRename (12.46s) --- FAIL: TestFileRename/off,forceCache=false (2.08s) --- FAIL: TestFileRename/minimal,forceCache=false (2.06s) --- FAIL: TestFileRename/minimal,forceCache=true (2.08s) --- FAIL: TestFileRename/writes,forceCache=false (2.13s) --- FAIL: TestFileRename/writes,forceCache=true (2.06s) --- FAIL: TestFileRename/full,forceCache=false (2.05s) === RUN TestRcGetVFS rc_test.go:29: Error Trace: rc_test.go:29 Error: "more than one VFS active - need \"fs\" parameter" does not contain "no VFS active" Test: TestRcGetVFS run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:28 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache rc_test.go:36: Error Trace: rc_test.go:36 Error: Received unexpected error: more than one VFS active - need "fs" parameter Test: TestRcGetVFS 2022/05/18 06:12:28 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:12:28 DEBUG : : Looking for writers 2022/05/18 06:12:28 DEBUG : >WaitForWriters: 2022/05/18 06:12:28 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:29 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:29 ERROR : : error listing: directory not found --- FAIL: TestRcGetVFS (0.62s) === RUN TestReadFileHandleMethods run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:29 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:29 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:29 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:29 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:31 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestReadFileHandleMethods (2.09s) === RUN TestReadFileHandleSeek run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:31 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:31 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:31 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:31 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:33 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestReadFileHandleSeek (2.29s) === RUN TestReadFileHandleReadAt run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:33 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:33 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:34 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:34 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:35 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestReadFileHandleReadAt (2.22s) === RUN TestReadFileHandleFlush run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:35 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:35 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:36 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:36 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:37 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestReadFileHandleFlush (2.08s) === RUN TestReadFileHandleRelease run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:37 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:37 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:38 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:38 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:40 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestReadFileHandleRelease (2.10s) === RUN TestRWFileHandleMethodsRead run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:40 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:40 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:40 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:40 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:42 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestRWFileHandleMethodsRead (2.10s) === RUN TestRWFileHandleSeek run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:42 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:42 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:42 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:42 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestRWFileHandleSeek (2.09s) === RUN TestRWFileHandleReadAt run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:46 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestRWFileHandleReadAt (2.09s) === RUN TestRWFileHandleFlushRead run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:46 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:46 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:46 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:46 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:48 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestRWFileHandleFlushRead (2.10s) === RUN TestRWFileHandleReleaseRead run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:48 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:48 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:49 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:49 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:50 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestRWFileHandleReleaseRead (2.08s) === RUN TestRWFileHandleMethodsWrite run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:12:50 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:12:50 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2022/05/18 06:12:50 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:51 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:51 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2022/05/18 06:12:51 DEBUG : file1: newRWFileHandle: 2022/05/18 06:12:51 DEBUG : file1(0xc00077edc0): openPending: 2022/05/18 06:12:51 DEBUG : file1: vfs cache: truncate to size=0 2022/05/18 06:12:51 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:12:51 DEBUG : file1(0xc00077edc0): >openPending: err= 2022/05/18 06:12:51 DEBUG : file1: >newRWFileHandle: err= 2022/05/18 06:12:51 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:12:51 DEBUG : file1: >Open: fd=file1 (rw), err= 2022/05/18 06:12:51 DEBUG : file1: >OpenFile: fd=file1 (rw), err= 2022/05/18 06:12:51 DEBUG : file1(0xc00077edc0): _writeAt: size=5, off=0 2022/05/18 06:12:51 DEBUG : file1(0xc00077edc0): >_writeAt: n=5, err= 2022/05/18 06:12:51 DEBUG : file1(0xc00077edc0): _writeAt: size=7, off=5 2022/05/18 06:12:51 DEBUG : file1(0xc00077edc0): >_writeAt: n=7, err= 2022/05/18 06:12:51 DEBUG : file1: vfs cache: truncate to size=11 2022/05/18 06:12:51 DEBUG : file1(0xc00077edc0): close: 2022/05/18 06:12:51 DEBUG : file1: vfs cache: setting modification time to 2022-05-18 06:12:51.106067178 +0000 UTC m=+99.595169470 2022/05/18 06:12:51 INFO : file1: vfs cache: queuing for upload in 100ms 2022/05/18 06:12:51 DEBUG : file1(0xc00077edc0): >close: err= 2022/05/18 06:12:51 DEBUG : file1(0xc00077edc0): close: 2022/05/18 06:12:51 DEBUG : file1(0xc00077edc0): >close: err=file already closed 2022/05/18 06:12:51 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:12:51 DEBUG : : Looking for writers 2022/05/18 06:12:51 DEBUG : file1: reading active writers 2022/05/18 06:12:51 DEBUG : Still 0 writers active and 1 cache items in use, waiting 10ms 2022/05/18 06:12:51 DEBUG : : Looking for writers 2022/05/18 06:12:51 DEBUG : file1: reading active writers 2022/05/18 06:12:51 DEBUG : Still 0 writers active and 1 cache items in use, waiting 20ms 2022/05/18 06:12:51 DEBUG : : Looking for writers 2022/05/18 06:12:51 DEBUG : file1: reading active writers 2022/05/18 06:12:51 DEBUG : Still 0 writers active and 1 cache items in use, waiting 40ms 2022/05/18 06:12:51 DEBUG : : Looking for writers 2022/05/18 06:12:51 DEBUG : file1: reading active writers 2022/05/18 06:12:51 DEBUG : Still 0 writers active and 1 cache items in use, waiting 80ms 2022/05/18 06:12:51 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:12:51 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:51 DEBUG : : Looking for writers 2022/05/18 06:12:51 DEBUG : file1: reading active writers 2022/05/18 06:12:51 DEBUG : Still 0 writers active and 1 cache items in use, waiting 160ms 2022/05/18 06:12:51 DEBUG : : Looking for writers 2022/05/18 06:12:51 DEBUG : file1: reading active writers 2022/05/18 06:12:51 DEBUG : Still 0 writers active and 1 cache items in use, waiting 320ms 2022/05/18 06:12:51 DEBUG : : Looking for writers 2022/05/18 06:12:51 DEBUG : file1: reading active writers 2022/05/18 06:12:51 DEBUG : Still 0 writers active and 1 cache items in use, waiting 640ms 2022/05/18 06:12:51 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:51 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:52 DEBUG : : Looking for writers 2022/05/18 06:12:52 DEBUG : file1: reading active writers 2022/05/18 06:12:52 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:12:53 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:12:53 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:12:53 ERROR : file1: vfs cache: failed to upload try #1, will retry in 200ms: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:12:53 DEBUG : : Looking for writers 2022/05/18 06:12:53 DEBUG : file1: reading active writers 2022/05/18 06:12:53 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:12:53 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:12:53 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:54 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:54 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:54 DEBUG : : Looking for writers 2022/05/18 06:12:54 DEBUG : file1: reading active writers 2022/05/18 06:12:54 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:12:55 DEBUG : : Looking for writers 2022/05/18 06:12:55 DEBUG : file1: reading active writers 2022/05/18 06:12:55 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:12:55 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:12:55 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:12:55 ERROR : file1: vfs cache: failed to upload try #2, will retry in 400ms: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:12:55 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:12:55 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:56 DEBUG : : Looking for writers 2022/05/18 06:12:56 DEBUG : file1: reading active writers 2022/05/18 06:12:56 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:12:56 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:56 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:12:57 DEBUG : : Looking for writers 2022/05/18 06:12:57 DEBUG : file1: reading active writers 2022/05/18 06:12:57 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:12:58 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:12:58 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:12:58 ERROR : file1: vfs cache: failed to upload try #3, will retry in 800ms: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:12:58 DEBUG : : Looking for writers 2022/05/18 06:12:58 DEBUG : file1: reading active writers 2022/05/18 06:12:58 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:12:58 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:12:58 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:12:59 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item file1 not removed, freed 0 bytes 2022/05/18 06:12:59 INFO : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 0, uploading 1, total size 11 (was 11) 2022/05/18 06:12:59 DEBUG : : Looking for writers 2022/05/18 06:12:59 DEBUG : file1: reading active writers 2022/05/18 06:12:59 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:12:59 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:12:59 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:13:00 DEBUG : : Looking for writers 2022/05/18 06:13:00 DEBUG : file1: reading active writers 2022/05/18 06:13:00 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:00 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:13:00 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:13:00 ERROR : file1: vfs cache: failed to upload try #4, will retry in 1.6s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:13:01 DEBUG : : Looking for writers 2022/05/18 06:13:01 DEBUG : file1: reading active writers 2022/05/18 06:13:01 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:02 DEBUG : : Looking for writers 2022/05/18 06:13:02 DEBUG : file1: reading active writers 2022/05/18 06:13:02 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:02 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:13:02 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:13:03 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:13:03 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:13:03 DEBUG : : Looking for writers 2022/05/18 06:13:03 DEBUG : file1: reading active writers 2022/05/18 06:13:03 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:04 DEBUG : : Looking for writers 2022/05/18 06:13:04 DEBUG : file1: reading active writers 2022/05/18 06:13:04 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:04 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:13:04 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:13:04 ERROR : file1: vfs cache: failed to upload try #5, will retry in 3.2s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:13:05 DEBUG : : Looking for writers 2022/05/18 06:13:05 DEBUG : file1: reading active writers 2022/05/18 06:13:05 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:06 DEBUG : : Looking for writers 2022/05/18 06:13:06 DEBUG : file1: reading active writers 2022/05/18 06:13:06 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:07 DEBUG : : Looking for writers 2022/05/18 06:13:07 DEBUG : file1: reading active writers 2022/05/18 06:13:07 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:07 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:13:07 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:13:08 DEBUG : : Looking for writers 2022/05/18 06:13:08 DEBUG : file1: reading active writers 2022/05/18 06:13:08 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:08 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:13:08 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:13:09 DEBUG : : Looking for writers 2022/05/18 06:13:09 DEBUG : file1: reading active writers 2022/05/18 06:13:09 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:09 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:13:09 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:13:09 ERROR : file1: vfs cache: failed to upload try #6, will retry in 6.4s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:13:10 DEBUG : : Looking for writers 2022/05/18 06:13:10 DEBUG : file1: reading active writers 2022/05/18 06:13:10 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:11 DEBUG : : Looking for writers 2022/05/18 06:13:11 DEBUG : file1: reading active writers 2022/05/18 06:13:11 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:12 DEBUG : : Looking for writers 2022/05/18 06:13:12 DEBUG : file1: reading active writers 2022/05/18 06:13:12 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:13 DEBUG : : Looking for writers 2022/05/18 06:13:13 DEBUG : file1: reading active writers 2022/05/18 06:13:13 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:14 DEBUG : : Looking for writers 2022/05/18 06:13:14 DEBUG : file1: reading active writers 2022/05/18 06:13:14 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:15 DEBUG : : Looking for writers 2022/05/18 06:13:15 DEBUG : file1: reading active writers 2022/05/18 06:13:15 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:16 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:13:16 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:13:16 DEBUG : : Looking for writers 2022/05/18 06:13:16 DEBUG : file1: reading active writers 2022/05/18 06:13:16 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:16 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:13:16 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:13:17 DEBUG : : Looking for writers 2022/05/18 06:13:17 DEBUG : file1: reading active writers 2022/05/18 06:13:17 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:18 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2022/05/18 06:13:18 DEBUG : : Looking for writers 2022/05/18 06:13:18 DEBUG : file1: reading active writers 2022/05/18 06:13:18 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:18 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:13:18 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:13:18 ERROR : file1: vfs cache: failed to upload try #7, will retry in 12.8s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:13:19 DEBUG : : Looking for writers 2022/05/18 06:13:19 DEBUG : file1: reading active writers 2022/05/18 06:13:19 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:20 DEBUG : : Looking for writers 2022/05/18 06:13:20 DEBUG : file1: reading active writers 2022/05/18 06:13:20 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:21 ERROR : Exiting even though 0 writers active and 1 cache items in use after 30s Cache{ "file1": &{c:0xc0005f29c0 mu:{state:0 sema:0} cond:0xc00077ed80 name:file1 opens:0 downloaders: o: fd: modified:false info:{ModTime:{wall:13878195713698067690 ext:99595169470 loc:0x2b8b560} ATime:{wall:13878195713698087476 ext:99595189217 loc:0x2b8b560} Size:11 Rs:[{Pos:0 Size:11}] Fingerprint: Dirty:true} writeBackID:1 pendingAccesses:0 beingReset:false}, } 2022/05/18 06:13:21 DEBUG : >WaitForWriters: 2022/05/18 06:13:21 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:13:21 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:13:21 ERROR : : error listing: directory not found fstest.go:299: Sleeping for 1s for list eventual consistency: 1/3 2022/05/18 06:13:22 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) fstest.go:302: Flushing the directory cache 2022/05/18 06:13:22 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:13:23 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:13:23 ERROR : : error listing: directory not found fstest.go:299: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:302: Flushing the directory cache 2022/05/18 06:13:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:13:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:13:25 ERROR : : error listing: directory not found fstest.go:299: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:302: Flushing the directory cache fstest.go:306: Error Trace: fstest.go:306 fstest.go:339 read_write_test.go:344 Error: Should be true Test: TestRWFileHandleMethodsWrite Messages: listing wrong, want file1 (11) got fstest.go:204: Not found "file1" fstest.go:207: Error Trace: fstest.go:207 fstest.go:311 fstest.go:339 read_write_test.go:344 Error: Not equal: expected: 0 actual : 1 Test: TestRWFileHandleMethodsWrite Messages: 1 objects not found 2022/05/18 06:13:29 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:13:29 DEBUG : : Looking for writers 2022/05/18 06:13:29 DEBUG : file1: reading active writers 2022/05/18 06:13:29 DEBUG : Still 0 writers active and 1 cache items in use, waiting 10ms 2022/05/18 06:13:29 DEBUG : : Looking for writers 2022/05/18 06:13:29 DEBUG : file1: reading active writers 2022/05/18 06:13:29 DEBUG : Still 0 writers active and 1 cache items in use, waiting 20ms 2022/05/18 06:13:30 DEBUG : : Looking for writers 2022/05/18 06:13:30 DEBUG : file1: reading active writers 2022/05/18 06:13:30 DEBUG : Still 0 writers active and 1 cache items in use, waiting 40ms 2022/05/18 06:13:30 DEBUG : : Looking for writers 2022/05/18 06:13:30 DEBUG : file1: reading active writers 2022/05/18 06:13:30 DEBUG : Still 0 writers active and 1 cache items in use, waiting 80ms 2022/05/18 06:13:30 DEBUG : : Looking for writers 2022/05/18 06:13:30 DEBUG : file1: reading active writers 2022/05/18 06:13:30 DEBUG : Still 0 writers active and 1 cache items in use, waiting 160ms 2022/05/18 06:13:30 DEBUG : : Looking for writers 2022/05/18 06:13:30 DEBUG : file1: reading active writers 2022/05/18 06:13:30 DEBUG : Still 0 writers active and 1 cache items in use, waiting 320ms 2022/05/18 06:13:30 DEBUG : : Looking for writers 2022/05/18 06:13:30 DEBUG : file1: reading active writers 2022/05/18 06:13:30 DEBUG : Still 0 writers active and 1 cache items in use, waiting 640ms 2022/05/18 06:13:31 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:13:31 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:13:31 DEBUG : : Looking for writers 2022/05/18 06:13:31 DEBUG : file1: reading active writers 2022/05/18 06:13:31 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:31 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:13:31 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:13:32 DEBUG : : Looking for writers 2022/05/18 06:13:32 DEBUG : file1: reading active writers 2022/05/18 06:13:32 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:33 DEBUG : : Looking for writers 2022/05/18 06:13:33 DEBUG : file1: reading active writers 2022/05/18 06:13:33 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:33 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:13:33 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:13:33 ERROR : file1: vfs cache: failed to upload try #8, will retry in 25.6s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:13:34 DEBUG : : Looking for writers 2022/05/18 06:13:34 DEBUG : file1: reading active writers 2022/05/18 06:13:34 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:35 DEBUG : : Looking for writers 2022/05/18 06:13:35 DEBUG : file1: reading active writers 2022/05/18 06:13:35 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:36 DEBUG : : Looking for writers 2022/05/18 06:13:36 DEBUG : file1: reading active writers 2022/05/18 06:13:36 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:37 DEBUG : : Looking for writers 2022/05/18 06:13:37 DEBUG : file1: reading active writers 2022/05/18 06:13:37 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:38 DEBUG : : Looking for writers 2022/05/18 06:13:38 DEBUG : file1: reading active writers 2022/05/18 06:13:38 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:39 DEBUG : : Looking for writers 2022/05/18 06:13:39 DEBUG : file1: reading active writers 2022/05/18 06:13:39 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:40 DEBUG : : Looking for writers 2022/05/18 06:13:40 DEBUG : file1: reading active writers 2022/05/18 06:13:40 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:41 DEBUG : : Looking for writers 2022/05/18 06:13:41 DEBUG : file1: reading active writers 2022/05/18 06:13:41 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:42 DEBUG : : Looking for writers 2022/05/18 06:13:42 DEBUG : file1: reading active writers 2022/05/18 06:13:42 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:43 DEBUG : : Looking for writers 2022/05/18 06:13:43 DEBUG : file1: reading active writers 2022/05/18 06:13:43 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:44 DEBUG : : Looking for writers 2022/05/18 06:13:44 DEBUG : file1: reading active writers 2022/05/18 06:13:44 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:45 DEBUG : : Looking for writers 2022/05/18 06:13:45 DEBUG : file1: reading active writers 2022/05/18 06:13:45 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:46 DEBUG : : Looking for writers 2022/05/18 06:13:46 DEBUG : file1: reading active writers 2022/05/18 06:13:46 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:47 DEBUG : : Looking for writers 2022/05/18 06:13:47 DEBUG : file1: reading active writers 2022/05/18 06:13:47 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:48 DEBUG : : Looking for writers 2022/05/18 06:13:48 DEBUG : file1: reading active writers 2022/05/18 06:13:48 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:49 DEBUG : : Looking for writers 2022/05/18 06:13:49 DEBUG : file1: reading active writers 2022/05/18 06:13:49 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:50 DEBUG : : Looking for writers 2022/05/18 06:13:50 DEBUG : file1: reading active writers 2022/05/18 06:13:50 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:51 DEBUG : : Looking for writers 2022/05/18 06:13:51 DEBUG : file1: reading active writers 2022/05/18 06:13:51 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:52 DEBUG : : Looking for writers 2022/05/18 06:13:52 DEBUG : file1: reading active writers 2022/05/18 06:13:52 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:53 DEBUG : : Looking for writers 2022/05/18 06:13:53 DEBUG : file1: reading active writers 2022/05/18 06:13:53 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:54 DEBUG : : Looking for writers 2022/05/18 06:13:54 DEBUG : file1: reading active writers 2022/05/18 06:13:54 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:55 DEBUG : : Looking for writers 2022/05/18 06:13:55 DEBUG : file1: reading active writers 2022/05/18 06:13:55 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:56 DEBUG : : Looking for writers 2022/05/18 06:13:56 DEBUG : file1: reading active writers 2022/05/18 06:13:56 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:57 DEBUG : : Looking for writers 2022/05/18 06:13:57 DEBUG : file1: reading active writers 2022/05/18 06:13:57 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:58 DEBUG : : Looking for writers 2022/05/18 06:13:58 DEBUG : file1: reading active writers 2022/05/18 06:13:58 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:58 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:13:58 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:13:59 DEBUG : : Looking for writers 2022/05/18 06:13:59 DEBUG : file1: reading active writers 2022/05/18 06:13:59 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:13:59 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item file1 not removed, freed 0 bytes 2022/05/18 06:13:59 INFO : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 0, uploading 1, total size 11 (was 11) 2022/05/18 06:13:59 ERROR : Exiting even though 0 writers active and 1 cache items in use after 30s Cache{ "file1": &{c:0xc0005f29c0 mu:{state:0 sema:0} cond:0xc00077ed80 name:file1 opens:0 downloaders: o: fd: modified:false info:{ModTime:{wall:13878195713698067690 ext:99595169470 loc:0x2b8b560} ATime:{wall:13878195713698087476 ext:99595189217 loc:0x2b8b560} Size:11 Rs:[{Pos:0 Size:11}] Fingerprint: Dirty:true} writeBackID:1 pendingAccesses:0 beingReset:false}, } 2022/05/18 06:13:59 DEBUG : >WaitForWriters: 2022/05/18 06:14:00 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:14:00 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:14:01 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:01 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:14:01 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:01 ERROR : file1: vfs cache: failed to upload try #9, will retry in 51.2s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:02 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:14:02 ERROR : : error listing: directory not found --- FAIL: TestRWFileHandleMethodsWrite (71.56s) === RUN TestRWFileHandleWriteAt run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:14:02 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:14:02 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2022/05/18 06:14:02 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2022/05/18 06:14:02 DEBUG : file1: newRWFileHandle: 2022/05/18 06:14:02 DEBUG : file1(0xc0009e6a40): openPending: 2022/05/18 06:14:02 ERROR : file1: vfs cache: detected external removal of cache file 2022/05/18 06:14:02 DEBUG : file1: vfs cache: truncate to size=0 2022/05/18 06:14:02 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:14:02 DEBUG : file1(0xc0009e6a40): >openPending: err= 2022/05/18 06:14:02 DEBUG : file1: vfs cache: cancelling writeback (uploading false) 0xc00044ac40 item 1 2022/05/18 06:14:02 DEBUG : file1: >newRWFileHandle: err= 2022/05/18 06:14:02 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:14:02 DEBUG : file1: >Open: fd=file1 (rw), err= 2022/05/18 06:14:02 DEBUG : file1: >OpenFile: fd=file1 (rw), err= 2022/05/18 06:14:02 DEBUG : file1(0xc0009e6a40): _writeAt: size=7, off=0 2022/05/18 06:14:02 DEBUG : file1(0xc0009e6a40): >_writeAt: n=7, err= 2022/05/18 06:14:02 DEBUG : file1(0xc0009e6a40): _writeAt: size=6, off=5 2022/05/18 06:14:02 DEBUG : file1(0xc0009e6a40): >_writeAt: n=6, err= 2022/05/18 06:14:02 DEBUG : file1(0xc0009e6a40): close: 2022/05/18 06:14:02 DEBUG : file1: vfs cache: setting modification time to 2022-05-18 06:14:02.065569816 +0000 UTC m=+170.554672069 2022/05/18 06:14:02 INFO : file1: vfs cache: queuing for upload in 100ms 2022/05/18 06:14:02 DEBUG : file1(0xc0009e6a40): >close: err= 2022/05/18 06:14:02 DEBUG : file1(0xc0009e6a40): _writeAt: size=5, off=0 2022/05/18 06:14:02 DEBUG : file1(0xc0009e6a40): >_writeAt: n=0, err=file already closed 2022/05/18 06:14:02 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:14:02 DEBUG : : Looking for writers 2022/05/18 06:14:02 DEBUG : file1: reading active writers 2022/05/18 06:14:02 DEBUG : Still 0 writers active and 1 cache items in use, waiting 10ms 2022/05/18 06:14:02 DEBUG : : Looking for writers 2022/05/18 06:14:02 DEBUG : file1: reading active writers 2022/05/18 06:14:02 DEBUG : Still 0 writers active and 1 cache items in use, waiting 20ms 2022/05/18 06:14:02 DEBUG : : Looking for writers 2022/05/18 06:14:02 DEBUG : file1: reading active writers 2022/05/18 06:14:02 DEBUG : Still 0 writers active and 1 cache items in use, waiting 40ms 2022/05/18 06:14:02 DEBUG : : Looking for writers 2022/05/18 06:14:02 DEBUG : file1: reading active writers 2022/05/18 06:14:02 DEBUG : Still 0 writers active and 1 cache items in use, waiting 80ms 2022/05/18 06:14:02 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:14:02 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:14:02 DEBUG : : Looking for writers 2022/05/18 06:14:02 DEBUG : file1: reading active writers 2022/05/18 06:14:02 DEBUG : Still 0 writers active and 1 cache items in use, waiting 160ms 2022/05/18 06:14:02 DEBUG : : Looking for writers 2022/05/18 06:14:02 DEBUG : file1: reading active writers 2022/05/18 06:14:02 DEBUG : Still 0 writers active and 1 cache items in use, waiting 320ms 2022/05/18 06:14:02 DEBUG : : Looking for writers 2022/05/18 06:14:02 DEBUG : file1: reading active writers 2022/05/18 06:14:02 DEBUG : Still 0 writers active and 1 cache items in use, waiting 640ms 2022/05/18 06:14:02 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:14:02 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:14:03 DEBUG : : Looking for writers 2022/05/18 06:14:03 DEBUG : file1: reading active writers 2022/05/18 06:14:03 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:04 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:04 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:04 ERROR : file1: vfs cache: failed to upload try #1, will retry in 200ms: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:04 DEBUG : : Looking for writers 2022/05/18 06:14:04 DEBUG : file1: reading active writers 2022/05/18 06:14:04 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:04 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:14:04 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:14:05 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:14:05 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:14:05 DEBUG : : Looking for writers 2022/05/18 06:14:05 DEBUG : file1: reading active writers 2022/05/18 06:14:05 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:06 DEBUG : : Looking for writers 2022/05/18 06:14:06 DEBUG : file1: reading active writers 2022/05/18 06:14:06 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:06 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:06 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:06 ERROR : file1: vfs cache: failed to upload try #2, will retry in 400ms: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:06 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:14:06 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:14:07 DEBUG : : Looking for writers 2022/05/18 06:14:07 DEBUG : file1: reading active writers 2022/05/18 06:14:07 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:07 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:14:07 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:14:08 DEBUG : : Looking for writers 2022/05/18 06:14:08 DEBUG : file1: reading active writers 2022/05/18 06:14:08 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:09 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:09 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:09 ERROR : file1: vfs cache: failed to upload try #3, will retry in 800ms: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:09 DEBUG : : Looking for writers 2022/05/18 06:14:09 DEBUG : file1: reading active writers 2022/05/18 06:14:09 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:09 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:14:09 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:14:10 DEBUG : : Looking for writers 2022/05/18 06:14:10 DEBUG : file1: reading active writers 2022/05/18 06:14:10 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:10 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:14:10 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:14:11 DEBUG : : Looking for writers 2022/05/18 06:14:11 DEBUG : file1: reading active writers 2022/05/18 06:14:11 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:11 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:11 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:11 ERROR : file1: vfs cache: failed to upload try #4, will retry in 1.6s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:12 DEBUG : : Looking for writers 2022/05/18 06:14:12 DEBUG : file1: reading active writers 2022/05/18 06:14:12 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:13 DEBUG : : Looking for writers 2022/05/18 06:14:13 DEBUG : file1: reading active writers 2022/05/18 06:14:13 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:13 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:14:13 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:14:14 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:14:14 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:14:14 DEBUG : : Looking for writers 2022/05/18 06:14:14 DEBUG : file1: reading active writers 2022/05/18 06:14:14 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:15 DEBUG : : Looking for writers 2022/05/18 06:14:15 DEBUG : file1: reading active writers 2022/05/18 06:14:15 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:15 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:15 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:15 ERROR : file1: vfs cache: failed to upload try #5, will retry in 3.2s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:16 DEBUG : : Looking for writers 2022/05/18 06:14:16 DEBUG : file1: reading active writers 2022/05/18 06:14:16 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:17 DEBUG : : Looking for writers 2022/05/18 06:14:17 DEBUG : file1: reading active writers 2022/05/18 06:14:17 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:18 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2022/05/18 06:14:18 DEBUG : : Looking for writers 2022/05/18 06:14:18 DEBUG : file1: reading active writers 2022/05/18 06:14:18 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:18 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:14:18 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:14:19 DEBUG : : Looking for writers 2022/05/18 06:14:19 DEBUG : file1: reading active writers 2022/05/18 06:14:19 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:19 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:14:19 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:14:20 DEBUG : : Looking for writers 2022/05/18 06:14:20 DEBUG : file1: reading active writers 2022/05/18 06:14:20 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:20 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:20 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:20 ERROR : file1: vfs cache: failed to upload try #6, will retry in 6.4s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:21 DEBUG : : Looking for writers 2022/05/18 06:14:21 DEBUG : file1: reading active writers 2022/05/18 06:14:21 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:22 DEBUG : : Looking for writers 2022/05/18 06:14:22 DEBUG : file1: reading active writers 2022/05/18 06:14:22 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:22 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2022/05/18 06:14:23 DEBUG : : Looking for writers 2022/05/18 06:14:23 DEBUG : file1: reading active writers 2022/05/18 06:14:23 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:24 DEBUG : : Looking for writers 2022/05/18 06:14:24 DEBUG : file1: reading active writers 2022/05/18 06:14:24 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:25 DEBUG : : Looking for writers 2022/05/18 06:14:25 DEBUG : file1: reading active writers 2022/05/18 06:14:25 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:26 DEBUG : : Looking for writers 2022/05/18 06:14:26 DEBUG : file1: reading active writers 2022/05/18 06:14:26 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:27 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:14:27 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:14:27 DEBUG : : Looking for writers 2022/05/18 06:14:27 DEBUG : file1: reading active writers 2022/05/18 06:14:27 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:27 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:14:27 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:14:28 DEBUG : : Looking for writers 2022/05/18 06:14:28 DEBUG : file1: reading active writers 2022/05/18 06:14:28 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:29 DEBUG : : Looking for writers 2022/05/18 06:14:29 DEBUG : file1: reading active writers 2022/05/18 06:14:29 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:29 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:29 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:29 ERROR : file1: vfs cache: failed to upload try #7, will retry in 12.8s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:30 DEBUG : : Looking for writers 2022/05/18 06:14:30 DEBUG : file1: reading active writers 2022/05/18 06:14:30 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:31 DEBUG : : Looking for writers 2022/05/18 06:14:31 DEBUG : file1: reading active writers 2022/05/18 06:14:31 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:32 ERROR : Exiting even though 0 writers active and 1 cache items in use after 30s Cache{ "file1": &{c:0xc0005f29c0 mu:{state:0 sema:0} cond:0xc00077ed80 name:file1 opens:0 downloaders: o: fd: modified:false info:{ModTime:{wall:13878195789893239832 ext:170554672069 loc:0x2b8b560} ATime:{wall:13878195789893257015 ext:170554689261 loc:0x2b8b560} Size:11 Rs:[{Pos:0 Size:11}] Fingerprint: Dirty:true} writeBackID:1 pendingAccesses:0 beingReset:false}, } 2022/05/18 06:14:32 DEBUG : >WaitForWriters: 2022/05/18 06:14:32 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:14:32 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:14:32 ERROR : : error listing: directory not found fstest.go:299: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:302: Flushing the directory cache 2022/05/18 06:14:33 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:14:34 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:14:34 ERROR : : error listing: directory not found fstest.go:299: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:302: Flushing the directory cache 2022/05/18 06:14:36 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:14:36 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:14:36 ERROR : : error listing: directory not found fstest.go:299: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:302: Flushing the directory cache fstest.go:306: Error Trace: fstest.go:306 fstest.go:339 read_write_test.go:392 Error: Should be true Test: TestRWFileHandleWriteAt Messages: listing wrong, want file1 (11) got fstest.go:204: Not found "file1" fstest.go:207: Error Trace: fstest.go:207 fstest.go:311 fstest.go:339 read_write_test.go:392 Error: Not equal: expected: 0 actual : 1 Test: TestRWFileHandleWriteAt Messages: 1 objects not found 2022/05/18 06:14:40 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:14:40 DEBUG : : Looking for writers 2022/05/18 06:14:40 DEBUG : file1: reading active writers 2022/05/18 06:14:40 DEBUG : Still 0 writers active and 1 cache items in use, waiting 10ms 2022/05/18 06:14:41 DEBUG : : Looking for writers 2022/05/18 06:14:41 DEBUG : file1: reading active writers 2022/05/18 06:14:41 DEBUG : Still 0 writers active and 1 cache items in use, waiting 20ms 2022/05/18 06:14:41 DEBUG : : Looking for writers 2022/05/18 06:14:41 DEBUG : file1: reading active writers 2022/05/18 06:14:41 DEBUG : Still 0 writers active and 1 cache items in use, waiting 40ms 2022/05/18 06:14:41 DEBUG : : Looking for writers 2022/05/18 06:14:41 DEBUG : file1: reading active writers 2022/05/18 06:14:41 DEBUG : Still 0 writers active and 1 cache items in use, waiting 80ms 2022/05/18 06:14:41 DEBUG : : Looking for writers 2022/05/18 06:14:41 DEBUG : file1: reading active writers 2022/05/18 06:14:41 DEBUG : Still 0 writers active and 1 cache items in use, waiting 160ms 2022/05/18 06:14:41 DEBUG : : Looking for writers 2022/05/18 06:14:41 DEBUG : file1: reading active writers 2022/05/18 06:14:41 DEBUG : Still 0 writers active and 1 cache items in use, waiting 320ms 2022/05/18 06:14:41 DEBUG : : Looking for writers 2022/05/18 06:14:41 DEBUG : file1: reading active writers 2022/05/18 06:14:41 DEBUG : Still 0 writers active and 1 cache items in use, waiting 640ms 2022/05/18 06:14:42 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:14:42 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:14:42 DEBUG : : Looking for writers 2022/05/18 06:14:42 DEBUG : file1: reading active writers 2022/05/18 06:14:42 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:42 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:14:42 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:14:43 DEBUG : : Looking for writers 2022/05/18 06:14:43 DEBUG : file1: reading active writers 2022/05/18 06:14:43 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:44 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:44 ERROR : file1: vfs cache: failed to upload try #8, will retry in 25.6s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:14:44 DEBUG : : Looking for writers 2022/05/18 06:14:44 DEBUG : file1: reading active writers 2022/05/18 06:14:44 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:45 DEBUG : : Looking for writers 2022/05/18 06:14:45 DEBUG : file1: reading active writers 2022/05/18 06:14:45 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:46 DEBUG : : Looking for writers 2022/05/18 06:14:46 DEBUG : file1: reading active writers 2022/05/18 06:14:46 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:47 DEBUG : : Looking for writers 2022/05/18 06:14:47 DEBUG : file1: reading active writers 2022/05/18 06:14:47 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:48 DEBUG : : Looking for writers 2022/05/18 06:14:48 DEBUG : file1: reading active writers 2022/05/18 06:14:48 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:49 DEBUG : : Looking for writers 2022/05/18 06:14:49 DEBUG : file1: reading active writers 2022/05/18 06:14:49 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:50 DEBUG : : Looking for writers 2022/05/18 06:14:50 DEBUG : file1: reading active writers 2022/05/18 06:14:50 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:51 DEBUG : : Looking for writers 2022/05/18 06:14:51 DEBUG : file1: reading active writers 2022/05/18 06:14:51 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:52 DEBUG : : Looking for writers 2022/05/18 06:14:52 DEBUG : file1: reading active writers 2022/05/18 06:14:52 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:53 DEBUG : : Looking for writers 2022/05/18 06:14:53 DEBUG : file1: reading active writers 2022/05/18 06:14:53 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:54 DEBUG : : Looking for writers 2022/05/18 06:14:54 DEBUG : file1: reading active writers 2022/05/18 06:14:54 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:55 DEBUG : : Looking for writers 2022/05/18 06:14:55 DEBUG : file1: reading active writers 2022/05/18 06:14:55 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:56 DEBUG : : Looking for writers 2022/05/18 06:14:56 DEBUG : file1: reading active writers 2022/05/18 06:14:56 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:57 DEBUG : : Looking for writers 2022/05/18 06:14:57 DEBUG : file1: reading active writers 2022/05/18 06:14:57 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:58 DEBUG : : Looking for writers 2022/05/18 06:14:58 DEBUG : file1: reading active writers 2022/05/18 06:14:58 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:59 DEBUG : : Looking for writers 2022/05/18 06:14:59 DEBUG : file1: reading active writers 2022/05/18 06:14:59 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:14:59 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item file1 not removed, freed 0 bytes 2022/05/18 06:14:59 INFO : vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 1, uploading 0, total size 11 (was 11) 2022/05/18 06:15:00 DEBUG : : Looking for writers 2022/05/18 06:15:00 DEBUG : file1: reading active writers 2022/05/18 06:15:00 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:15:01 DEBUG : : Looking for writers 2022/05/18 06:15:01 DEBUG : file1: reading active writers 2022/05/18 06:15:01 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:15:02 DEBUG : : Looking for writers 2022/05/18 06:15:02 DEBUG : file1: reading active writers 2022/05/18 06:15:02 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:15:03 DEBUG : : Looking for writers 2022/05/18 06:15:03 DEBUG : file1: reading active writers 2022/05/18 06:15:03 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:15:04 DEBUG : : Looking for writers 2022/05/18 06:15:04 DEBUG : file1: reading active writers 2022/05/18 06:15:04 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:15:05 DEBUG : : Looking for writers 2022/05/18 06:15:05 DEBUG : file1: reading active writers 2022/05/18 06:15:05 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:15:06 DEBUG : : Looking for writers 2022/05/18 06:15:06 DEBUG : file1: reading active writers 2022/05/18 06:15:06 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:15:07 DEBUG : : Looking for writers 2022/05/18 06:15:07 DEBUG : file1: reading active writers 2022/05/18 06:15:07 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:15:08 DEBUG : : Looking for writers 2022/05/18 06:15:08 DEBUG : file1: reading active writers 2022/05/18 06:15:08 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:15:09 DEBUG : : Looking for writers 2022/05/18 06:15:09 DEBUG : file1: reading active writers 2022/05/18 06:15:09 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:15:09 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:15:09 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:10 DEBUG : : Looking for writers 2022/05/18 06:15:10 DEBUG : file1: reading active writers 2022/05/18 06:15:10 DEBUG : Still 0 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:15:10 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:10 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:15:10 ERROR : Exiting even though 0 writers active and 1 cache items in use after 30s Cache{ "file1": &{c:0xc0005f29c0 mu:{state:0 sema:0} cond:0xc00077ed80 name:file1 opens:0 downloaders: o: fd: modified:false info:{ModTime:{wall:13878195789893239832 ext:170554672069 loc:0x2b8b560} ATime:{wall:13878195789893257015 ext:170554689261 loc:0x2b8b560} Size:11 Rs:[{Pos:0 Size:11}] Fingerprint: Dirty:true} writeBackID:1 pendingAccesses:0 beingReset:false}, } 2022/05/18 06:15:10 DEBUG : >WaitForWriters: 2022/05/18 06:15:11 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:11 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:11 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:11 ERROR : file1: vfs cache: failed to upload try #9, will retry in 51.2s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:12 ERROR : : error listing: directory not found --- FAIL: TestRWFileHandleWriteAt (70.46s) === RUN TestRWFileHandleWriteNoWrite run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:15:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:15:12 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2022/05/18 06:15:12 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2022/05/18 06:15:12 DEBUG : file1: newRWFileHandle: 2022/05/18 06:15:12 DEBUG : file1(0xc00094cec0): openPending: 2022/05/18 06:15:12 ERROR : file1: vfs cache: detected external removal of cache file 2022/05/18 06:15:12 DEBUG : file1: vfs cache: truncate to size=0 2022/05/18 06:15:12 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:15:12 DEBUG : file1(0xc00094cec0): >openPending: err= 2022/05/18 06:15:12 DEBUG : file1: vfs cache: cancelling writeback (uploading false) 0xc0002d0c40 item 1 2022/05/18 06:15:12 DEBUG : file1: >newRWFileHandle: err= 2022/05/18 06:15:12 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:15:12 DEBUG : file1: >Open: fd=file1 (rw), err= 2022/05/18 06:15:12 DEBUG : file1: >OpenFile: fd=file1 (rw), err= 2022/05/18 06:15:12 DEBUG : file1(0xc00094cec0): close: 2022/05/18 06:15:12 DEBUG : file1: vfs cache: setting modification time to 2022-05-18 06:15:12.524804744 +0000 UTC m=+241.013906976 2022/05/18 06:15:12 INFO : file1: vfs cache: queuing for upload in 100ms 2022/05/18 06:15:12 DEBUG : file1(0xc00094cec0): >close: err= 2022/05/18 06:15:12 DEBUG : file2: OpenFile: flags=O_WRONLY|O_CREATE|O_TRUNC, perm=-rwxrwxrwx 2022/05/18 06:15:12 DEBUG : file2: Open: flags=O_WRONLY|O_CREATE|O_TRUNC 2022/05/18 06:15:12 DEBUG : file2: newRWFileHandle: 2022/05/18 06:15:12 DEBUG : file2(0xc00094cfc0): openPending: 2022/05/18 06:15:12 DEBUG : file2: vfs cache: truncate to size=0 2022/05/18 06:15:12 DEBUG : : Added virtual directory entry vAddFile: "file2" 2022/05/18 06:15:12 DEBUG : file2(0xc00094cfc0): >openPending: err= 2022/05/18 06:15:12 DEBUG : file2: >newRWFileHandle: err= 2022/05/18 06:15:12 DEBUG : : Added virtual directory entry vAddFile: "file2" 2022/05/18 06:15:12 DEBUG : file2: >Open: fd=file2 (rw), err= 2022/05/18 06:15:12 DEBUG : file2: >OpenFile: fd=file2 (rw), err= 2022/05/18 06:15:12 DEBUG : file2(0xc00094cfc0): RWFileHandle.Flush 2022/05/18 06:15:12 DEBUG : file2(0xc00094cfc0): RWFileHandle.Release 2022/05/18 06:15:12 DEBUG : file2(0xc00094cfc0): close: 2022/05/18 06:15:12 DEBUG : file2: vfs cache: setting modification time to 2022-05-18 06:15:12.525406735 +0000 UTC m=+241.014508957 2022/05/18 06:15:12 INFO : file2: vfs cache: queuing for upload in 100ms 2022/05/18 06:15:12 DEBUG : file2(0xc00094cfc0): >close: err= 2022/05/18 06:15:12 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:15:12 DEBUG : : Looking for writers 2022/05/18 06:15:12 DEBUG : file1: reading active writers 2022/05/18 06:15:12 DEBUG : file2: reading active writers 2022/05/18 06:15:12 DEBUG : Still 0 writers active and 2 cache items in use, waiting 10ms 2022/05/18 06:15:12 DEBUG : : Looking for writers 2022/05/18 06:15:12 DEBUG : file1: reading active writers 2022/05/18 06:15:12 DEBUG : file2: reading active writers 2022/05/18 06:15:12 DEBUG : Still 0 writers active and 2 cache items in use, waiting 20ms 2022/05/18 06:15:12 DEBUG : : Looking for writers 2022/05/18 06:15:12 DEBUG : file2: reading active writers 2022/05/18 06:15:12 DEBUG : file1: reading active writers 2022/05/18 06:15:12 DEBUG : Still 0 writers active and 2 cache items in use, waiting 40ms 2022/05/18 06:15:12 DEBUG : : Looking for writers 2022/05/18 06:15:12 DEBUG : file2: reading active writers 2022/05/18 06:15:12 DEBUG : file1: reading active writers 2022/05/18 06:15:12 DEBUG : Still 0 writers active and 2 cache items in use, waiting 80ms 2022/05/18 06:15:12 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:15:12 DEBUG : file2: vfs cache: starting upload 2022/05/18 06:15:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:12 DEBUG : : Looking for writers 2022/05/18 06:15:12 DEBUG : file1: reading active writers 2022/05/18 06:15:12 DEBUG : file2: reading active writers 2022/05/18 06:15:12 DEBUG : Still 0 writers active and 2 cache items in use, waiting 160ms 2022/05/18 06:15:12 DEBUG : : Looking for writers 2022/05/18 06:15:12 DEBUG : file1: reading active writers 2022/05/18 06:15:12 DEBUG : file2: reading active writers 2022/05/18 06:15:12 DEBUG : Still 0 writers active and 2 cache items in use, waiting 320ms 2022/05/18 06:15:13 DEBUG : : Looking for writers 2022/05/18 06:15:13 DEBUG : file1: reading active writers 2022/05/18 06:15:13 DEBUG : file2: reading active writers 2022/05/18 06:15:13 DEBUG : Still 0 writers active and 2 cache items in use, waiting 640ms 2022/05/18 06:15:13 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:13 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:15:13 DEBUG : : Looking for writers 2022/05/18 06:15:13 DEBUG : file1: reading active writers 2022/05/18 06:15:13 DEBUG : file2: reading active writers 2022/05/18 06:15:13 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:14 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:14 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:14 ERROR : file1: vfs cache: failed to upload try #1, will retry in 200ms: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:14 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:14 DEBUG : : Looking for writers 2022/05/18 06:15:14 DEBUG : file1: reading active writers 2022/05/18 06:15:14 DEBUG : file2: reading active writers 2022/05/18 06:15:14 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:14 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:15:15 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:15 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:15:15 DEBUG : : Looking for writers 2022/05/18 06:15:15 DEBUG : file1: reading active writers 2022/05/18 06:15:15 DEBUG : file2: reading active writers 2022/05/18 06:15:15 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:16 DEBUG : : Looking for writers 2022/05/18 06:15:16 DEBUG : file1: reading active writers 2022/05/18 06:15:16 DEBUG : file2: reading active writers 2022/05/18 06:15:16 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:16 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:16 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:16 ERROR : file2: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:16 ERROR : file2: vfs cache: failed to upload try #1, will retry in 200ms: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:17 DEBUG : file2: vfs cache: starting upload 2022/05/18 06:15:17 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:17 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:15:17 DEBUG : : Looking for writers 2022/05/18 06:15:17 DEBUG : file1: reading active writers 2022/05/18 06:15:17 DEBUG : file2: reading active writers 2022/05/18 06:15:17 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:18 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2022/05/18 06:15:18 DEBUG : : Looking for writers 2022/05/18 06:15:18 DEBUG : file1: reading active writers 2022/05/18 06:15:18 DEBUG : file2: reading active writers 2022/05/18 06:15:18 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:18 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:18 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:18 ERROR : file1: vfs cache: failed to upload try #2, will retry in 400ms: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:18 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:19 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:15:19 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:19 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:15:19 DEBUG : : Looking for writers 2022/05/18 06:15:19 DEBUG : file1: reading active writers 2022/05/18 06:15:19 DEBUG : file2: reading active writers 2022/05/18 06:15:19 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:20 DEBUG : : Looking for writers 2022/05/18 06:15:20 DEBUG : file1: reading active writers 2022/05/18 06:15:20 DEBUG : file2: reading active writers 2022/05/18 06:15:20 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:20 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:20 ERROR : file2: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:20 ERROR : file2: vfs cache: failed to upload try #2, will retry in 400ms: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:20 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:21 DEBUG : file2: vfs cache: starting upload 2022/05/18 06:15:21 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:21 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:15:21 DEBUG : : Looking for writers 2022/05/18 06:15:21 DEBUG : file1: reading active writers 2022/05/18 06:15:21 DEBUG : file2: reading active writers 2022/05/18 06:15:21 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:22 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2022/05/18 06:15:22 DEBUG : : Looking for writers 2022/05/18 06:15:22 DEBUG : file1: reading active writers 2022/05/18 06:15:22 DEBUG : file2: reading active writers 2022/05/18 06:15:22 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:23 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:23 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:23 ERROR : file1: vfs cache: failed to upload try #3, will retry in 800ms: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:23 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:23 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:23 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:15:23 DEBUG : : Looking for writers 2022/05/18 06:15:23 DEBUG : file1: reading active writers 2022/05/18 06:15:23 DEBUG : file2: reading active writers 2022/05/18 06:15:23 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:23 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:15:24 DEBUG : : Looking for writers 2022/05/18 06:15:24 DEBUG : file1: reading active writers 2022/05/18 06:15:24 DEBUG : file2: reading active writers 2022/05/18 06:15:24 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:25 ERROR : file2: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:25 ERROR : file2: vfs cache: failed to upload try #3, will retry in 800ms: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:25 DEBUG : : Looking for writers 2022/05/18 06:15:25 DEBUG : file1: reading active writers 2022/05/18 06:15:25 DEBUG : file2: reading active writers 2022/05/18 06:15:25 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:15:26 DEBUG : file2: vfs cache: starting upload 2022/05/18 06:15:26 DEBUG : : Looking for writers 2022/05/18 06:15:26 DEBUG : file1: reading active writers 2022/05/18 06:15:26 DEBUG : file2: reading active writers 2022/05/18 06:15:26 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:27 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:27 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:27 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:27 ERROR : file1: vfs cache: failed to upload try #4, will retry in 1.6s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:27 DEBUG : : Looking for writers 2022/05/18 06:15:27 DEBUG : file1: reading active writers 2022/05/18 06:15:27 DEBUG : file2: reading active writers 2022/05/18 06:15:27 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:28 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:28 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:15:28 DEBUG : : Looking for writers 2022/05/18 06:15:28 DEBUG : file1: reading active writers 2022/05/18 06:15:28 DEBUG : file2: reading active writers 2022/05/18 06:15:28 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:29 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:15:29 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:29 ERROR : file2: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:29 ERROR : file2: vfs cache: failed to upload try #4, will retry in 1.6s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:29 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:29 DEBUG : : Looking for writers 2022/05/18 06:15:29 DEBUG : file1: reading active writers 2022/05/18 06:15:29 DEBUG : file2: reading active writers 2022/05/18 06:15:29 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:30 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:30 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:15:30 DEBUG : : Looking for writers 2022/05/18 06:15:30 DEBUG : file1: reading active writers 2022/05/18 06:15:30 DEBUG : file2: reading active writers 2022/05/18 06:15:30 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:31 DEBUG : file2: vfs cache: starting upload 2022/05/18 06:15:31 DEBUG : : Looking for writers 2022/05/18 06:15:31 DEBUG : file1: reading active writers 2022/05/18 06:15:31 DEBUG : file2: reading active writers 2022/05/18 06:15:31 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:31 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:31 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:31 ERROR : file1: vfs cache: failed to upload try #5, will retry in 3.2s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:31 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:32 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:32 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:15:32 DEBUG : : Looking for writers 2022/05/18 06:15:32 DEBUG : file1: reading active writers 2022/05/18 06:15:32 DEBUG : file2: reading active writers 2022/05/18 06:15:32 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:33 DEBUG : : Looking for writers 2022/05/18 06:15:33 DEBUG : file1: reading active writers 2022/05/18 06:15:33 DEBUG : file2: reading active writers 2022/05/18 06:15:33 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:33 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:33 ERROR : file2: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:33 ERROR : file2: vfs cache: failed to upload try #5, will retry in 3.2s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:34 DEBUG : : Looking for writers 2022/05/18 06:15:34 DEBUG : file1: reading active writers 2022/05/18 06:15:34 DEBUG : file2: reading active writers 2022/05/18 06:15:34 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:35 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:15:35 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:35 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:35 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:15:35 DEBUG : : Looking for writers 2022/05/18 06:15:35 DEBUG : file2: reading active writers 2022/05/18 06:15:35 DEBUG : file1: reading active writers 2022/05/18 06:15:35 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:36 DEBUG : : Looking for writers 2022/05/18 06:15:36 DEBUG : file1: reading active writers 2022/05/18 06:15:36 DEBUG : file2: reading active writers 2022/05/18 06:15:36 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:37 DEBUG : file2: vfs cache: starting upload 2022/05/18 06:15:37 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:37 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:37 ERROR : file1: vfs cache: failed to upload try #6, will retry in 6.4s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:37 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:37 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:37 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:15:37 DEBUG : : Looking for writers 2022/05/18 06:15:37 DEBUG : file2: reading active writers 2022/05/18 06:15:37 DEBUG : file1: reading active writers 2022/05/18 06:15:37 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:38 DEBUG : : Looking for writers 2022/05/18 06:15:38 DEBUG : file1: reading active writers 2022/05/18 06:15:38 DEBUG : file2: reading active writers 2022/05/18 06:15:38 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:39 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:39 ERROR : file2: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:39 ERROR : file2: vfs cache: failed to upload try #6, will retry in 6.4s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:39 DEBUG : : Looking for writers 2022/05/18 06:15:39 DEBUG : file1: reading active writers 2022/05/18 06:15:39 DEBUG : file2: reading active writers 2022/05/18 06:15:39 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:40 DEBUG : : Looking for writers 2022/05/18 06:15:40 DEBUG : file1: reading active writers 2022/05/18 06:15:40 DEBUG : file2: reading active writers 2022/05/18 06:15:40 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:41 DEBUG : : Looking for writers 2022/05/18 06:15:41 DEBUG : file1: reading active writers 2022/05/18 06:15:41 DEBUG : file2: reading active writers 2022/05/18 06:15:41 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:42 ERROR : Exiting even though 0 writers active and 2 cache items in use after 30s Cache{ "file2": &{c:0xc0005f29c0 mu:{state:0 sema:0} cond:0xc00094cf80 name:file2 opens:0 downloaders: o: fd: modified:false info:{ModTime:{wall:13878195865515004431 ext:241014508957 loc:0x2b8b560} ATime:{wall:13878195865515094679 ext:241014599206 loc:0x2b8b560} Size:0 Rs:[] Fingerprint: Dirty:true} writeBackID:2 pendingAccesses:0 beingReset:false}, "file1": &{c:0xc0005f29c0 mu:{state:0 sema:0} cond:0xc00077ed80 name:file1 opens:0 downloaders: o: fd: modified:false info:{ModTime:{wall:13878195865514402440 ext:241013906976 loc:0x2b8b560} ATime:{wall:13878195865514547270 ext:241014051796 loc:0x2b8b560} Size:0 Rs:[] Fingerprint: Dirty:true} writeBackID:1 pendingAccesses:0 beingReset:false}, } 2022/05/18 06:15:42 DEBUG : >WaitForWriters: 2022/05/18 06:15:42 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:43 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:43 ERROR : : error listing: directory not found fstest.go:299: Sleeping for 1s for list eventual consistency: 1/3 2022/05/18 06:15:43 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:15:43 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] fstest.go:302: Flushing the directory cache 2022/05/18 06:15:45 DEBUG : file2: vfs cache: starting upload 2022/05/18 06:15:45 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:45 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:45 ERROR : file1: vfs cache: failed to upload try #7, will retry in 12.8s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:45 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:46 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:46 ERROR : : error listing: directory not found 2022/05/18 06:15:46 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] fstest.go:299: Sleeping for 2s for list eventual consistency: 2/3 2022/05/18 06:15:46 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:46 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] fstest.go:302: Flushing the directory cache 2022/05/18 06:15:48 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:48 ERROR : file2: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:48 ERROR : file2: vfs cache: failed to upload try #7, will retry in 12.8s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:15:48 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:48 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:48 ERROR : : error listing: directory not found fstest.go:299: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:302: Flushing the directory cache fstest.go:306: Error Trace: fstest.go:306 fstest.go:339 read_write_test.go:426 Error: Should be true Test: TestRWFileHandleWriteNoWrite Messages: listing wrong, want file1 (0), file2 (0) got fstest.go:204: Not found "file1" fstest.go:204: Not found "file2" fstest.go:207: Error Trace: fstest.go:207 fstest.go:311 fstest.go:339 read_write_test.go:426 Error: Not equal: expected: 0 actual : 2 Test: TestRWFileHandleWriteNoWrite Messages: 2 objects not found 2022/05/18 06:15:52 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:15:52 DEBUG : : Looking for writers 2022/05/18 06:15:52 DEBUG : file1: reading active writers 2022/05/18 06:15:52 DEBUG : file2: reading active writers 2022/05/18 06:15:52 DEBUG : Still 0 writers active and 2 cache items in use, waiting 10ms 2022/05/18 06:15:52 DEBUG : : Looking for writers 2022/05/18 06:15:52 DEBUG : file1: reading active writers 2022/05/18 06:15:52 DEBUG : file2: reading active writers 2022/05/18 06:15:52 DEBUG : Still 0 writers active and 2 cache items in use, waiting 20ms 2022/05/18 06:15:52 DEBUG : : Looking for writers 2022/05/18 06:15:52 DEBUG : file2: reading active writers 2022/05/18 06:15:52 DEBUG : file1: reading active writers 2022/05/18 06:15:52 DEBUG : Still 0 writers active and 2 cache items in use, waiting 40ms 2022/05/18 06:15:52 DEBUG : : Looking for writers 2022/05/18 06:15:52 DEBUG : file1: reading active writers 2022/05/18 06:15:52 DEBUG : file2: reading active writers 2022/05/18 06:15:52 DEBUG : Still 0 writers active and 2 cache items in use, waiting 80ms 2022/05/18 06:15:53 DEBUG : : Looking for writers 2022/05/18 06:15:53 DEBUG : file1: reading active writers 2022/05/18 06:15:53 DEBUG : file2: reading active writers 2022/05/18 06:15:53 DEBUG : Still 0 writers active and 2 cache items in use, waiting 160ms 2022/05/18 06:15:53 DEBUG : : Looking for writers 2022/05/18 06:15:53 DEBUG : file2: reading active writers 2022/05/18 06:15:53 DEBUG : file1: reading active writers 2022/05/18 06:15:53 DEBUG : Still 0 writers active and 2 cache items in use, waiting 320ms 2022/05/18 06:15:53 DEBUG : : Looking for writers 2022/05/18 06:15:53 DEBUG : file1: reading active writers 2022/05/18 06:15:53 DEBUG : file2: reading active writers 2022/05/18 06:15:53 DEBUG : Still 0 writers active and 2 cache items in use, waiting 640ms 2022/05/18 06:15:54 DEBUG : : Looking for writers 2022/05/18 06:15:54 DEBUG : file1: reading active writers 2022/05/18 06:15:54 DEBUG : file2: reading active writers 2022/05/18 06:15:54 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:55 DEBUG : : Looking for writers 2022/05/18 06:15:55 DEBUG : file1: reading active writers 2022/05/18 06:15:55 DEBUG : file2: reading active writers 2022/05/18 06:15:55 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:56 DEBUG : : Looking for writers 2022/05/18 06:15:56 DEBUG : file1: reading active writers 2022/05/18 06:15:56 DEBUG : file2: reading active writers 2022/05/18 06:15:56 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:57 DEBUG : : Looking for writers 2022/05/18 06:15:57 DEBUG : file1: reading active writers 2022/05/18 06:15:57 DEBUG : file2: reading active writers 2022/05/18 06:15:57 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:58 DEBUG : : Looking for writers 2022/05/18 06:15:58 DEBUG : file1: reading active writers 2022/05/18 06:15:58 DEBUG : file2: reading active writers 2022/05/18 06:15:58 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:58 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:15:58 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:15:59 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:15:59 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:15:59 DEBUG : : Looking for writers 2022/05/18 06:15:59 DEBUG : file1: reading active writers 2022/05/18 06:15:59 DEBUG : file2: reading active writers 2022/05/18 06:15:59 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:15:59 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item file1 not removed, freed 0 bytes 2022/05/18 06:15:59 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item file2 not removed, freed 0 bytes 2022/05/18 06:15:59 INFO : vfs cache: cleaned: objects 2 (was 2) in use 2, to upload 1, uploading 1, total size 0 (was 0) 2022/05/18 06:16:00 DEBUG : : Looking for writers 2022/05/18 06:16:00 DEBUG : file1: reading active writers 2022/05/18 06:16:00 DEBUG : file2: reading active writers 2022/05/18 06:16:00 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:00 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:16:00 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:16:00 ERROR : file1: vfs cache: failed to upload try #8, will retry in 25.6s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:16:01 DEBUG : file2: vfs cache: starting upload 2022/05/18 06:16:01 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:16:01 DEBUG : : Looking for writers 2022/05/18 06:16:01 DEBUG : file1: reading active writers 2022/05/18 06:16:01 DEBUG : file2: reading active writers 2022/05/18 06:16:01 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:01 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:16:01 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:16:02 DEBUG : : Looking for writers 2022/05/18 06:16:02 DEBUG : file1: reading active writers 2022/05/18 06:16:02 DEBUG : file2: reading active writers 2022/05/18 06:16:02 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:03 DEBUG : : Looking for writers 2022/05/18 06:16:03 DEBUG : file1: reading active writers 2022/05/18 06:16:03 DEBUG : file2: reading active writers 2022/05/18 06:16:03 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:03 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:16:03 ERROR : file2: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:16:03 ERROR : file2: vfs cache: failed to upload try #8, will retry in 25.6s: vfs cache: failed to transfer file from cache to remote: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:16:04 DEBUG : : Looking for writers 2022/05/18 06:16:04 DEBUG : file1: reading active writers 2022/05/18 06:16:04 DEBUG : file2: reading active writers 2022/05/18 06:16:04 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:05 DEBUG : : Looking for writers 2022/05/18 06:16:05 DEBUG : file1: reading active writers 2022/05/18 06:16:05 DEBUG : file2: reading active writers 2022/05/18 06:16:05 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:06 DEBUG : : Looking for writers 2022/05/18 06:16:06 DEBUG : file1: reading active writers 2022/05/18 06:16:06 DEBUG : file2: reading active writers 2022/05/18 06:16:06 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:07 DEBUG : : Looking for writers 2022/05/18 06:16:07 DEBUG : file1: reading active writers 2022/05/18 06:16:07 DEBUG : file2: reading active writers 2022/05/18 06:16:07 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:08 DEBUG : : Looking for writers 2022/05/18 06:16:08 DEBUG : file1: reading active writers 2022/05/18 06:16:08 DEBUG : file2: reading active writers 2022/05/18 06:16:08 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:09 DEBUG : : Looking for writers 2022/05/18 06:16:09 DEBUG : file1: reading active writers 2022/05/18 06:16:09 DEBUG : file2: reading active writers 2022/05/18 06:16:09 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:10 DEBUG : : Looking for writers 2022/05/18 06:16:10 DEBUG : file1: reading active writers 2022/05/18 06:16:10 DEBUG : file2: reading active writers 2022/05/18 06:16:10 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:11 DEBUG : : Looking for writers 2022/05/18 06:16:11 DEBUG : file1: reading active writers 2022/05/18 06:16:11 DEBUG : file2: reading active writers 2022/05/18 06:16:11 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:12 DEBUG : : Looking for writers 2022/05/18 06:16:12 DEBUG : file1: reading active writers 2022/05/18 06:16:12 DEBUG : file2: reading active writers 2022/05/18 06:16:12 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:13 DEBUG : : Looking for writers 2022/05/18 06:16:13 DEBUG : file1: reading active writers 2022/05/18 06:16:13 DEBUG : file2: reading active writers 2022/05/18 06:16:13 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:14 DEBUG : : Looking for writers 2022/05/18 06:16:14 DEBUG : file1: reading active writers 2022/05/18 06:16:14 DEBUG : file2: reading active writers 2022/05/18 06:16:14 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:15 DEBUG : : Looking for writers 2022/05/18 06:16:15 DEBUG : file1: reading active writers 2022/05/18 06:16:15 DEBUG : file2: reading active writers 2022/05/18 06:16:15 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:16 DEBUG : : Looking for writers 2022/05/18 06:16:16 DEBUG : file2: reading active writers 2022/05/18 06:16:16 DEBUG : file1: reading active writers 2022/05/18 06:16:16 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:17 DEBUG : : Looking for writers 2022/05/18 06:16:17 DEBUG : file1: reading active writers 2022/05/18 06:16:17 DEBUG : file2: reading active writers 2022/05/18 06:16:17 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:18 DEBUG : : Looking for writers 2022/05/18 06:16:18 DEBUG : file1: reading active writers 2022/05/18 06:16:18 DEBUG : file2: reading active writers 2022/05/18 06:16:18 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:18 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2022/05/18 06:16:19 DEBUG : : Looking for writers 2022/05/18 06:16:19 DEBUG : file1: reading active writers 2022/05/18 06:16:19 DEBUG : file2: reading active writers 2022/05/18 06:16:19 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:20 DEBUG : : Looking for writers 2022/05/18 06:16:20 DEBUG : file1: reading active writers 2022/05/18 06:16:20 DEBUG : file2: reading active writers 2022/05/18 06:16:20 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:21 DEBUG : : Looking for writers 2022/05/18 06:16:21 DEBUG : file1: reading active writers 2022/05/18 06:16:21 DEBUG : file2: reading active writers 2022/05/18 06:16:21 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:22 DEBUG : : Looking for writers 2022/05/18 06:16:22 DEBUG : file1: reading active writers 2022/05/18 06:16:22 DEBUG : file2: reading active writers 2022/05/18 06:16:22 DEBUG : Still 0 writers active and 2 cache items in use, waiting 1s 2022/05/18 06:16:22 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2022/05/18 06:16:22 ERROR : Exiting even though 0 writers active and 2 cache items in use after 30s Cache{ "file1": &{c:0xc0005f29c0 mu:{state:0 sema:0} cond:0xc00077ed80 name:file1 opens:0 downloaders: o: fd: modified:false info:{ModTime:{wall:13878195865514402440 ext:241013906976 loc:0x2b8b560} ATime:{wall:13878195865514547270 ext:241014051796 loc:0x2b8b560} Size:0 Rs:[] Fingerprint: Dirty:true} writeBackID:1 pendingAccesses:0 beingReset:false}, "file2": &{c:0xc0005f29c0 mu:{state:0 sema:0} cond:0xc00094cf80 name:file2 opens:0 downloaders: o: fd: modified:false info:{ModTime:{wall:13878195865515004431 ext:241014508957 loc:0x2b8b560} ATime:{wall:13878195865515094679 ext:241014599206 loc:0x2b8b560} Size:0 Rs:[] Fingerprint: Dirty:true} writeBackID:2 pendingAccesses:0 beingReset:false}, } 2022/05/18 06:16:22 DEBUG : >WaitForWriters: 2022/05/18 06:16:22 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:16:23 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:16:23 ERROR : : error listing: directory not found --- FAIL: TestRWFileHandleWriteNoWrite (71.00s) === RUN TestRWFileHandleSizeTruncateExisting run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:16:23 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:16:23 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:16:24 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:16:24 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:16:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestRWFileHandleSizeTruncateExisting (2.29s) === RUN TestRWFileHandleSizeCreateExisting run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:16:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:16:25 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:16:26 DEBUG : file1: vfs cache: starting upload 2022/05/18 06:16:26 ERROR : file1: vfs cache: failed to write metadata file: vfs cache item: failed to write metadata: open /home/rclone/.cache/rclone/vfsMeta/TestFileFabric/rclone-test-guyumar1gucuroj1gejusov5/file1: no such file or directory 2022/05/18 06:16:26 INFO : file1: vfs cache: upload succeeded try #9 2022/05/18 06:16:26 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:16:26 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:16:27 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) --- FAIL: TestRWFileHandleSizeCreateExisting (2.11s) === RUN TestRWFileModTimeWithOpenWriters run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:16:27 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:16:27 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2022/05/18 06:16:27 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2022/05/18 06:16:27 DEBUG : file1: newRWFileHandle: 2022/05/18 06:16:27 DEBUG : file1(0xc0002ce640): openPending: 2022/05/18 06:16:27 DEBUG : file1: vfs cache: truncate to size=0 2022/05/18 06:16:27 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:16:27 DEBUG : file1(0xc0002ce640): >openPending: err= 2022/05/18 06:16:27 DEBUG : file1: >newRWFileHandle: err= 2022/05/18 06:16:27 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:16:27 DEBUG : file1: >Open: fd=file1 (rw), err= 2022/05/18 06:16:27 DEBUG : file1: >OpenFile: fd=file1 (rw), err= 2022/05/18 06:16:27 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:16:28 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:16:28 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:16:28 DEBUG : file2: vfs cache: starting upload 2022/05/18 06:16:28 INFO : file2: vfs cache: upload succeeded try #9 2022/05/18 06:16:30 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:16:30 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:16:30 DEBUG : : Looking for writers 2022/05/18 06:16:30 DEBUG : file1: reading active writers 2022/05/18 06:16:30 DEBUG : file1: active writers 1 2022/05/18 06:16:30 DEBUG : file2: reading active writers 2022/05/18 06:16:30 DEBUG : Still 1 writers active and 1 cache items in use, waiting 10ms 2022/05/18 06:16:30 DEBUG : : Looking for writers 2022/05/18 06:16:30 DEBUG : file1: reading active writers 2022/05/18 06:16:30 DEBUG : file1: active writers 1 2022/05/18 06:16:30 DEBUG : file2: reading active writers 2022/05/18 06:16:30 DEBUG : Still 1 writers active and 1 cache items in use, waiting 20ms 2022/05/18 06:16:30 DEBUG : : Looking for writers 2022/05/18 06:16:30 DEBUG : file1: reading active writers 2022/05/18 06:16:30 DEBUG : file1: active writers 1 2022/05/18 06:16:30 DEBUG : file2: reading active writers 2022/05/18 06:16:30 DEBUG : Still 1 writers active and 1 cache items in use, waiting 40ms 2022/05/18 06:16:30 DEBUG : : Looking for writers 2022/05/18 06:16:30 DEBUG : file1: reading active writers 2022/05/18 06:16:30 DEBUG : file1: active writers 1 2022/05/18 06:16:30 DEBUG : file2: reading active writers 2022/05/18 06:16:30 DEBUG : Still 1 writers active and 1 cache items in use, waiting 80ms 2022/05/18 06:16:30 DEBUG : : Looking for writers 2022/05/18 06:16:30 DEBUG : file1: reading active writers 2022/05/18 06:16:30 DEBUG : file1: active writers 1 2022/05/18 06:16:30 DEBUG : file2: reading active writers 2022/05/18 06:16:30 DEBUG : Still 1 writers active and 1 cache items in use, waiting 160ms 2022/05/18 06:16:30 DEBUG : : Looking for writers 2022/05/18 06:16:30 DEBUG : file1: reading active writers 2022/05/18 06:16:30 DEBUG : file1: active writers 1 2022/05/18 06:16:30 DEBUG : file2: reading active writers 2022/05/18 06:16:30 DEBUG : Still 1 writers active and 1 cache items in use, waiting 320ms 2022/05/18 06:16:30 DEBUG : : Looking for writers 2022/05/18 06:16:30 DEBUG : file1: reading active writers 2022/05/18 06:16:30 DEBUG : file1: active writers 1 2022/05/18 06:16:30 DEBUG : file2: reading active writers 2022/05/18 06:16:30 DEBUG : Still 1 writers active and 1 cache items in use, waiting 640ms 2022/05/18 06:16:31 DEBUG : : Looking for writers 2022/05/18 06:16:31 DEBUG : file1: reading active writers 2022/05/18 06:16:31 DEBUG : file1: active writers 1 2022/05/18 06:16:31 DEBUG : file2: reading active writers 2022/05/18 06:16:31 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:32 DEBUG : : Looking for writers 2022/05/18 06:16:32 DEBUG : file1: reading active writers 2022/05/18 06:16:32 DEBUG : file1: active writers 1 2022/05/18 06:16:32 DEBUG : file2: reading active writers 2022/05/18 06:16:32 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:33 DEBUG : : Looking for writers 2022/05/18 06:16:33 DEBUG : file1: reading active writers 2022/05/18 06:16:33 DEBUG : file1: active writers 1 2022/05/18 06:16:33 DEBUG : file2: reading active writers 2022/05/18 06:16:33 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:34 DEBUG : : Looking for writers 2022/05/18 06:16:34 DEBUG : file1: reading active writers 2022/05/18 06:16:34 DEBUG : file1: active writers 1 2022/05/18 06:16:34 DEBUG : file2: reading active writers 2022/05/18 06:16:34 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:35 DEBUG : : Looking for writers 2022/05/18 06:16:35 DEBUG : file1: reading active writers 2022/05/18 06:16:35 DEBUG : file1: active writers 1 2022/05/18 06:16:35 DEBUG : file2: reading active writers 2022/05/18 06:16:35 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:36 DEBUG : : Looking for writers 2022/05/18 06:16:36 DEBUG : file1: reading active writers 2022/05/18 06:16:36 DEBUG : file1: active writers 1 2022/05/18 06:16:36 DEBUG : file2: reading active writers 2022/05/18 06:16:36 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:37 DEBUG : : Looking for writers 2022/05/18 06:16:37 DEBUG : file1: reading active writers 2022/05/18 06:16:37 DEBUG : file1: active writers 1 2022/05/18 06:16:37 DEBUG : file2: reading active writers 2022/05/18 06:16:37 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:38 DEBUG : : Looking for writers 2022/05/18 06:16:38 DEBUG : file1: reading active writers 2022/05/18 06:16:38 DEBUG : file1: active writers 1 2022/05/18 06:16:38 DEBUG : file2: reading active writers 2022/05/18 06:16:38 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:39 DEBUG : : Looking for writers 2022/05/18 06:16:39 DEBUG : file1: reading active writers 2022/05/18 06:16:39 DEBUG : file1: active writers 1 2022/05/18 06:16:39 DEBUG : file2: reading active writers 2022/05/18 06:16:39 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:40 DEBUG : : Looking for writers 2022/05/18 06:16:40 DEBUG : file1: reading active writers 2022/05/18 06:16:40 DEBUG : file1: active writers 1 2022/05/18 06:16:40 DEBUG : file2: reading active writers 2022/05/18 06:16:40 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:41 DEBUG : : Looking for writers 2022/05/18 06:16:41 DEBUG : file1: reading active writers 2022/05/18 06:16:41 DEBUG : file1: active writers 1 2022/05/18 06:16:41 DEBUG : file2: reading active writers 2022/05/18 06:16:41 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:42 DEBUG : : Looking for writers 2022/05/18 06:16:42 DEBUG : file1: reading active writers 2022/05/18 06:16:42 DEBUG : file1: active writers 1 2022/05/18 06:16:42 DEBUG : file2: reading active writers 2022/05/18 06:16:42 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:43 DEBUG : : Looking for writers 2022/05/18 06:16:43 DEBUG : file2: reading active writers 2022/05/18 06:16:43 DEBUG : file1: reading active writers 2022/05/18 06:16:43 DEBUG : file1: active writers 1 2022/05/18 06:16:43 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:44 DEBUG : : Looking for writers 2022/05/18 06:16:44 DEBUG : file1: reading active writers 2022/05/18 06:16:44 DEBUG : file1: active writers 1 2022/05/18 06:16:44 DEBUG : file2: reading active writers 2022/05/18 06:16:44 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:45 DEBUG : : Looking for writers 2022/05/18 06:16:45 DEBUG : file1: reading active writers 2022/05/18 06:16:45 DEBUG : file1: active writers 1 2022/05/18 06:16:45 DEBUG : file2: reading active writers 2022/05/18 06:16:45 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:46 DEBUG : : Looking for writers 2022/05/18 06:16:46 DEBUG : file1: reading active writers 2022/05/18 06:16:46 DEBUG : file1: active writers 1 2022/05/18 06:16:46 DEBUG : file2: reading active writers 2022/05/18 06:16:46 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:47 DEBUG : : Looking for writers 2022/05/18 06:16:47 DEBUG : file1: reading active writers 2022/05/18 06:16:47 DEBUG : file1: active writers 1 2022/05/18 06:16:47 DEBUG : file2: reading active writers 2022/05/18 06:16:47 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:48 DEBUG : : Looking for writers 2022/05/18 06:16:48 DEBUG : file1: reading active writers 2022/05/18 06:16:48 DEBUG : file1: active writers 1 2022/05/18 06:16:48 DEBUG : file2: reading active writers 2022/05/18 06:16:48 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:49 DEBUG : : Looking for writers 2022/05/18 06:16:49 DEBUG : file1: reading active writers 2022/05/18 06:16:49 DEBUG : file1: active writers 1 2022/05/18 06:16:49 DEBUG : file2: reading active writers 2022/05/18 06:16:49 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:50 DEBUG : : Looking for writers 2022/05/18 06:16:50 DEBUG : file1: reading active writers 2022/05/18 06:16:50 DEBUG : file1: active writers 1 2022/05/18 06:16:50 DEBUG : file2: reading active writers 2022/05/18 06:16:50 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:51 DEBUG : : Looking for writers 2022/05/18 06:16:51 DEBUG : file1: reading active writers 2022/05/18 06:16:51 DEBUG : file1: active writers 1 2022/05/18 06:16:51 DEBUG : file2: reading active writers 2022/05/18 06:16:51 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:52 DEBUG : : Looking for writers 2022/05/18 06:16:52 DEBUG : file2: reading active writers 2022/05/18 06:16:52 DEBUG : file1: reading active writers 2022/05/18 06:16:52 DEBUG : file1: active writers 1 2022/05/18 06:16:52 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:53 DEBUG : : Looking for writers 2022/05/18 06:16:53 DEBUG : file1: reading active writers 2022/05/18 06:16:53 DEBUG : file1: active writers 1 2022/05/18 06:16:53 DEBUG : file2: reading active writers 2022/05/18 06:16:53 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:54 DEBUG : : Looking for writers 2022/05/18 06:16:54 DEBUG : file2: reading active writers 2022/05/18 06:16:54 DEBUG : file1: reading active writers 2022/05/18 06:16:54 DEBUG : file1: active writers 1 2022/05/18 06:16:54 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:55 DEBUG : : Looking for writers 2022/05/18 06:16:55 DEBUG : file1: reading active writers 2022/05/18 06:16:55 DEBUG : file1: active writers 1 2022/05/18 06:16:55 DEBUG : file2: reading active writers 2022/05/18 06:16:55 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:56 DEBUG : : Looking for writers 2022/05/18 06:16:56 DEBUG : file2: reading active writers 2022/05/18 06:16:56 DEBUG : file1: reading active writers 2022/05/18 06:16:56 DEBUG : file1: active writers 1 2022/05/18 06:16:56 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:57 DEBUG : : Looking for writers 2022/05/18 06:16:57 DEBUG : file1: reading active writers 2022/05/18 06:16:57 DEBUG : file1: active writers 1 2022/05/18 06:16:57 DEBUG : file2: reading active writers 2022/05/18 06:16:57 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:58 DEBUG : : Looking for writers 2022/05/18 06:16:58 DEBUG : file1: reading active writers 2022/05/18 06:16:58 DEBUG : file1: active writers 1 2022/05/18 06:16:58 DEBUG : file2: reading active writers 2022/05/18 06:16:58 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:59 DEBUG : : Looking for writers 2022/05/18 06:16:59 DEBUG : file1: reading active writers 2022/05/18 06:16:59 DEBUG : file1: active writers 1 2022/05/18 06:16:59 DEBUG : file2: reading active writers 2022/05/18 06:16:59 DEBUG : Still 1 writers active and 1 cache items in use, waiting 1s 2022/05/18 06:16:59 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item file1 not removed, freed 0 bytes 2022/05/18 06:16:59 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item file2 not removed, freed 0 bytes 2022/05/18 06:16:59 INFO : vfs cache: cleaned: objects 2 (was 2) in use 1, to upload 0, uploading 0, total size 0 (was 0) 2022/05/18 06:17:00 ERROR : Exiting even though 1 writers active and 1 cache items in use after 30s Cache{ "file1": &{c:0xc0005f29c0 mu:{state:0 sema:0} cond:0xc00077ed80 name:file1 opens:1 downloaders: o: fd:0xc0004f8098 modified:true info:{ModTime:{wall:13878195946436424975 ext:316405292711 loc:0x2b8b560} ATime:{wall:13878195946436424975 ext:316405292711 loc:0x2b8b560} Size:0 Rs:[] Fingerprint: Dirty:true} writeBackID:1 pendingAccesses:0 beingReset:false}, "file2": &{c:0xc0005f29c0 mu:{state:0 sema:0} cond:0xc00094cf80 name:file2 opens:0 downloaders: o: fd: modified:false info:{ModTime:{wall:13878195865515004431 ext:241014508957 loc:0x2b8b560} ATime:{wall:13878195865515094679 ext:241014599206 loc:0x2b8b560} Size:0 Rs:[] Fingerprint: Dirty:false} writeBackID:2 pendingAccesses:0 beingReset:false}, } 2022/05/18 06:17:00 DEBUG : >WaitForWriters: 2022/05/18 06:17:00 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:00 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:00 ERROR : : error listing: directory not found --- FAIL: TestRWFileModTimeWithOpenWriters (32.76s) === RUN TestVFSNew vfs_test.go:134: Error Trace: vfs_test.go:134 vfs_test.go:137 Error: Not equal: expected: 0 actual : 5 Test: TestVFSNew run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:17:00 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache vfs_test.go:134: Error Trace: vfs_test.go:134 vfs_test.go:147 Error: Not equal: expected: 1 actual : 5 Test: TestVFSNew 2022/05/18 06:17:00 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache vfs_test.go:134: Error Trace: vfs_test.go:134 vfs_test.go:154 Error: Not equal: expected: 1 actual : 5 Test: TestVFSNew vfs_test.go:134: Error Trace: vfs_test.go:134 vfs_test.go:159 Error: Not equal: expected: 1 actual : 5 Test: TestVFSNew 2022/05/18 06:17:00 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:17:00 DEBUG : : Looking for writers 2022/05/18 06:17:00 DEBUG : >WaitForWriters: 2022/05/18 06:17:00 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:01 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:01 ERROR : : error listing: directory not found vfs_test.go:134: Error Trace: vfs_test.go:134 vfs_test.go:163 Error: Not equal: expected: 0 actual : 5 Test: TestVFSNew --- FAIL: TestVFSNew (0.62s) === RUN TestVFSStat run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:17:01 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:17:01 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:02 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:02 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:17:03 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:17:03 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:17:03 DEBUG : : Looking for writers 2022/05/18 06:17:03 DEBUG : >WaitForWriters: 2022/05/18 06:17:03 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:04 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:04 ERROR : : error listing: directory not found --- FAIL: TestVFSStat (3.14s) === RUN TestVFSStatParent run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:17:04 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:17:04 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:05 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:05 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:17:06 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:17:06 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:17:06 DEBUG : : Looking for writers 2022/05/18 06:17:06 DEBUG : >WaitForWriters: 2022/05/18 06:17:06 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:07 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:07 ERROR : : error listing: directory not found --- FAIL: TestVFSStatParent (2.76s) === RUN TestVFSOpenFile run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:17:07 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:17:07 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:07 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:07 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:17:09 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:17:09 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:17:09 DEBUG : : Looking for writers 2022/05/18 06:17:09 DEBUG : >WaitForWriters: 2022/05/18 06:17:09 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:09 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:09 ERROR : : error listing: directory not found --- FAIL: TestVFSOpenFile (2.74s) === RUN TestVFSRename run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:17:09 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:17:09 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:10 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:10 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:17:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) run.go:239: Failed to mkdir "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'": failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:17:12 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:17:12 DEBUG : : Looking for writers 2022/05/18 06:17:12 DEBUG : >WaitForWriters: 2022/05/18 06:17:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:12 ERROR : : error listing: directory not found --- FAIL: TestVFSRename (2.69s) === RUN TestWriteFileHandleMethods run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:17:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:17:12 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2022/05/18 06:17:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:13 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:13 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2022/05/18 06:17:13 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:17:13 DEBUG : file1: >Open: fd=file1 (w), err= 2022/05/18 06:17:13 DEBUG : file1: >OpenFile: fd=file1 (w), err= 2022/05/18 06:17:13 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:17:13 ERROR : file1: WriteFileHandle: Read: Can't read and write to file without --vfs-cache-mode >= minimal 2022/05/18 06:17:13 ERROR : file1: WriteFileHandle: ReadAt: Can't read and write to file without --vfs-cache-mode >= minimal 2022/05/18 06:17:13 ERROR : file1: WriteFileHandle: Truncate: Can't change size without --vfs-cache-mode >= writes 2022/05/18 06:17:13 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': File to upload is small (5 bytes), uploading instead of streaming 2022/05/18 06:17:13 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:13 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:13 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:17:15 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:17:15 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:17:15 ERROR : file1: WriteFileHandle.New Rcat failed: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) write_test.go:84: Error Trace: write_test.go:84 Error: Received unexpected error: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) Test: TestWriteFileHandleMethods 2022/05/18 06:17:15 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:15 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:15 ERROR : : error listing: directory not found fstest.go:299: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:302: Flushing the directory cache 2022/05/18 06:17:16 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:17 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:17 ERROR : : error listing: directory not found fstest.go:299: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:302: Flushing the directory cache 2022/05/18 06:17:19 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:20 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:20 ERROR : : error listing: directory not found fstest.go:299: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:302: Flushing the directory cache fstest.go:306: Error Trace: fstest.go:306 fstest.go:339 write_test.go:97 Error: Should be true Test: TestWriteFileHandleMethods Messages: listing wrong, want file1 (5) got fstest.go:204: Not found "file1" fstest.go:207: Error Trace: fstest.go:207 fstest.go:311 fstest.go:339 write_test.go:97 Error: Not equal: expected: 0 actual : 1 Test: TestWriteFileHandleMethods Messages: 1 objects not found 2022/05/18 06:17:24 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2022/05/18 06:17:24 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2022/05/18 06:17:24 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:17:24 DEBUG : file1: >Open: fd=file1 (w), err= 2022/05/18 06:17:24 DEBUG : file1: >OpenFile: fd=file1 (w), err= 2022/05/18 06:17:24 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:17:24 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': File to upload is small (0 bytes), uploading instead of streaming 2022/05/18 06:17:24 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:24 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:24 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:17:26 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:17:26 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:17:26 ERROR : file1: WriteFileHandle.New Rcat failed: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) write_test.go:103: Error Trace: write_test.go:103 Error: Received unexpected error: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) Test: TestWriteFileHandleMethods dir_test.go:252: Error Trace: dir_test.go:252 write_test.go:104 Error: Not equal: expected: []string{"file1,5,false"} actual : []string{"file1,0,false"} Diff: --- Expected +++ Actual @@ -1,3 +1,3 @@ ([]string) (len=1) { - (string) (len=13) "file1,5,false" + (string) (len=13) "file1,0,false" } Test: TestWriteFileHandleMethods 2022/05/18 06:17:26 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2022/05/18 06:17:26 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2022/05/18 06:17:26 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:17:26 DEBUG : file1: >Open: fd=file1 (w), err= 2022/05/18 06:17:26 DEBUG : file1: >OpenFile: fd=file1 (w), err= 2022/05/18 06:17:26 DEBUG : : Added virtual directory entry vAddFile: "file1" write_test.go:111: Error Trace: write_test.go:111 Error: Not equal: expected: *errors.errorString(&errors.errorString{s:"permission denied"}) actual : () Test: TestWriteFileHandleMethods 2022/05/18 06:17:26 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:17:26 DEBUG : : Looking for writers 2022/05/18 06:17:26 DEBUG : file1: reading active writers 2022/05/18 06:17:26 DEBUG : file1: active writers 1 2022/05/18 06:17:26 DEBUG : Still 1 writers active and 0 cache items in use, waiting 10ms 2022/05/18 06:17:26 DEBUG : : Looking for writers 2022/05/18 06:17:26 DEBUG : file1: reading active writers 2022/05/18 06:17:26 DEBUG : file1: active writers 1 2022/05/18 06:17:26 DEBUG : Still 1 writers active and 0 cache items in use, waiting 20ms 2022/05/18 06:17:26 DEBUG : : Looking for writers 2022/05/18 06:17:26 DEBUG : file1: reading active writers 2022/05/18 06:17:26 DEBUG : file1: active writers 1 2022/05/18 06:17:26 DEBUG : Still 1 writers active and 0 cache items in use, waiting 40ms 2022/05/18 06:17:26 DEBUG : : Looking for writers 2022/05/18 06:17:26 DEBUG : file1: reading active writers 2022/05/18 06:17:26 DEBUG : file1: active writers 1 2022/05/18 06:17:26 DEBUG : Still 1 writers active and 0 cache items in use, waiting 80ms 2022/05/18 06:17:26 DEBUG : : Looking for writers 2022/05/18 06:17:26 DEBUG : file1: reading active writers 2022/05/18 06:17:26 DEBUG : file1: active writers 1 2022/05/18 06:17:26 DEBUG : Still 1 writers active and 0 cache items in use, waiting 160ms 2022/05/18 06:17:26 DEBUG : : Looking for writers 2022/05/18 06:17:26 DEBUG : file1: reading active writers 2022/05/18 06:17:26 DEBUG : file1: active writers 1 2022/05/18 06:17:26 DEBUG : Still 1 writers active and 0 cache items in use, waiting 320ms 2022/05/18 06:17:26 DEBUG : : Looking for writers 2022/05/18 06:17:26 DEBUG : file1: reading active writers 2022/05/18 06:17:26 DEBUG : file1: active writers 1 2022/05/18 06:17:26 DEBUG : Still 1 writers active and 0 cache items in use, waiting 640ms 2022/05/18 06:17:27 DEBUG : : Looking for writers 2022/05/18 06:17:27 DEBUG : file1: reading active writers 2022/05/18 06:17:27 DEBUG : file1: active writers 1 2022/05/18 06:17:27 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:28 DEBUG : : Looking for writers 2022/05/18 06:17:28 DEBUG : file1: reading active writers 2022/05/18 06:17:28 DEBUG : file1: active writers 1 2022/05/18 06:17:28 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:29 DEBUG : : Looking for writers 2022/05/18 06:17:29 DEBUG : file1: reading active writers 2022/05/18 06:17:29 DEBUG : file1: active writers 1 2022/05/18 06:17:29 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:30 DEBUG : : Looking for writers 2022/05/18 06:17:30 DEBUG : file1: reading active writers 2022/05/18 06:17:30 DEBUG : file1: active writers 1 2022/05/18 06:17:30 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:31 DEBUG : : Looking for writers 2022/05/18 06:17:31 DEBUG : file1: reading active writers 2022/05/18 06:17:31 DEBUG : file1: active writers 1 2022/05/18 06:17:31 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:32 DEBUG : : Looking for writers 2022/05/18 06:17:32 DEBUG : file1: reading active writers 2022/05/18 06:17:32 DEBUG : file1: active writers 1 2022/05/18 06:17:32 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:33 DEBUG : : Looking for writers 2022/05/18 06:17:33 DEBUG : file1: reading active writers 2022/05/18 06:17:33 DEBUG : file1: active writers 1 2022/05/18 06:17:33 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:34 DEBUG : : Looking for writers 2022/05/18 06:17:34 DEBUG : file1: reading active writers 2022/05/18 06:17:34 DEBUG : file1: active writers 1 2022/05/18 06:17:34 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:35 DEBUG : : Looking for writers 2022/05/18 06:17:35 DEBUG : file1: reading active writers 2022/05/18 06:17:35 DEBUG : file1: active writers 1 2022/05/18 06:17:35 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:36 DEBUG : : Looking for writers 2022/05/18 06:17:36 DEBUG : file1: reading active writers 2022/05/18 06:17:36 DEBUG : file1: active writers 1 2022/05/18 06:17:36 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:37 DEBUG : : Looking for writers 2022/05/18 06:17:37 DEBUG : file1: reading active writers 2022/05/18 06:17:37 DEBUG : file1: active writers 1 2022/05/18 06:17:37 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:38 DEBUG : : Looking for writers 2022/05/18 06:17:38 DEBUG : file1: reading active writers 2022/05/18 06:17:38 DEBUG : file1: active writers 1 2022/05/18 06:17:38 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:39 DEBUG : : Looking for writers 2022/05/18 06:17:39 DEBUG : file1: reading active writers 2022/05/18 06:17:39 DEBUG : file1: active writers 1 2022/05/18 06:17:39 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:40 DEBUG : : Looking for writers 2022/05/18 06:17:40 DEBUG : file1: reading active writers 2022/05/18 06:17:40 DEBUG : file1: active writers 1 2022/05/18 06:17:40 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:41 DEBUG : : Looking for writers 2022/05/18 06:17:41 DEBUG : file1: reading active writers 2022/05/18 06:17:41 DEBUG : file1: active writers 1 2022/05/18 06:17:41 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:42 DEBUG : : Looking for writers 2022/05/18 06:17:42 DEBUG : file1: reading active writers 2022/05/18 06:17:42 DEBUG : file1: active writers 1 2022/05/18 06:17:42 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:43 DEBUG : : Looking for writers 2022/05/18 06:17:43 DEBUG : file1: reading active writers 2022/05/18 06:17:43 DEBUG : file1: active writers 1 2022/05/18 06:17:43 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:44 DEBUG : : Looking for writers 2022/05/18 06:17:44 DEBUG : file1: reading active writers 2022/05/18 06:17:44 DEBUG : file1: active writers 1 2022/05/18 06:17:44 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:45 DEBUG : : Looking for writers 2022/05/18 06:17:45 DEBUG : file1: reading active writers 2022/05/18 06:17:45 DEBUG : file1: active writers 1 2022/05/18 06:17:45 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:46 DEBUG : : Looking for writers 2022/05/18 06:17:46 DEBUG : file1: reading active writers 2022/05/18 06:17:46 DEBUG : file1: active writers 1 2022/05/18 06:17:46 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:47 DEBUG : : Looking for writers 2022/05/18 06:17:47 DEBUG : file1: reading active writers 2022/05/18 06:17:47 DEBUG : file1: active writers 1 2022/05/18 06:17:47 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:48 DEBUG : : Looking for writers 2022/05/18 06:17:48 DEBUG : file1: reading active writers 2022/05/18 06:17:48 DEBUG : file1: active writers 1 2022/05/18 06:17:48 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:49 DEBUG : : Looking for writers 2022/05/18 06:17:49 DEBUG : file1: reading active writers 2022/05/18 06:17:49 DEBUG : file1: active writers 1 2022/05/18 06:17:49 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:50 DEBUG : : Looking for writers 2022/05/18 06:17:50 DEBUG : file1: reading active writers 2022/05/18 06:17:50 DEBUG : file1: active writers 1 2022/05/18 06:17:50 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:51 DEBUG : : Looking for writers 2022/05/18 06:17:51 DEBUG : file1: reading active writers 2022/05/18 06:17:51 DEBUG : file1: active writers 1 2022/05/18 06:17:51 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:52 DEBUG : : Looking for writers 2022/05/18 06:17:52 DEBUG : file1: reading active writers 2022/05/18 06:17:52 DEBUG : file1: active writers 1 2022/05/18 06:17:52 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:53 DEBUG : : Looking for writers 2022/05/18 06:17:53 DEBUG : file1: reading active writers 2022/05/18 06:17:53 DEBUG : file1: active writers 1 2022/05/18 06:17:53 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:54 DEBUG : : Looking for writers 2022/05/18 06:17:54 DEBUG : file1: reading active writers 2022/05/18 06:17:54 DEBUG : file1: active writers 1 2022/05/18 06:17:54 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:55 DEBUG : : Looking for writers 2022/05/18 06:17:55 DEBUG : file1: reading active writers 2022/05/18 06:17:55 DEBUG : file1: active writers 1 2022/05/18 06:17:55 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:17:56 ERROR : Exiting even though 1 writers active and 0 cache items in use after 30s Cache: 2022/05/18 06:17:56 DEBUG : >WaitForWriters: 2022/05/18 06:17:56 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:56 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:56 ERROR : : error listing: directory not found --- FAIL: TestWriteFileHandleMethods (44.27s) === RUN TestWriteFileHandleWriteAt run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:17:56 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:17:56 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2022/05/18 06:17:56 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2022/05/18 06:17:56 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:17:56 DEBUG : file1: >Open: fd=file1 (w), err= 2022/05/18 06:17:56 DEBUG : file1: >OpenFile: fd=file1 (w), err= 2022/05/18 06:17:56 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:17:56 DEBUG : file1: waiting for in-sequence write to 100 for 1s 2022/05/18 06:17:57 DEBUG : file1: aborting in-sequence write wait, off=100 2022/05/18 06:17:57 DEBUG : file1: failed to wait for in-sequence write to 100 2022/05/18 06:17:57 ERROR : file1: WriteFileHandle.Write: can't seek in file without --vfs-cache-mode >= writes 2022/05/18 06:17:57 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': File to upload is small (11 bytes), uploading instead of streaming 2022/05/18 06:17:57 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:17:58 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:17:58 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:17:59 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:17:59 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:17:59 ERROR : file1: WriteFileHandle.New Rcat failed: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) write_test.go:162: Error Trace: write_test.go:162 Error: Received unexpected error: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) Test: TestWriteFileHandleWriteAt 2022/05/18 06:17:59 ERROR : file1: WriteFileHandle.Write: error: Bad file descriptor 2022/05/18 06:17:59 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:18:00 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:18:00 ERROR : : error listing: directory not found fstest.go:299: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:302: Flushing the directory cache 2022/05/18 06:18:01 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:18:02 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:18:02 ERROR : : error listing: directory not found fstest.go:299: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:302: Flushing the directory cache 2022/05/18 06:18:04 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:18:04 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:18:04 ERROR : : error listing: directory not found fstest.go:299: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:302: Flushing the directory cache fstest.go:306: Error Trace: fstest.go:306 fstest.go:339 write_test.go:176 Error: Should be true Test: TestWriteFileHandleWriteAt Messages: listing wrong, want file1 (11) got fstest.go:204: Not found "file1" fstest.go:207: Error Trace: fstest.go:207 fstest.go:311 fstest.go:339 write_test.go:176 Error: Not equal: expected: 0 actual : 1 Test: TestWriteFileHandleWriteAt Messages: 1 objects not found 2022/05/18 06:18:08 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:18:08 DEBUG : : Looking for writers 2022/05/18 06:18:08 DEBUG : file1: reading active writers 2022/05/18 06:18:08 DEBUG : file1: active writers 1 2022/05/18 06:18:08 DEBUG : Still 1 writers active and 0 cache items in use, waiting 10ms 2022/05/18 06:18:08 DEBUG : : Looking for writers 2022/05/18 06:18:08 DEBUG : file1: reading active writers 2022/05/18 06:18:08 DEBUG : file1: active writers 1 2022/05/18 06:18:08 DEBUG : Still 1 writers active and 0 cache items in use, waiting 20ms 2022/05/18 06:18:08 DEBUG : : Looking for writers 2022/05/18 06:18:08 DEBUG : file1: reading active writers 2022/05/18 06:18:08 DEBUG : file1: active writers 1 2022/05/18 06:18:08 DEBUG : Still 1 writers active and 0 cache items in use, waiting 40ms 2022/05/18 06:18:08 DEBUG : : Looking for writers 2022/05/18 06:18:08 DEBUG : file1: reading active writers 2022/05/18 06:18:08 DEBUG : file1: active writers 1 2022/05/18 06:18:08 DEBUG : Still 1 writers active and 0 cache items in use, waiting 80ms 2022/05/18 06:18:09 DEBUG : : Looking for writers 2022/05/18 06:18:09 DEBUG : file1: reading active writers 2022/05/18 06:18:09 DEBUG : file1: active writers 1 2022/05/18 06:18:09 DEBUG : Still 1 writers active and 0 cache items in use, waiting 160ms 2022/05/18 06:18:09 DEBUG : : Looking for writers 2022/05/18 06:18:09 DEBUG : file1: reading active writers 2022/05/18 06:18:09 DEBUG : file1: active writers 1 2022/05/18 06:18:09 DEBUG : Still 1 writers active and 0 cache items in use, waiting 320ms 2022/05/18 06:18:09 DEBUG : : Looking for writers 2022/05/18 06:18:09 DEBUG : file1: reading active writers 2022/05/18 06:18:09 DEBUG : file1: active writers 1 2022/05/18 06:18:09 DEBUG : Still 1 writers active and 0 cache items in use, waiting 640ms 2022/05/18 06:18:10 DEBUG : : Looking for writers 2022/05/18 06:18:10 DEBUG : file1: reading active writers 2022/05/18 06:18:10 DEBUG : file1: active writers 1 2022/05/18 06:18:10 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:11 DEBUG : : Looking for writers 2022/05/18 06:18:11 DEBUG : file1: reading active writers 2022/05/18 06:18:11 DEBUG : file1: active writers 1 2022/05/18 06:18:11 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:12 DEBUG : : Looking for writers 2022/05/18 06:18:12 DEBUG : file1: reading active writers 2022/05/18 06:18:12 DEBUG : file1: active writers 1 2022/05/18 06:18:12 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:13 DEBUG : : Looking for writers 2022/05/18 06:18:13 DEBUG : file1: reading active writers 2022/05/18 06:18:13 DEBUG : file1: active writers 1 2022/05/18 06:18:13 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:14 DEBUG : : Looking for writers 2022/05/18 06:18:14 DEBUG : file1: reading active writers 2022/05/18 06:18:14 DEBUG : file1: active writers 1 2022/05/18 06:18:14 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:15 DEBUG : : Looking for writers 2022/05/18 06:18:15 DEBUG : file1: reading active writers 2022/05/18 06:18:15 DEBUG : file1: active writers 1 2022/05/18 06:18:15 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:16 DEBUG : : Looking for writers 2022/05/18 06:18:16 DEBUG : file1: reading active writers 2022/05/18 06:18:16 DEBUG : file1: active writers 1 2022/05/18 06:18:16 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:17 DEBUG : : Looking for writers 2022/05/18 06:18:17 DEBUG : file1: reading active writers 2022/05/18 06:18:17 DEBUG : file1: active writers 1 2022/05/18 06:18:17 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:18 DEBUG : : Looking for writers 2022/05/18 06:18:18 DEBUG : file1: reading active writers 2022/05/18 06:18:18 DEBUG : file1: active writers 1 2022/05/18 06:18:18 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:19 DEBUG : : Looking for writers 2022/05/18 06:18:19 DEBUG : file1: reading active writers 2022/05/18 06:18:19 DEBUG : file1: active writers 1 2022/05/18 06:18:19 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:20 DEBUG : : Looking for writers 2022/05/18 06:18:20 DEBUG : file1: reading active writers 2022/05/18 06:18:20 DEBUG : file1: active writers 1 2022/05/18 06:18:20 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:21 DEBUG : : Looking for writers 2022/05/18 06:18:21 DEBUG : file1: reading active writers 2022/05/18 06:18:21 DEBUG : file1: active writers 1 2022/05/18 06:18:21 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:22 DEBUG : : Looking for writers 2022/05/18 06:18:22 DEBUG : file1: reading active writers 2022/05/18 06:18:22 DEBUG : file1: active writers 1 2022/05/18 06:18:22 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:23 DEBUG : : Looking for writers 2022/05/18 06:18:23 DEBUG : file1: reading active writers 2022/05/18 06:18:23 DEBUG : file1: active writers 1 2022/05/18 06:18:23 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:24 DEBUG : : Looking for writers 2022/05/18 06:18:24 DEBUG : file1: reading active writers 2022/05/18 06:18:24 DEBUG : file1: active writers 1 2022/05/18 06:18:24 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:25 DEBUG : : Looking for writers 2022/05/18 06:18:25 DEBUG : file1: reading active writers 2022/05/18 06:18:25 DEBUG : file1: active writers 1 2022/05/18 06:18:25 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:26 DEBUG : : Looking for writers 2022/05/18 06:18:26 DEBUG : file1: reading active writers 2022/05/18 06:18:26 DEBUG : file1: active writers 1 2022/05/18 06:18:26 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:27 DEBUG : : Looking for writers 2022/05/18 06:18:27 DEBUG : file1: reading active writers 2022/05/18 06:18:27 DEBUG : file1: active writers 1 2022/05/18 06:18:27 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:28 DEBUG : : Looking for writers 2022/05/18 06:18:28 DEBUG : file1: reading active writers 2022/05/18 06:18:28 DEBUG : file1: active writers 1 2022/05/18 06:18:28 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:29 DEBUG : : Looking for writers 2022/05/18 06:18:29 DEBUG : file1: reading active writers 2022/05/18 06:18:29 DEBUG : file1: active writers 1 2022/05/18 06:18:29 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:30 DEBUG : : Looking for writers 2022/05/18 06:18:30 DEBUG : file1: reading active writers 2022/05/18 06:18:30 DEBUG : file1: active writers 1 2022/05/18 06:18:30 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:31 DEBUG : : Looking for writers 2022/05/18 06:18:31 DEBUG : file1: reading active writers 2022/05/18 06:18:31 DEBUG : file1: active writers 1 2022/05/18 06:18:31 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:32 DEBUG : : Looking for writers 2022/05/18 06:18:32 DEBUG : file1: reading active writers 2022/05/18 06:18:32 DEBUG : file1: active writers 1 2022/05/18 06:18:32 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:33 DEBUG : : Looking for writers 2022/05/18 06:18:33 DEBUG : file1: reading active writers 2022/05/18 06:18:33 DEBUG : file1: active writers 1 2022/05/18 06:18:33 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:34 DEBUG : : Looking for writers 2022/05/18 06:18:34 DEBUG : file1: reading active writers 2022/05/18 06:18:34 DEBUG : file1: active writers 1 2022/05/18 06:18:34 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:35 DEBUG : : Looking for writers 2022/05/18 06:18:35 DEBUG : file1: reading active writers 2022/05/18 06:18:35 DEBUG : file1: active writers 1 2022/05/18 06:18:35 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:36 DEBUG : : Looking for writers 2022/05/18 06:18:36 DEBUG : file1: reading active writers 2022/05/18 06:18:36 DEBUG : file1: active writers 1 2022/05/18 06:18:36 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:37 DEBUG : : Looking for writers 2022/05/18 06:18:37 DEBUG : file1: reading active writers 2022/05/18 06:18:37 DEBUG : file1: active writers 1 2022/05/18 06:18:37 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:38 DEBUG : : Looking for writers 2022/05/18 06:18:38 DEBUG : file1: reading active writers 2022/05/18 06:18:38 DEBUG : file1: active writers 1 2022/05/18 06:18:38 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:38 ERROR : Exiting even though 1 writers active and 0 cache items in use after 30s Cache: 2022/05/18 06:18:38 DEBUG : >WaitForWriters: 2022/05/18 06:18:38 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:18:39 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:18:39 ERROR : : error listing: directory not found --- FAIL: TestWriteFileHandleWriteAt (42.62s) === RUN TestWriteFileHandleFlush run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:18:39 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:18:39 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2022/05/18 06:18:39 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2022/05/18 06:18:39 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:18:39 DEBUG : file1: >Open: fd=file1 (w), err= 2022/05/18 06:18:39 DEBUG : file1: >OpenFile: fd=file1 (w), err= 2022/05/18 06:18:39 DEBUG : file1: WriteFileHandle.Flush unwritten handle, writing 0 bytes to avoid race conditions 2022/05/18 06:18:39 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:18:39 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': File to upload is small (5 bytes), uploading instead of streaming 2022/05/18 06:18:39 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:18:40 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:18:40 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:18:41 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:18:41 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:18:41 ERROR : file1: WriteFileHandle.New Rcat failed: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:18:41 ERROR : file1: WriteFileHandle.Flush error: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) write_test.go:198: Error Trace: write_test.go:198 Error: Received unexpected error: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) Test: TestWriteFileHandleFlush 2022/05/18 06:18:41 DEBUG : file1: WriteFileHandle.Flush nothing to do 2022/05/18 06:18:41 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:18:41 DEBUG : : Looking for writers 2022/05/18 06:18:41 DEBUG : file1: reading active writers 2022/05/18 06:18:41 DEBUG : file1: active writers 1 2022/05/18 06:18:41 DEBUG : Still 1 writers active and 0 cache items in use, waiting 10ms 2022/05/18 06:18:41 DEBUG : : Looking for writers 2022/05/18 06:18:41 DEBUG : file1: reading active writers 2022/05/18 06:18:41 DEBUG : file1: active writers 1 2022/05/18 06:18:41 DEBUG : Still 1 writers active and 0 cache items in use, waiting 20ms 2022/05/18 06:18:41 DEBUG : : Looking for writers 2022/05/18 06:18:41 DEBUG : file1: reading active writers 2022/05/18 06:18:41 DEBUG : file1: active writers 1 2022/05/18 06:18:41 DEBUG : Still 1 writers active and 0 cache items in use, waiting 40ms 2022/05/18 06:18:41 DEBUG : : Looking for writers 2022/05/18 06:18:41 DEBUG : file1: reading active writers 2022/05/18 06:18:41 DEBUG : file1: active writers 1 2022/05/18 06:18:41 DEBUG : Still 1 writers active and 0 cache items in use, waiting 80ms 2022/05/18 06:18:41 DEBUG : : Looking for writers 2022/05/18 06:18:41 DEBUG : file1: reading active writers 2022/05/18 06:18:41 DEBUG : file1: active writers 1 2022/05/18 06:18:41 DEBUG : Still 1 writers active and 0 cache items in use, waiting 160ms 2022/05/18 06:18:41 DEBUG : : Looking for writers 2022/05/18 06:18:41 DEBUG : file1: reading active writers 2022/05/18 06:18:41 DEBUG : file1: active writers 1 2022/05/18 06:18:41 DEBUG : Still 1 writers active and 0 cache items in use, waiting 320ms 2022/05/18 06:18:42 DEBUG : : Looking for writers 2022/05/18 06:18:42 DEBUG : file1: reading active writers 2022/05/18 06:18:42 DEBUG : file1: active writers 1 2022/05/18 06:18:42 DEBUG : Still 1 writers active and 0 cache items in use, waiting 640ms 2022/05/18 06:18:42 DEBUG : : Looking for writers 2022/05/18 06:18:42 DEBUG : file1: reading active writers 2022/05/18 06:18:42 DEBUG : file1: active writers 1 2022/05/18 06:18:42 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:43 DEBUG : : Looking for writers 2022/05/18 06:18:43 DEBUG : file1: reading active writers 2022/05/18 06:18:43 DEBUG : file1: active writers 1 2022/05/18 06:18:43 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:44 DEBUG : : Looking for writers 2022/05/18 06:18:44 DEBUG : file1: reading active writers 2022/05/18 06:18:44 DEBUG : file1: active writers 1 2022/05/18 06:18:44 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:45 DEBUG : : Looking for writers 2022/05/18 06:18:45 DEBUG : file1: reading active writers 2022/05/18 06:18:45 DEBUG : file1: active writers 1 2022/05/18 06:18:45 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:46 DEBUG : : Looking for writers 2022/05/18 06:18:46 DEBUG : file1: reading active writers 2022/05/18 06:18:46 DEBUG : file1: active writers 1 2022/05/18 06:18:46 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:47 DEBUG : : Looking for writers 2022/05/18 06:18:47 DEBUG : file1: reading active writers 2022/05/18 06:18:47 DEBUG : file1: active writers 1 2022/05/18 06:18:47 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:48 DEBUG : : Looking for writers 2022/05/18 06:18:48 DEBUG : file1: reading active writers 2022/05/18 06:18:48 DEBUG : file1: active writers 1 2022/05/18 06:18:48 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:49 DEBUG : : Looking for writers 2022/05/18 06:18:49 DEBUG : file1: reading active writers 2022/05/18 06:18:49 DEBUG : file1: active writers 1 2022/05/18 06:18:49 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:50 DEBUG : : Looking for writers 2022/05/18 06:18:50 DEBUG : file1: reading active writers 2022/05/18 06:18:50 DEBUG : file1: active writers 1 2022/05/18 06:18:50 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:51 DEBUG : : Looking for writers 2022/05/18 06:18:51 DEBUG : file1: reading active writers 2022/05/18 06:18:51 DEBUG : file1: active writers 1 2022/05/18 06:18:51 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:52 DEBUG : : Looking for writers 2022/05/18 06:18:52 DEBUG : file1: reading active writers 2022/05/18 06:18:52 DEBUG : file1: active writers 1 2022/05/18 06:18:52 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:53 DEBUG : : Looking for writers 2022/05/18 06:18:53 DEBUG : file1: reading active writers 2022/05/18 06:18:53 DEBUG : file1: active writers 1 2022/05/18 06:18:53 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:54 DEBUG : : Looking for writers 2022/05/18 06:18:54 DEBUG : file1: reading active writers 2022/05/18 06:18:54 DEBUG : file1: active writers 1 2022/05/18 06:18:54 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:55 DEBUG : : Looking for writers 2022/05/18 06:18:55 DEBUG : file1: reading active writers 2022/05/18 06:18:55 DEBUG : file1: active writers 1 2022/05/18 06:18:55 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:56 DEBUG : : Looking for writers 2022/05/18 06:18:56 DEBUG : file1: reading active writers 2022/05/18 06:18:56 DEBUG : file1: active writers 1 2022/05/18 06:18:56 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:57 DEBUG : : Looking for writers 2022/05/18 06:18:57 DEBUG : file1: reading active writers 2022/05/18 06:18:57 DEBUG : file1: active writers 1 2022/05/18 06:18:57 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:58 DEBUG : : Looking for writers 2022/05/18 06:18:58 DEBUG : file1: reading active writers 2022/05/18 06:18:58 DEBUG : file1: active writers 1 2022/05/18 06:18:58 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:18:59 DEBUG : : Looking for writers 2022/05/18 06:18:59 DEBUG : file1: reading active writers 2022/05/18 06:18:59 DEBUG : file1: active writers 1 2022/05/18 06:18:59 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:00 DEBUG : : Looking for writers 2022/05/18 06:19:00 DEBUG : file1: reading active writers 2022/05/18 06:19:00 DEBUG : file1: active writers 1 2022/05/18 06:19:00 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:01 DEBUG : : Looking for writers 2022/05/18 06:19:01 DEBUG : file1: reading active writers 2022/05/18 06:19:01 DEBUG : file1: active writers 1 2022/05/18 06:19:01 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:02 DEBUG : : Looking for writers 2022/05/18 06:19:02 DEBUG : file1: reading active writers 2022/05/18 06:19:02 DEBUG : file1: active writers 1 2022/05/18 06:19:02 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:03 DEBUG : : Looking for writers 2022/05/18 06:19:03 DEBUG : file1: reading active writers 2022/05/18 06:19:03 DEBUG : file1: active writers 1 2022/05/18 06:19:03 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:04 DEBUG : : Looking for writers 2022/05/18 06:19:04 DEBUG : file1: reading active writers 2022/05/18 06:19:04 DEBUG : file1: active writers 1 2022/05/18 06:19:04 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:05 DEBUG : : Looking for writers 2022/05/18 06:19:05 DEBUG : file1: reading active writers 2022/05/18 06:19:05 DEBUG : file1: active writers 1 2022/05/18 06:19:05 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:06 DEBUG : : Looking for writers 2022/05/18 06:19:06 DEBUG : file1: reading active writers 2022/05/18 06:19:06 DEBUG : file1: active writers 1 2022/05/18 06:19:06 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:07 DEBUG : : Looking for writers 2022/05/18 06:19:07 DEBUG : file1: reading active writers 2022/05/18 06:19:07 DEBUG : file1: active writers 1 2022/05/18 06:19:07 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:08 DEBUG : : Looking for writers 2022/05/18 06:19:08 DEBUG : file1: reading active writers 2022/05/18 06:19:08 DEBUG : file1: active writers 1 2022/05/18 06:19:08 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:09 DEBUG : : Looking for writers 2022/05/18 06:19:09 DEBUG : file1: reading active writers 2022/05/18 06:19:09 DEBUG : file1: active writers 1 2022/05/18 06:19:09 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:10 DEBUG : : Looking for writers 2022/05/18 06:19:10 DEBUG : file1: reading active writers 2022/05/18 06:19:10 DEBUG : file1: active writers 1 2022/05/18 06:19:10 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:11 ERROR : Exiting even though 1 writers active and 0 cache items in use after 30s Cache: 2022/05/18 06:19:11 DEBUG : >WaitForWriters: 2022/05/18 06:19:11 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:19:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:19:12 ERROR : : error listing: directory not found --- FAIL: TestWriteFileHandleFlush (32.67s) === RUN TestWriteFileHandleRelease run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:19:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:19:12 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2022/05/18 06:19:12 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2022/05/18 06:19:12 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:19:12 DEBUG : file1: >Open: fd=file1 (w), err= 2022/05/18 06:19:12 DEBUG : file1: >OpenFile: fd=file1 (w), err= 2022/05/18 06:19:12 DEBUG : file1: WriteFileHandle.Release closing 2022/05/18 06:19:12 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:19:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': File to upload is small (0 bytes), uploading instead of streaming 2022/05/18 06:19:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:19:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:19:12 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:19:14 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:19:14 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:19:14 ERROR : file1: WriteFileHandle.New Rcat failed: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:19:14 ERROR : file1: WriteFileHandle.Release error: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) write_test.go:222: Error Trace: write_test.go:222 Error: Received unexpected error: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) Test: TestWriteFileHandleRelease 2022/05/18 06:19:14 DEBUG : file1: WriteFileHandle.Release nothing to do 2022/05/18 06:19:14 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:19:14 DEBUG : : Looking for writers 2022/05/18 06:19:14 DEBUG : file1: reading active writers 2022/05/18 06:19:14 DEBUG : file1: active writers 1 2022/05/18 06:19:14 DEBUG : Still 1 writers active and 0 cache items in use, waiting 10ms 2022/05/18 06:19:14 DEBUG : : Looking for writers 2022/05/18 06:19:14 DEBUG : file1: reading active writers 2022/05/18 06:19:14 DEBUG : file1: active writers 1 2022/05/18 06:19:14 DEBUG : Still 1 writers active and 0 cache items in use, waiting 20ms 2022/05/18 06:19:14 DEBUG : : Looking for writers 2022/05/18 06:19:14 DEBUG : file1: reading active writers 2022/05/18 06:19:14 DEBUG : file1: active writers 1 2022/05/18 06:19:14 DEBUG : Still 1 writers active and 0 cache items in use, waiting 40ms 2022/05/18 06:19:14 DEBUG : : Looking for writers 2022/05/18 06:19:14 DEBUG : file1: reading active writers 2022/05/18 06:19:14 DEBUG : file1: active writers 1 2022/05/18 06:19:14 DEBUG : Still 1 writers active and 0 cache items in use, waiting 80ms 2022/05/18 06:19:14 DEBUG : : Looking for writers 2022/05/18 06:19:14 DEBUG : file1: reading active writers 2022/05/18 06:19:14 DEBUG : file1: active writers 1 2022/05/18 06:19:14 DEBUG : Still 1 writers active and 0 cache items in use, waiting 160ms 2022/05/18 06:19:14 DEBUG : : Looking for writers 2022/05/18 06:19:14 DEBUG : file1: reading active writers 2022/05/18 06:19:14 DEBUG : file1: active writers 1 2022/05/18 06:19:14 DEBUG : Still 1 writers active and 0 cache items in use, waiting 320ms 2022/05/18 06:19:14 DEBUG : : Looking for writers 2022/05/18 06:19:14 DEBUG : file1: reading active writers 2022/05/18 06:19:14 DEBUG : file1: active writers 1 2022/05/18 06:19:14 DEBUG : Still 1 writers active and 0 cache items in use, waiting 640ms 2022/05/18 06:19:15 DEBUG : : Looking for writers 2022/05/18 06:19:15 DEBUG : file1: reading active writers 2022/05/18 06:19:15 DEBUG : file1: active writers 1 2022/05/18 06:19:15 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:16 DEBUG : : Looking for writers 2022/05/18 06:19:16 DEBUG : file1: reading active writers 2022/05/18 06:19:16 DEBUG : file1: active writers 1 2022/05/18 06:19:16 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:17 DEBUG : : Looking for writers 2022/05/18 06:19:17 DEBUG : file1: reading active writers 2022/05/18 06:19:17 DEBUG : file1: active writers 1 2022/05/18 06:19:17 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:18 DEBUG : : Looking for writers 2022/05/18 06:19:18 DEBUG : file1: reading active writers 2022/05/18 06:19:18 DEBUG : file1: active writers 1 2022/05/18 06:19:18 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:19 DEBUG : : Looking for writers 2022/05/18 06:19:19 DEBUG : file1: reading active writers 2022/05/18 06:19:19 DEBUG : file1: active writers 1 2022/05/18 06:19:19 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:20 DEBUG : : Looking for writers 2022/05/18 06:19:20 DEBUG : file1: reading active writers 2022/05/18 06:19:20 DEBUG : file1: active writers 1 2022/05/18 06:19:20 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:21 DEBUG : : Looking for writers 2022/05/18 06:19:21 DEBUG : file1: reading active writers 2022/05/18 06:19:21 DEBUG : file1: active writers 1 2022/05/18 06:19:21 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:22 DEBUG : : Looking for writers 2022/05/18 06:19:22 DEBUG : file1: reading active writers 2022/05/18 06:19:22 DEBUG : file1: active writers 1 2022/05/18 06:19:22 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:23 DEBUG : : Looking for writers 2022/05/18 06:19:23 DEBUG : file1: reading active writers 2022/05/18 06:19:23 DEBUG : file1: active writers 1 2022/05/18 06:19:23 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:24 DEBUG : : Looking for writers 2022/05/18 06:19:24 DEBUG : file1: reading active writers 2022/05/18 06:19:24 DEBUG : file1: active writers 1 2022/05/18 06:19:24 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:25 DEBUG : : Looking for writers 2022/05/18 06:19:25 DEBUG : file1: reading active writers 2022/05/18 06:19:25 DEBUG : file1: active writers 1 2022/05/18 06:19:25 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:26 DEBUG : : Looking for writers 2022/05/18 06:19:26 DEBUG : file1: reading active writers 2022/05/18 06:19:26 DEBUG : file1: active writers 1 2022/05/18 06:19:26 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:27 DEBUG : : Looking for writers 2022/05/18 06:19:27 DEBUG : file1: reading active writers 2022/05/18 06:19:27 DEBUG : file1: active writers 1 2022/05/18 06:19:27 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:28 DEBUG : : Looking for writers 2022/05/18 06:19:28 DEBUG : file1: reading active writers 2022/05/18 06:19:28 DEBUG : file1: active writers 1 2022/05/18 06:19:28 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:29 DEBUG : : Looking for writers 2022/05/18 06:19:29 DEBUG : file1: reading active writers 2022/05/18 06:19:29 DEBUG : file1: active writers 1 2022/05/18 06:19:29 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:30 DEBUG : : Looking for writers 2022/05/18 06:19:30 DEBUG : file1: reading active writers 2022/05/18 06:19:30 DEBUG : file1: active writers 1 2022/05/18 06:19:30 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:31 DEBUG : : Looking for writers 2022/05/18 06:19:31 DEBUG : file1: reading active writers 2022/05/18 06:19:31 DEBUG : file1: active writers 1 2022/05/18 06:19:31 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:32 DEBUG : : Looking for writers 2022/05/18 06:19:32 DEBUG : file1: reading active writers 2022/05/18 06:19:32 DEBUG : file1: active writers 1 2022/05/18 06:19:32 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:33 DEBUG : : Looking for writers 2022/05/18 06:19:33 DEBUG : file1: reading active writers 2022/05/18 06:19:33 DEBUG : file1: active writers 1 2022/05/18 06:19:33 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:34 DEBUG : : Looking for writers 2022/05/18 06:19:34 DEBUG : file1: reading active writers 2022/05/18 06:19:34 DEBUG : file1: active writers 1 2022/05/18 06:19:34 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:35 DEBUG : : Looking for writers 2022/05/18 06:19:35 DEBUG : file1: reading active writers 2022/05/18 06:19:35 DEBUG : file1: active writers 1 2022/05/18 06:19:35 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:36 DEBUG : : Looking for writers 2022/05/18 06:19:36 DEBUG : file1: reading active writers 2022/05/18 06:19:36 DEBUG : file1: active writers 1 2022/05/18 06:19:36 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:37 DEBUG : : Looking for writers 2022/05/18 06:19:37 DEBUG : file1: reading active writers 2022/05/18 06:19:37 DEBUG : file1: active writers 1 2022/05/18 06:19:37 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:38 DEBUG : : Looking for writers 2022/05/18 06:19:38 DEBUG : file1: reading active writers 2022/05/18 06:19:38 DEBUG : file1: active writers 1 2022/05/18 06:19:38 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:39 DEBUG : : Looking for writers 2022/05/18 06:19:39 DEBUG : file1: reading active writers 2022/05/18 06:19:39 DEBUG : file1: active writers 1 2022/05/18 06:19:39 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:40 DEBUG : : Looking for writers 2022/05/18 06:19:40 DEBUG : file1: reading active writers 2022/05/18 06:19:40 DEBUG : file1: active writers 1 2022/05/18 06:19:40 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:41 DEBUG : : Looking for writers 2022/05/18 06:19:41 DEBUG : file1: reading active writers 2022/05/18 06:19:41 DEBUG : file1: active writers 1 2022/05/18 06:19:41 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:42 DEBUG : : Looking for writers 2022/05/18 06:19:42 DEBUG : file1: reading active writers 2022/05/18 06:19:42 DEBUG : file1: active writers 1 2022/05/18 06:19:42 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:43 DEBUG : : Looking for writers 2022/05/18 06:19:43 DEBUG : file1: reading active writers 2022/05/18 06:19:43 DEBUG : file1: active writers 1 2022/05/18 06:19:43 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:44 ERROR : Exiting even though 1 writers active and 0 cache items in use after 30s Cache: 2022/05/18 06:19:44 DEBUG : >WaitForWriters: 2022/05/18 06:19:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:19:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:19:44 ERROR : : error listing: directory not found --- FAIL: TestWriteFileHandleRelease (32.74s) === RUN TestWriteFileModTimeWithOpenWriters run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:19:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:19:44 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2022/05/18 06:19:44 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2022/05/18 06:19:44 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:19:44 DEBUG : file1: >Open: fd=file1 (w), err= 2022/05/18 06:19:44 DEBUG : file1: >OpenFile: fd=file1 (w), err= 2022/05/18 06:19:44 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:19:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': File to upload is small (2 bytes), uploading instead of streaming 2022/05/18 06:19:44 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:19:45 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:19:45 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:19:47 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:19:47 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:19:47 ERROR : file1: WriteFileHandle.New Rcat failed: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) write_test.go:277: Error Trace: write_test.go:277 Error: Received unexpected error: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) Test: TestWriteFileModTimeWithOpenWriters 2022/05/18 06:19:47 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:19:47 DEBUG : : Looking for writers 2022/05/18 06:19:47 DEBUG : file1: reading active writers 2022/05/18 06:19:47 DEBUG : file1: active writers 1 2022/05/18 06:19:47 DEBUG : Still 1 writers active and 0 cache items in use, waiting 10ms 2022/05/18 06:19:47 DEBUG : : Looking for writers 2022/05/18 06:19:47 DEBUG : file1: reading active writers 2022/05/18 06:19:47 DEBUG : file1: active writers 1 2022/05/18 06:19:47 DEBUG : Still 1 writers active and 0 cache items in use, waiting 20ms 2022/05/18 06:19:47 DEBUG : : Looking for writers 2022/05/18 06:19:47 DEBUG : file1: reading active writers 2022/05/18 06:19:47 DEBUG : file1: active writers 1 2022/05/18 06:19:47 DEBUG : Still 1 writers active and 0 cache items in use, waiting 40ms 2022/05/18 06:19:47 DEBUG : : Looking for writers 2022/05/18 06:19:47 DEBUG : file1: reading active writers 2022/05/18 06:19:47 DEBUG : file1: active writers 1 2022/05/18 06:19:47 DEBUG : Still 1 writers active and 0 cache items in use, waiting 80ms 2022/05/18 06:19:47 DEBUG : : Looking for writers 2022/05/18 06:19:47 DEBUG : file1: reading active writers 2022/05/18 06:19:47 DEBUG : file1: active writers 1 2022/05/18 06:19:47 DEBUG : Still 1 writers active and 0 cache items in use, waiting 160ms 2022/05/18 06:19:47 DEBUG : : Looking for writers 2022/05/18 06:19:47 DEBUG : file1: reading active writers 2022/05/18 06:19:47 DEBUG : file1: active writers 1 2022/05/18 06:19:47 DEBUG : Still 1 writers active and 0 cache items in use, waiting 320ms 2022/05/18 06:19:47 DEBUG : : Looking for writers 2022/05/18 06:19:47 DEBUG : file1: reading active writers 2022/05/18 06:19:47 DEBUG : file1: active writers 1 2022/05/18 06:19:47 DEBUG : Still 1 writers active and 0 cache items in use, waiting 640ms 2022/05/18 06:19:48 DEBUG : : Looking for writers 2022/05/18 06:19:48 DEBUG : file1: reading active writers 2022/05/18 06:19:48 DEBUG : file1: active writers 1 2022/05/18 06:19:48 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:49 DEBUG : : Looking for writers 2022/05/18 06:19:49 DEBUG : file1: reading active writers 2022/05/18 06:19:49 DEBUG : file1: active writers 1 2022/05/18 06:19:49 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:50 DEBUG : : Looking for writers 2022/05/18 06:19:50 DEBUG : file1: reading active writers 2022/05/18 06:19:50 DEBUG : file1: active writers 1 2022/05/18 06:19:50 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:51 DEBUG : : Looking for writers 2022/05/18 06:19:51 DEBUG : file1: reading active writers 2022/05/18 06:19:51 DEBUG : file1: active writers 1 2022/05/18 06:19:51 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:52 DEBUG : : Looking for writers 2022/05/18 06:19:52 DEBUG : file1: reading active writers 2022/05/18 06:19:52 DEBUG : file1: active writers 1 2022/05/18 06:19:52 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:53 DEBUG : : Looking for writers 2022/05/18 06:19:53 DEBUG : file1: reading active writers 2022/05/18 06:19:53 DEBUG : file1: active writers 1 2022/05/18 06:19:53 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:54 DEBUG : : Looking for writers 2022/05/18 06:19:54 DEBUG : file1: reading active writers 2022/05/18 06:19:54 DEBUG : file1: active writers 1 2022/05/18 06:19:54 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:55 DEBUG : : Looking for writers 2022/05/18 06:19:55 DEBUG : file1: reading active writers 2022/05/18 06:19:55 DEBUG : file1: active writers 1 2022/05/18 06:19:55 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:56 DEBUG : : Looking for writers 2022/05/18 06:19:56 DEBUG : file1: reading active writers 2022/05/18 06:19:56 DEBUG : file1: active writers 1 2022/05/18 06:19:56 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:57 DEBUG : : Looking for writers 2022/05/18 06:19:57 DEBUG : file1: reading active writers 2022/05/18 06:19:57 DEBUG : file1: active writers 1 2022/05/18 06:19:57 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:58 DEBUG : : Looking for writers 2022/05/18 06:19:58 DEBUG : file1: reading active writers 2022/05/18 06:19:58 DEBUG : file1: active writers 1 2022/05/18 06:19:58 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:19:59 DEBUG : : Looking for writers 2022/05/18 06:19:59 DEBUG : file1: reading active writers 2022/05/18 06:19:59 DEBUG : file1: active writers 1 2022/05/18 06:19:59 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:00 DEBUG : : Looking for writers 2022/05/18 06:20:00 DEBUG : file1: reading active writers 2022/05/18 06:20:00 DEBUG : file1: active writers 1 2022/05/18 06:20:00 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:01 DEBUG : : Looking for writers 2022/05/18 06:20:01 DEBUG : file1: reading active writers 2022/05/18 06:20:01 DEBUG : file1: active writers 1 2022/05/18 06:20:01 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:02 DEBUG : : Looking for writers 2022/05/18 06:20:02 DEBUG : file1: reading active writers 2022/05/18 06:20:02 DEBUG : file1: active writers 1 2022/05/18 06:20:02 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:03 DEBUG : : Looking for writers 2022/05/18 06:20:03 DEBUG : file1: reading active writers 2022/05/18 06:20:03 DEBUG : file1: active writers 1 2022/05/18 06:20:03 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:04 DEBUG : : Looking for writers 2022/05/18 06:20:04 DEBUG : file1: reading active writers 2022/05/18 06:20:04 DEBUG : file1: active writers 1 2022/05/18 06:20:04 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:05 DEBUG : : Looking for writers 2022/05/18 06:20:05 DEBUG : file1: reading active writers 2022/05/18 06:20:05 DEBUG : file1: active writers 1 2022/05/18 06:20:05 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:06 DEBUG : : Looking for writers 2022/05/18 06:20:06 DEBUG : file1: reading active writers 2022/05/18 06:20:06 DEBUG : file1: active writers 1 2022/05/18 06:20:06 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:07 DEBUG : : Looking for writers 2022/05/18 06:20:07 DEBUG : file1: reading active writers 2022/05/18 06:20:07 DEBUG : file1: active writers 1 2022/05/18 06:20:07 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:08 DEBUG : : Looking for writers 2022/05/18 06:20:08 DEBUG : file1: reading active writers 2022/05/18 06:20:08 DEBUG : file1: active writers 1 2022/05/18 06:20:08 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:09 DEBUG : : Looking for writers 2022/05/18 06:20:09 DEBUG : file1: reading active writers 2022/05/18 06:20:09 DEBUG : file1: active writers 1 2022/05/18 06:20:09 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:10 DEBUG : : Looking for writers 2022/05/18 06:20:10 DEBUG : file1: reading active writers 2022/05/18 06:20:10 DEBUG : file1: active writers 1 2022/05/18 06:20:10 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:11 DEBUG : : Looking for writers 2022/05/18 06:20:11 DEBUG : file1: reading active writers 2022/05/18 06:20:11 DEBUG : file1: active writers 1 2022/05/18 06:20:11 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:12 DEBUG : : Looking for writers 2022/05/18 06:20:12 DEBUG : file1: reading active writers 2022/05/18 06:20:12 DEBUG : file1: active writers 1 2022/05/18 06:20:12 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:13 DEBUG : : Looking for writers 2022/05/18 06:20:13 DEBUG : file1: reading active writers 2022/05/18 06:20:13 DEBUG : file1: active writers 1 2022/05/18 06:20:13 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:14 DEBUG : : Looking for writers 2022/05/18 06:20:14 DEBUG : file1: reading active writers 2022/05/18 06:20:14 DEBUG : file1: active writers 1 2022/05/18 06:20:14 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:15 DEBUG : : Looking for writers 2022/05/18 06:20:15 DEBUG : file1: reading active writers 2022/05/18 06:20:15 DEBUG : file1: active writers 1 2022/05/18 06:20:15 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:16 DEBUG : : Looking for writers 2022/05/18 06:20:16 DEBUG : file1: reading active writers 2022/05/18 06:20:16 DEBUG : file1: active writers 1 2022/05/18 06:20:16 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:17 ERROR : Exiting even though 1 writers active and 0 cache items in use after 30s Cache: 2022/05/18 06:20:17 DEBUG : >WaitForWriters: 2022/05/18 06:20:17 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:20:17 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:20:17 ERROR : : error listing: directory not found --- FAIL: TestWriteFileModTimeWithOpenWriters (32.74s) === RUN TestFileReadAtZeroLength run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:20:17 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:20:17 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2022/05/18 06:20:17 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2022/05/18 06:20:17 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:20:17 DEBUG : file1: >Open: fd=file1 (w), err= 2022/05/18 06:20:17 DEBUG : file1: >OpenFile: fd=file1 (w), err= 2022/05/18 06:20:17 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:20:17 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': File to upload is small (0 bytes), uploading instead of streaming 2022/05/18 06:20:17 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:20:18 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:20:18 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:20:19 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:20:19 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:20:19 ERROR : file1: WriteFileHandle.New Rcat failed: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) write_test.go:305: Error Trace: write_test.go:305 write_test.go:325 Error: Received unexpected error: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) Test: TestFileReadAtZeroLength 2022/05/18 06:20:19 DEBUG : file1: OpenFile: flags=O_RDONLY, perm=---------- 2022/05/18 06:20:19 DEBUG : file1: Open: flags=O_RDONLY 2022/05/18 06:20:24 DEBUG : file1: >Open: fd=, err=file does not exist 2022/05/18 06:20:24 DEBUG : file1: >OpenFile: fd=, err=file does not exist write_test.go:310: Error Trace: write_test.go:310 write_test.go:325 Error: Received unexpected error: file does not exist Test: TestFileReadAtZeroLength 2022/05/18 06:20:24 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:20:24 DEBUG : : Looking for writers 2022/05/18 06:20:24 DEBUG : file1: reading active writers 2022/05/18 06:20:24 DEBUG : file1: active writers 1 2022/05/18 06:20:24 DEBUG : Still 1 writers active and 0 cache items in use, waiting 10ms 2022/05/18 06:20:24 DEBUG : : Looking for writers 2022/05/18 06:20:24 DEBUG : file1: reading active writers 2022/05/18 06:20:24 DEBUG : file1: active writers 1 2022/05/18 06:20:24 DEBUG : Still 1 writers active and 0 cache items in use, waiting 20ms 2022/05/18 06:20:24 DEBUG : : Looking for writers 2022/05/18 06:20:24 DEBUG : file1: reading active writers 2022/05/18 06:20:24 DEBUG : file1: active writers 1 2022/05/18 06:20:24 DEBUG : Still 1 writers active and 0 cache items in use, waiting 40ms 2022/05/18 06:20:24 DEBUG : : Looking for writers 2022/05/18 06:20:24 DEBUG : file1: reading active writers 2022/05/18 06:20:24 DEBUG : file1: active writers 1 2022/05/18 06:20:24 DEBUG : Still 1 writers active and 0 cache items in use, waiting 80ms 2022/05/18 06:20:24 DEBUG : : Looking for writers 2022/05/18 06:20:24 DEBUG : file1: reading active writers 2022/05/18 06:20:24 DEBUG : file1: active writers 1 2022/05/18 06:20:24 DEBUG : Still 1 writers active and 0 cache items in use, waiting 160ms 2022/05/18 06:20:25 DEBUG : : Looking for writers 2022/05/18 06:20:25 DEBUG : file1: reading active writers 2022/05/18 06:20:25 DEBUG : file1: active writers 1 2022/05/18 06:20:25 DEBUG : Still 1 writers active and 0 cache items in use, waiting 320ms 2022/05/18 06:20:25 DEBUG : : Looking for writers 2022/05/18 06:20:25 DEBUG : file1: reading active writers 2022/05/18 06:20:25 DEBUG : file1: active writers 1 2022/05/18 06:20:25 DEBUG : Still 1 writers active and 0 cache items in use, waiting 640ms 2022/05/18 06:20:26 DEBUG : : Looking for writers 2022/05/18 06:20:26 DEBUG : file1: reading active writers 2022/05/18 06:20:26 DEBUG : file1: active writers 1 2022/05/18 06:20:26 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:27 DEBUG : : Looking for writers 2022/05/18 06:20:27 DEBUG : file1: reading active writers 2022/05/18 06:20:27 DEBUG : file1: active writers 1 2022/05/18 06:20:27 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:28 DEBUG : : Looking for writers 2022/05/18 06:20:28 DEBUG : file1: reading active writers 2022/05/18 06:20:28 DEBUG : file1: active writers 1 2022/05/18 06:20:28 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:29 DEBUG : : Looking for writers 2022/05/18 06:20:29 DEBUG : file1: reading active writers 2022/05/18 06:20:29 DEBUG : file1: active writers 1 2022/05/18 06:20:29 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:30 DEBUG : : Looking for writers 2022/05/18 06:20:30 DEBUG : file1: reading active writers 2022/05/18 06:20:30 DEBUG : file1: active writers 1 2022/05/18 06:20:30 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:31 DEBUG : : Looking for writers 2022/05/18 06:20:31 DEBUG : file1: reading active writers 2022/05/18 06:20:31 DEBUG : file1: active writers 1 2022/05/18 06:20:31 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:32 DEBUG : : Looking for writers 2022/05/18 06:20:32 DEBUG : file1: reading active writers 2022/05/18 06:20:32 DEBUG : file1: active writers 1 2022/05/18 06:20:32 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:33 DEBUG : : Looking for writers 2022/05/18 06:20:33 DEBUG : file1: reading active writers 2022/05/18 06:20:33 DEBUG : file1: active writers 1 2022/05/18 06:20:33 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:34 DEBUG : : Looking for writers 2022/05/18 06:20:34 DEBUG : file1: reading active writers 2022/05/18 06:20:34 DEBUG : file1: active writers 1 2022/05/18 06:20:34 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:35 DEBUG : : Looking for writers 2022/05/18 06:20:35 DEBUG : file1: reading active writers 2022/05/18 06:20:35 DEBUG : file1: active writers 1 2022/05/18 06:20:35 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:36 DEBUG : : Looking for writers 2022/05/18 06:20:36 DEBUG : file1: reading active writers 2022/05/18 06:20:36 DEBUG : file1: active writers 1 2022/05/18 06:20:36 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:37 DEBUG : : Looking for writers 2022/05/18 06:20:37 DEBUG : file1: reading active writers 2022/05/18 06:20:37 DEBUG : file1: active writers 1 2022/05/18 06:20:37 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:38 DEBUG : : Looking for writers 2022/05/18 06:20:38 DEBUG : file1: reading active writers 2022/05/18 06:20:38 DEBUG : file1: active writers 1 2022/05/18 06:20:38 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:39 DEBUG : : Looking for writers 2022/05/18 06:20:39 DEBUG : file1: reading active writers 2022/05/18 06:20:39 DEBUG : file1: active writers 1 2022/05/18 06:20:39 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:40 DEBUG : : Looking for writers 2022/05/18 06:20:40 DEBUG : file1: reading active writers 2022/05/18 06:20:40 DEBUG : file1: active writers 1 2022/05/18 06:20:40 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:41 DEBUG : : Looking for writers 2022/05/18 06:20:41 DEBUG : file1: reading active writers 2022/05/18 06:20:41 DEBUG : file1: active writers 1 2022/05/18 06:20:41 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:42 DEBUG : : Looking for writers 2022/05/18 06:20:42 DEBUG : file1: reading active writers 2022/05/18 06:20:42 DEBUG : file1: active writers 1 2022/05/18 06:20:42 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:43 DEBUG : : Looking for writers 2022/05/18 06:20:43 DEBUG : file1: reading active writers 2022/05/18 06:20:43 DEBUG : file1: active writers 1 2022/05/18 06:20:43 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:44 DEBUG : : Looking for writers 2022/05/18 06:20:44 DEBUG : file1: reading active writers 2022/05/18 06:20:44 DEBUG : file1: active writers 1 2022/05/18 06:20:44 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:45 DEBUG : : Looking for writers 2022/05/18 06:20:45 DEBUG : file1: reading active writers 2022/05/18 06:20:45 DEBUG : file1: active writers 1 2022/05/18 06:20:45 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:46 DEBUG : : Looking for writers 2022/05/18 06:20:46 DEBUG : file1: reading active writers 2022/05/18 06:20:46 DEBUG : file1: active writers 1 2022/05/18 06:20:46 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:47 DEBUG : : Looking for writers 2022/05/18 06:20:47 DEBUG : file1: reading active writers 2022/05/18 06:20:47 DEBUG : file1: active writers 1 2022/05/18 06:20:47 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:48 DEBUG : : Looking for writers 2022/05/18 06:20:48 DEBUG : file1: reading active writers 2022/05/18 06:20:48 DEBUG : file1: active writers 1 2022/05/18 06:20:48 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:49 DEBUG : : Looking for writers 2022/05/18 06:20:49 DEBUG : file1: reading active writers 2022/05/18 06:20:49 DEBUG : file1: active writers 1 2022/05/18 06:20:49 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:50 DEBUG : : Looking for writers 2022/05/18 06:20:50 DEBUG : file1: reading active writers 2022/05/18 06:20:50 DEBUG : file1: active writers 1 2022/05/18 06:20:50 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:51 DEBUG : : Looking for writers 2022/05/18 06:20:51 DEBUG : file1: reading active writers 2022/05/18 06:20:51 DEBUG : file1: active writers 1 2022/05/18 06:20:51 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:52 DEBUG : : Looking for writers 2022/05/18 06:20:52 DEBUG : file1: reading active writers 2022/05/18 06:20:52 DEBUG : file1: active writers 1 2022/05/18 06:20:52 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:53 DEBUG : : Looking for writers 2022/05/18 06:20:53 DEBUG : file1: reading active writers 2022/05/18 06:20:53 DEBUG : file1: active writers 1 2022/05/18 06:20:53 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:54 DEBUG : : Looking for writers 2022/05/18 06:20:54 DEBUG : file1: reading active writers 2022/05/18 06:20:54 DEBUG : file1: active writers 1 2022/05/18 06:20:54 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:20:54 ERROR : Exiting even though 1 writers active and 0 cache items in use after 30s Cache: 2022/05/18 06:20:54 DEBUG : >WaitForWriters: 2022/05/18 06:20:54 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:20:55 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:20:55 ERROR : : error listing: directory not found --- FAIL: TestFileReadAtZeroLength (37.71s) === RUN TestFileReadAtNonZeroLength run.go:181: Remote "filefabric root 'rclone-test-guyumar1gucuroj1gejusov5'", Local "Local file system at /tmp/rclone24145073", Modify Window "1s" 2022/05/18 06:20:55 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Re-using VFS from active cache 2022/05/18 06:20:55 DEBUG : file1: OpenFile: flags=O_WRONLY|O_CREATE, perm=-rwxrwxrwx 2022/05/18 06:20:55 DEBUG : file1: Open: flags=O_WRONLY|O_CREATE 2022/05/18 06:20:55 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:20:55 DEBUG : file1: >Open: fd=file1 (w), err= 2022/05/18 06:20:55 DEBUG : file1: >OpenFile: fd=file1 (w), err= 2022/05/18 06:20:55 DEBUG : : Added virtual directory entry vAddFile: "file1" 2022/05/18 06:20:55 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': File to upload is small (100 bytes), uploading instead of streaming 2022/05/18 06:20:55 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:20:55 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:20:55 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: doCreateNewFolder(map[fi_name:rclone-test-guyumar1gucuroj1gejusov5 fi_pid:120673762]) options=[] 2022/05/18 06:20:57 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data), err=Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:20:57 ERROR : file1: Failed to copy: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) 2022/05/18 06:20:57 ERROR : file1: WriteFileHandle.New Rcat failed: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) write_test.go:305: Error Trace: write_test.go:305 write_test.go:329 Error: Received unexpected error: failed to make directory: failed to create directory: Open S3 provider access error. Message from Open S3 provider: "Unexpected response code 503". (error_data) Test: TestFileReadAtNonZeroLength 2022/05/18 06:20:57 DEBUG : file1: OpenFile: flags=O_RDONLY, perm=---------- 2022/05/18 06:20:57 DEBUG : file1: Open: flags=O_RDONLY 2022/05/18 06:21:02 DEBUG : file1: >Open: fd=, err=file does not exist 2022/05/18 06:21:02 DEBUG : file1: >OpenFile: fd=, err=file does not exist write_test.go:310: Error Trace: write_test.go:310 write_test.go:329 Error: Received unexpected error: file does not exist Test: TestFileReadAtNonZeroLength 2022/05/18 06:21:02 DEBUG : WaitForWriters: timeout=30s 2022/05/18 06:21:02 DEBUG : : Looking for writers 2022/05/18 06:21:02 DEBUG : file1: reading active writers 2022/05/18 06:21:02 DEBUG : file1: active writers 1 2022/05/18 06:21:02 DEBUG : Still 1 writers active and 0 cache items in use, waiting 10ms 2022/05/18 06:21:02 DEBUG : : Looking for writers 2022/05/18 06:21:02 DEBUG : file1: reading active writers 2022/05/18 06:21:02 DEBUG : file1: active writers 1 2022/05/18 06:21:02 DEBUG : Still 1 writers active and 0 cache items in use, waiting 20ms 2022/05/18 06:21:02 DEBUG : : Looking for writers 2022/05/18 06:21:02 DEBUG : file1: reading active writers 2022/05/18 06:21:02 DEBUG : file1: active writers 1 2022/05/18 06:21:02 DEBUG : Still 1 writers active and 0 cache items in use, waiting 40ms 2022/05/18 06:21:02 DEBUG : : Looking for writers 2022/05/18 06:21:02 DEBUG : file1: reading active writers 2022/05/18 06:21:02 DEBUG : file1: active writers 1 2022/05/18 06:21:02 DEBUG : Still 1 writers active and 0 cache items in use, waiting 80ms 2022/05/18 06:21:02 DEBUG : : Looking for writers 2022/05/18 06:21:02 DEBUG : file1: reading active writers 2022/05/18 06:21:02 DEBUG : file1: active writers 1 2022/05/18 06:21:02 DEBUG : Still 1 writers active and 0 cache items in use, waiting 160ms 2022/05/18 06:21:02 DEBUG : : Looking for writers 2022/05/18 06:21:02 DEBUG : file1: reading active writers 2022/05/18 06:21:02 DEBUG : file1: active writers 1 2022/05/18 06:21:02 DEBUG : Still 1 writers active and 0 cache items in use, waiting 320ms 2022/05/18 06:21:03 DEBUG : : Looking for writers 2022/05/18 06:21:03 DEBUG : file1: reading active writers 2022/05/18 06:21:03 DEBUG : file1: active writers 1 2022/05/18 06:21:03 DEBUG : Still 1 writers active and 0 cache items in use, waiting 640ms 2022/05/18 06:21:03 DEBUG : : Looking for writers 2022/05/18 06:21:03 DEBUG : file1: reading active writers 2022/05/18 06:21:03 DEBUG : file1: active writers 1 2022/05/18 06:21:03 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:04 DEBUG : : Looking for writers 2022/05/18 06:21:04 DEBUG : file1: reading active writers 2022/05/18 06:21:04 DEBUG : file1: active writers 1 2022/05/18 06:21:04 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:05 DEBUG : : Looking for writers 2022/05/18 06:21:05 DEBUG : file1: reading active writers 2022/05/18 06:21:05 DEBUG : file1: active writers 1 2022/05/18 06:21:05 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:06 DEBUG : : Looking for writers 2022/05/18 06:21:06 DEBUG : file1: reading active writers 2022/05/18 06:21:06 DEBUG : file1: active writers 1 2022/05/18 06:21:06 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:07 DEBUG : : Looking for writers 2022/05/18 06:21:07 DEBUG : file1: reading active writers 2022/05/18 06:21:07 DEBUG : file1: active writers 1 2022/05/18 06:21:07 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:08 DEBUG : : Looking for writers 2022/05/18 06:21:08 DEBUG : file1: reading active writers 2022/05/18 06:21:08 DEBUG : file1: active writers 1 2022/05/18 06:21:08 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:09 DEBUG : : Looking for writers 2022/05/18 06:21:09 DEBUG : file1: reading active writers 2022/05/18 06:21:09 DEBUG : file1: active writers 1 2022/05/18 06:21:09 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:10 DEBUG : : Looking for writers 2022/05/18 06:21:10 DEBUG : file1: reading active writers 2022/05/18 06:21:10 DEBUG : file1: active writers 1 2022/05/18 06:21:10 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:11 DEBUG : : Looking for writers 2022/05/18 06:21:11 DEBUG : file1: reading active writers 2022/05/18 06:21:11 DEBUG : file1: active writers 1 2022/05/18 06:21:11 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:12 DEBUG : : Looking for writers 2022/05/18 06:21:12 DEBUG : file1: reading active writers 2022/05/18 06:21:12 DEBUG : file1: active writers 1 2022/05/18 06:21:12 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:13 DEBUG : : Looking for writers 2022/05/18 06:21:13 DEBUG : file1: reading active writers 2022/05/18 06:21:13 DEBUG : file1: active writers 1 2022/05/18 06:21:13 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:14 DEBUG : : Looking for writers 2022/05/18 06:21:14 DEBUG : file1: reading active writers 2022/05/18 06:21:14 DEBUG : file1: active writers 1 2022/05/18 06:21:14 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:15 DEBUG : : Looking for writers 2022/05/18 06:21:15 DEBUG : file1: reading active writers 2022/05/18 06:21:15 DEBUG : file1: active writers 1 2022/05/18 06:21:15 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:16 DEBUG : : Looking for writers 2022/05/18 06:21:16 DEBUG : file1: reading active writers 2022/05/18 06:21:16 DEBUG : file1: active writers 1 2022/05/18 06:21:16 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:17 DEBUG : : Looking for writers 2022/05/18 06:21:17 DEBUG : file1: reading active writers 2022/05/18 06:21:17 DEBUG : file1: active writers 1 2022/05/18 06:21:17 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:18 DEBUG : : Looking for writers 2022/05/18 06:21:18 DEBUG : file1: reading active writers 2022/05/18 06:21:18 DEBUG : file1: active writers 1 2022/05/18 06:21:18 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:19 DEBUG : : Looking for writers 2022/05/18 06:21:19 DEBUG : file1: reading active writers 2022/05/18 06:21:19 DEBUG : file1: active writers 1 2022/05/18 06:21:19 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:20 DEBUG : : Looking for writers 2022/05/18 06:21:20 DEBUG : file1: reading active writers 2022/05/18 06:21:20 DEBUG : file1: active writers 1 2022/05/18 06:21:20 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:21 DEBUG : : Looking for writers 2022/05/18 06:21:21 DEBUG : file1: reading active writers 2022/05/18 06:21:21 DEBUG : file1: active writers 1 2022/05/18 06:21:21 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:22 DEBUG : : Looking for writers 2022/05/18 06:21:22 DEBUG : file1: reading active writers 2022/05/18 06:21:22 DEBUG : file1: active writers 1 2022/05/18 06:21:22 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:23 DEBUG : : Looking for writers 2022/05/18 06:21:23 DEBUG : file1: reading active writers 2022/05/18 06:21:23 DEBUG : file1: active writers 1 2022/05/18 06:21:23 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:24 DEBUG : : Looking for writers 2022/05/18 06:21:24 DEBUG : file1: reading active writers 2022/05/18 06:21:24 DEBUG : file1: active writers 1 2022/05/18 06:21:24 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:25 DEBUG : : Looking for writers 2022/05/18 06:21:25 DEBUG : file1: reading active writers 2022/05/18 06:21:25 DEBUG : file1: active writers 1 2022/05/18 06:21:25 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:26 DEBUG : : Looking for writers 2022/05/18 06:21:26 DEBUG : file1: reading active writers 2022/05/18 06:21:26 DEBUG : file1: active writers 1 2022/05/18 06:21:26 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:27 DEBUG : : Looking for writers 2022/05/18 06:21:27 DEBUG : file1: reading active writers 2022/05/18 06:21:27 DEBUG : file1: active writers 1 2022/05/18 06:21:27 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:28 DEBUG : : Looking for writers 2022/05/18 06:21:28 DEBUG : file1: reading active writers 2022/05/18 06:21:28 DEBUG : file1: active writers 1 2022/05/18 06:21:28 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:29 DEBUG : : Looking for writers 2022/05/18 06:21:29 DEBUG : file1: reading active writers 2022/05/18 06:21:29 DEBUG : file1: active writers 1 2022/05/18 06:21:29 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:30 DEBUG : : Looking for writers 2022/05/18 06:21:30 DEBUG : file1: reading active writers 2022/05/18 06:21:30 DEBUG : file1: active writers 1 2022/05/18 06:21:30 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:31 DEBUG : : Looking for writers 2022/05/18 06:21:31 DEBUG : file1: reading active writers 2022/05/18 06:21:31 DEBUG : file1: active writers 1 2022/05/18 06:21:31 DEBUG : Still 1 writers active and 0 cache items in use, waiting 1s 2022/05/18 06:21:32 ERROR : Exiting even though 1 writers active and 0 cache items in use after 30s Cache: 2022/05/18 06:21:32 DEBUG : >WaitForWriters: 2022/05/18 06:21:32 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:21:33 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:21:33 ERROR : : error listing: directory not found --- FAIL: TestFileReadAtNonZeroLength (37.71s) FAIL 2022/05/18 06:21:33 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': Purge remote 2022/05/18 06:21:33 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': rpc: getFolderContents(map[count:1000 fi_pid:120673762 options:filelist|fi_id|fi_pid|fi_name|fi_type|fi_size|fi_contenttype|fi_modified|fi_localtime|trash|subfolders subfolders:y]) options=[] 2022/05/18 06:21:33 DEBUG : filefabric root 'rclone-test-guyumar1gucuroj1gejusov5': >rpc: result=Success (ok), err= 2022/05/18 06:21:33 purge failed: directory not found "./vfs.test -test.v -test.timeout 2h0m0s -remote TestFileFabric: -verbose -test.run '^(TestDirCreate|TestDirForgetAll|TestDirForgetPath|TestDirHandleMethods|TestDirHandleReaddir|TestDirHandleReaddirnames|TestDirMethods|TestDirMkdir|TestDirMkdirSub|TestDirOpen|TestDirReadDirAll|TestDirRemove|TestDirRemoveAll|TestDirRemoveName|TestDirRename|TestDirSetModTime|TestDirStat|TestDirWalk|TestFileMethods|TestFileOpen|TestFileOpenRead|TestFileOpenWrite|TestFileReadAtNonZeroLength|TestFileReadAtZeroLength|TestFileRemove|TestFileRemoveAll|TestFileRename|TestFileSetModTime|TestRWFileHandleFlushRead|TestRWFileHandleMethodsRead|TestRWFileHandleMethodsWrite|TestRWFileHandleReadAt|TestRWFileHandleReleaseRead|TestRWFileHandleSeek|TestRWFileHandleSizeCreateExisting|TestRWFileHandleSizeTruncateExisting|TestRWFileHandleWriteAt|TestRWFileHandleWriteNoWrite|TestRWFileModTimeWithOpenWriters|TestRcGetVFS|TestReadFileHandleFlush|TestReadFileHandleMethods|TestReadFileHandleReadAt|TestReadFileHandleRelease|TestReadFileHandleSeek|TestVFSNew|TestVFSOpenFile|TestVFSRename|TestVFSStat|TestVFSStatParent|TestWriteFileHandleFlush|TestWriteFileHandleMethods|TestWriteFileHandleRelease|TestWriteFileHandleWriteAt|TestWriteFileModTimeWithOpenWriters)$/^(cache=full,open=false,write=false|cache=full,open=true,write=false|cache=full,open=true,write=true|cache=off,open=false,write=false|cache=off,open=true,write=false|cache=off,open=true,write=true|full,forceCache=false|minimal,forceCache=false|minimal,forceCache=true|off,forceCache=false|writes,forceCache=false|writes,forceCache=true)$'" - Finished ERROR in 10m22.243637025s (try 2/5): exit status 1: Failed [TestDirHandleMethods TestDirHandleReaddir TestDirHandleReaddirnames TestDirMethods TestDirForgetAll TestDirForgetPath TestDirWalk TestDirSetModTime TestDirStat TestDirReadDirAll TestDirOpen TestDirCreate TestDirMkdir TestDirMkdirSub TestDirRemove TestDirRemoveAll TestDirRemoveName TestDirRename TestFileMethods TestFileSetModTime/cache=off,open=false,write=false TestFileSetModTime/cache=off,open=true,write=false TestFileSetModTime/cache=off,open=true,write=true TestFileSetModTime/cache=full,open=false,write=false TestFileSetModTime/cache=full,open=true,write=false TestFileSetModTime/cache=full,open=true,write=true TestFileOpenRead TestFileOpenWrite TestFileRemove TestFileRemoveAll TestFileOpen TestFileRename/off,forceCache=false TestFileRename/minimal,forceCache=false TestFileRename/minimal,forceCache=true TestFileRename/writes,forceCache=false TestFileRename/writes,forceCache=true TestFileRename/full,forceCache=false TestRcGetVFS TestReadFileHandleMethods TestReadFileHandleSeek TestReadFileHandleReadAt TestReadFileHandleFlush TestReadFileHandleRelease TestRWFileHandleMethodsRead TestRWFileHandleSeek TestRWFileHandleReadAt TestRWFileHandleFlushRead TestRWFileHandleReleaseRead TestRWFileHandleMethodsWrite TestRWFileHandleWriteAt TestRWFileHandleWriteNoWrite TestRWFileHandleSizeTruncateExisting TestRWFileHandleSizeCreateExisting TestRWFileModTimeWithOpenWriters TestVFSNew TestVFSStat TestVFSStatParent TestVFSOpenFile TestVFSRename TestWriteFileHandleMethods TestWriteFileHandleWriteAt TestWriteFileHandleFlush TestWriteFileHandleRelease TestWriteFileModTimeWithOpenWriters TestFileReadAtZeroLength TestFileReadAtNonZeroLength]