"go test -v -timeout 1h0m0s -remote TestCompressDrive: -verbose -test.run '^TestIntegration$/^FsMkdir$/^FsChangeNotify$'" - Starting (try 2/5) === RUN TestIntegration fstests.go:418: Using remote "TestCompressDrive:" 2021/12/13 05:46:25 DEBUG : Creating backend with remote "TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7" 2021/12/13 05:46:25 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2021/12/13 05:46:26 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=847840889997, userRateLimitExceeded) 2021/12/13 05:46:26 DEBUG : pacer: Rate limited, increasing sleep to 1.40649899s 2021/12/13 05:46:27 DEBUG : pacer: Reducing sleep to 0s === RUN TestIntegration/FsMkdir === RUN TestIntegration/FsMkdir/FsChangeNotify 2021/12/13 05:46:28 DEBUG : dir: Making directory 2021/12/13 05:46:30 DEBUG : dir/subdir1: Making directory 2021/12/13 05:46:31 DEBUG : dir/subdir3: Making directory 2021/12/13 05:46:31 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:31 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/subdir1" entryType 0 2021/12/13 05:46:31 DEBUG : doChangeNotify("dir/subdir1", 0) 2021/12/13 05:46:31 DEBUG : dir/subdir2: Making directory 2021/12/13 05:46:32 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:32 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/subdir3" entryType 0 2021/12/13 05:46:32 DEBUG : doChangeNotify("dir/subdir3", 0) 2021/12/13 05:46:33 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:33 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/subdir2" entryType 0 2021/12/13 05:46:33 DEBUG : doChangeNotify("dir/subdir2", 0) 2021/12/13 05:46:34 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:35 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:35 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/file2.ZAAAAAAAAAA.gz" entryType 1 2021/12/13 05:46:35 DEBUG : doChangeNotify("dir/file2.ZAAAAAAAAAA.gz.json", 1) 2021/12/13 05:46:36 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:36 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/file2.json" entryType 1 2021/12/13 05:46:36 DEBUG : doChangeNotify("dir/file2.json.json", 1) 2021/12/13 05:46:37 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:38 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:39 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:40 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:40 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/file4.ZAAAAAAAAAA.gz" entryType 1 2021/12/13 05:46:40 DEBUG : doChangeNotify("dir/file4.ZAAAAAAAAAA.gz.json", 1) 2021/12/13 05:46:41 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:41 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/file4.json" entryType 1 2021/12/13 05:46:41 DEBUG : doChangeNotify("dir/file4.json.json", 1) 2021/12/13 05:46:41 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=847840889997, userRateLimitExceeded) 2021/12/13 05:46:41 DEBUG : pacer: Rate limited, increasing sleep to 1.356713994s 2021/12/13 05:46:42 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:42 DEBUG : pacer: Reducing sleep to 0s 2021/12/13 05:46:43 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:43 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=847840889997, userRateLimitExceeded) 2021/12/13 05:46:43 DEBUG : pacer: Rate limited, increasing sleep to 1.400771917s 2021/12/13 05:46:44 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=847840889997, userRateLimitExceeded) 2021/12/13 05:46:44 DEBUG : pacer: Rate limited, increasing sleep to 2.341519416s 2021/12/13 05:46:44 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:45 DEBUG : pacer: Reducing sleep to 0s 2021/12/13 05:46:47 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:48 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:49 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:49 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/file3.ZAAAAAAAAAA.gz" entryType 1 2021/12/13 05:46:49 DEBUG : doChangeNotify("dir/file3.ZAAAAAAAAAA.gz.json", 1) 2021/12/13 05:46:50 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:51 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:51 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/file3.json" entryType 1 2021/12/13 05:46:51 DEBUG : doChangeNotify("dir/file3.json.json", 1) 2021/12/13 05:46:51 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=847840889997, userRateLimitExceeded) 2021/12/13 05:46:51 DEBUG : pacer: Rate limited, increasing sleep to 1.386108007s 2021/12/13 05:46:52 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=847840889997, userRateLimitExceeded) 2021/12/13 05:46:52 DEBUG : pacer: Rate limited, increasing sleep to 2.709064912s 2021/12/13 05:46:52 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:53 DEBUG : pacer: Reducing sleep to 0s fstests.go:846: Try 1/10 waiting for dirChanges and objChanges 2021/12/13 05:46:56 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote fstests.go:846: Try 2/10 waiting for dirChanges and objChanges 2021/12/13 05:46:57 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:58 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:46:59 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote fstests.go:846: Try 3/10 waiting for dirChanges and objChanges 2021/12/13 05:47:00 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:01 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:02 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote fstests.go:846: Try 4/10 waiting for dirChanges and objChanges 2021/12/13 05:47:03 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:04 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:05 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote fstests.go:846: Try 5/10 waiting for dirChanges and objChanges 2021/12/13 05:47:06 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:07 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:08 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote fstests.go:846: Try 6/10 waiting for dirChanges and objChanges 2021/12/13 05:47:09 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:10 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:11 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote fstests.go:846: Try 7/10 waiting for dirChanges and objChanges 2021/12/13 05:47:12 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:13 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:14 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote fstests.go:846: Try 8/10 waiting for dirChanges and objChanges 2021/12/13 05:47:15 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:16 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:17 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote fstests.go:846: Try 9/10 waiting for dirChanges and objChanges 2021/12/13 05:47:18 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:19 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:20 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote fstests.go:850: map[dir/subdir1:{} dir/subdir2:{} dir/subdir3:{}] does not contain [dir/subdir1 dir/subdir3 dir/subdir2] or map[dir/file2.ZAAAAAAAAAA.gz.json:{} dir/file2.json.json:{} dir/file3.ZAAAAAAAAAA.gz.json:{} dir/file3.json.json:{} dir/file4.ZAAAAAAAAAA.gz.json:{} dir/file4.json.json:{}] does not contain [dir/file2 dir/file4 dir/file3] 2021/12/13 05:47:21 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:21 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/file2.json" entryType 1 2021/12/13 05:47:21 DEBUG : doChangeNotify("dir/file2.json.json", 1) 2021/12/13 05:47:22 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:22 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/file2.ZAAAAAAAAAA.gz" entryType 1 2021/12/13 05:47:22 DEBUG : doChangeNotify("dir/file2.ZAAAAAAAAAA.gz.json", 1) 2021/12/13 05:47:23 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:23 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/file4.json" entryType 1 2021/12/13 05:47:23 DEBUG : doChangeNotify("dir/file4.json.json", 1) 2021/12/13 05:47:24 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:24 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/file4.ZAAAAAAAAAA.gz" entryType 1 2021/12/13 05:47:24 DEBUG : doChangeNotify("dir/file4.ZAAAAAAAAAA.gz.json", 1) 2021/12/13 05:47:24 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/file3.json" entryType 1 2021/12/13 05:47:24 DEBUG : doChangeNotify("dir/file3.json.json", 1) 2021/12/13 05:47:25 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:25 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/file3.ZAAAAAAAAAA.gz" entryType 1 2021/12/13 05:47:25 DEBUG : doChangeNotify("dir/file3.ZAAAAAAAAAA.gz.json", 1) 2021/12/13 05:47:26 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:26 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/subdir1" entryType 0 2021/12/13 05:47:26 DEBUG : doChangeNotify("dir/subdir1", 0) 2021/12/13 05:47:26 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/subdir3" entryType 0 2021/12/13 05:47:26 DEBUG : doChangeNotify("dir/subdir3", 0) 2021/12/13 05:47:27 DEBUG : Google drive root 'rclone-test-bebamup1pokazoq2wewicun7': Checking for changes on remote 2021/12/13 05:47:27 DEBUG : dir: Rmdir: contains trashed file: "file3.ZAAAAAAAAAA.gz" 2021/12/13 05:47:27 DEBUG : dir: Rmdir: contains trashed file: "file3.json" 2021/12/13 05:47:27 DEBUG : dir: Rmdir: contains trashed file: "file4.ZAAAAAAAAAA.gz" 2021/12/13 05:47:27 DEBUG : dir: Rmdir: contains trashed file: "file4.json" 2021/12/13 05:47:27 DEBUG : dir: Rmdir: contains trashed file: "file2.ZAAAAAAAAAA.gz" 2021/12/13 05:47:27 DEBUG : dir: Rmdir: contains trashed file: "file2.json" 2021/12/13 05:47:27 DEBUG : dir: Rmdir: contains trashed file: "subdir2" 2021/12/13 05:47:27 DEBUG : dir: Rmdir: contains trashed file: "subdir3" 2021/12/13 05:47:27 DEBUG : dir: Rmdir: contains trashed file: "subdir1" 2021/12/13 05:47:27 NOTICE: Compressed: TestCompressDrive:rclone-test-bebamup1pokazoq2wewicun7: path "dir/subdir2" entryType 0 2021/12/13 05:47:27 DEBUG : doChangeNotify("dir/subdir2", 0) --- FAIL: TestIntegration (63.75s) --- FAIL: TestIntegration/FsMkdir (60.45s) --- FAIL: TestIntegration/FsMkdir/FsChangeNotify (58.99s) FAIL exit status 1 FAIL github.com/rclone/rclone/backend/compress 63.771s "go test -v -timeout 1h0m0s -remote TestCompressDrive: -verbose -test.run '^TestIntegration$/^FsMkdir$/^FsChangeNotify$'" - Finished ERROR in 1m4.66660409s (try 2/5): exit status 1: Failed [TestIntegration/FsMkdir/FsChangeNotify]