"./sync.test -test.v -test.timeout 1h0m0s -remote TestCompressDrive: -verbose -test.run '^TestSyncConcurrentTruncate$'" - Starting (try 2/5) 2021/12/13 05:58:38 DEBUG : Creating backend with remote "TestCompressDrive:rclone-test-jowetov1docujeb8mumodat4" 2021/12/13 05:58:38 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2021/12/13 05:58:39 DEBUG : Creating backend with remote "/tmp/rclone3689635039" === RUN TestSyncConcurrentTruncate run.go:181: Remote "Compressed: TestCompressDrive:rclone-test-jowetov1docujeb8mumodat4", Local "Local file system at /tmp/rclone3689635039", Modify Window "1ms" 2021/12/13 05:58:57 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=847840889997, userRateLimitExceeded) 2021/12/13 05:58:57 DEBUG : pacer: Rate limited, increasing sleep to 1.923046255s 2021/12/13 05:58:57 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=847840889997, userRateLimitExceeded) 2021/12/13 05:58:57 DEBUG : pacer: Rate limited, increasing sleep to 2.707046836s 2021/12/13 05:58:59 DEBUG : pacer: Reducing sleep to 0s 2021/12/13 05:59:47 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=847840889997, userRateLimitExceeded) 2021/12/13 05:59:47 DEBUG : pacer: Rate limited, increasing sleep to 1.597894812s 2021/12/13 05:59:47 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=847840889997, userRateLimitExceeded) 2021/12/13 05:59:47 DEBUG : pacer: Rate limited, increasing sleep to 2.637245994s 2021/12/13 05:59:49 DEBUG : pacer: Reducing sleep to 0s 2021/12/13 06:00:26 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=847840889997, userRateLimitExceeded) 2021/12/13 06:00:26 DEBUG : pacer: Rate limited, increasing sleep to 1.508544973s 2021/12/13 06:00:27 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=847840889997, userRateLimitExceeded) 2021/12/13 06:00:27 DEBUG : pacer: Rate limited, increasing sleep to 2.754923419s 2021/12/13 06:00:28 DEBUG : pacer: Reducing sleep to 0s 2021/12/13 06:01:07 DEBUG : TestDrive: Loaded invalid token from config file - ignoring 2021/12/13 06:01:07 DEBUG : Config file has changed externaly - reloading 2021/12/13 06:01:07 DEBUG : Saving config "token" in section "TestDrive" of the config file 2021/12/13 06:01:07 DEBUG : TestDrive: Saved new token in config file 2021/12/13 06:01:31 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=847840889997, userRateLimitExceeded) 2021/12/13 06:01:31 DEBUG : pacer: Rate limited, increasing sleep to 1.277299581s 2021/12/13 06:01:32 DEBUG : pacer: Reducing sleep to 0s 2021/12/13 06:01:37 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=847840889997, userRateLimitExceeded) 2021/12/13 06:01:37 DEBUG : pacer: Rate limited, increasing sleep to 1.837613221s 2021/12/13 06:01:38 DEBUG : pacer: Reducing sleep to 0s 2021/12/13 06:01:49 DEBUG : both1: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both11: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both11: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both12: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both12: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both13: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both13: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both14: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both14: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both15: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both15: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both16: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both16: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both17: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both17: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both18: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both18: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both19: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both19: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both2: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both2: Unchanged skipping 2021/12/13 06:01:49 DEBUG : Compressed: TestCompressDrive:rclone-test-jowetov1docujeb8mumodat4: Waiting for checks to finish 2021/12/13 06:01:49 DEBUG : both3: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both3: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both4: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both0: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both4: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both0: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both5: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both5: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both6: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both6: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both7: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both7: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both8: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both8: Unchanged skipping 2021/12/13 06:01:49 DEBUG : both9: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both9: Unchanged skipping 2021/12/13 06:01:49 DEBUG : only0: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only1: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only10: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only11: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only12: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only13: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only14: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only15: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only16: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only17: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only18: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only19: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : both10: Size and modification time the same (differ by -999.999µs, within tolerance 1ms) 2021/12/13 06:01:49 DEBUG : both10: Unchanged skipping 2021/12/13 06:01:49 DEBUG : only2: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only3: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : both1: Unchanged skipping 2021/12/13 06:01:49 DEBUG : only4: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only5: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only6: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only7: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only8: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : only9: Sizes differ (src 0 vs dst 6) 2021/12/13 06:01:49 DEBUG : Compressed: TestCompressDrive:rclone-test-jowetov1docujeb8mumodat4: Waiting for transfers to finish 2021/12/13 06:01:52 DEBUG : only1: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:01:52 INFO : only1: Copied (replaced existing) 2021/12/13 06:01:52 DEBUG : only0: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:01:52 INFO : only0: Copied (replaced existing) 2021/12/13 06:01:52 DEBUG : only11: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:01:52 INFO : only11: Copied (replaced existing) 2021/12/13 06:01:52 DEBUG : only10: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:01:52 INFO : only10: Copied (replaced existing) 2021/12/13 06:01:55 DEBUG : only12: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:01:55 INFO : only12: Copied (replaced existing) 2021/12/13 06:01:56 DEBUG : only13: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:01:56 INFO : only13: Copied (replaced existing) 2021/12/13 06:01:56 DEBUG : only15: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:01:56 INFO : only15: Copied (replaced existing) 2021/12/13 06:01:56 DEBUG : only14: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:01:56 INFO : only14: Copied (replaced existing) 2021/12/13 06:01:59 DEBUG : only16: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:01:59 INFO : only16: Copied (replaced existing) 2021/12/13 06:01:59 DEBUG : only19: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:01:59 INFO : only19: Copied (replaced existing) 2021/12/13 06:01:59 DEBUG : only17: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:01:59 INFO : only17: Copied (replaced existing) 2021/12/13 06:01:59 DEBUG : only18: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:01:59 INFO : only18: Copied (replaced existing) 2021/12/13 06:02:02 DEBUG : only5: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:02:02 INFO : only5: Copied (replaced existing) 2021/12/13 06:02:02 DEBUG : only7: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:02:02 INFO : only7: Copied (replaced existing) 2021/12/13 06:02:02 DEBUG : only6: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:02:02 INFO : only6: Copied (replaced existing) 2021/12/13 06:02:02 DEBUG : only8: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:02:02 INFO : only8: Copied (replaced existing) 2021/12/13 06:02:05 DEBUG : only9: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:02:05 INFO : only9: Copied (replaced existing) 2021/12/13 06:02:06 DEBUG : only4: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:02:06 INFO : only4: Copied (replaced existing) 2021/12/13 06:02:06 DEBUG : only2: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:02:06 INFO : only2: Copied (replaced existing) 2021/12/13 06:02:06 DEBUG : only3: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2021/12/13 06:02:06 INFO : only3: Copied (replaced existing) 2021/12/13 06:02:06 DEBUG : Waiting for deletions to finish 2021/12/13 06:03:24 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=847840889997, userRateLimitExceeded) 2021/12/13 06:03:24 DEBUG : pacer: Rate limited, increasing sleep to 1.942503832s 2021/12/13 06:03:25 DEBUG : pacer: Reducing sleep to 0s --- PASS: TestSyncConcurrentTruncate (309.29s) PASS 2021/12/13 06:03:48 DEBUG : Compressed: TestCompressDrive:rclone-test-jowetov1docujeb8mumodat4: Purge remote "./sync.test -test.v -test.timeout 1h0m0s -remote TestCompressDrive: -verbose -test.run '^TestSyncConcurrentTruncate$'" - Finished OK in 5m11.122044306s (try 2/5)