"./sync.test -test.v -test.timeout 1h0m0s -remote TestDropbox: -verbose -test.run '^TestFixCase$'" - Starting (try 3/5) 2025/06/19 03:00:28 DEBUG : Creating backend with remote "TestDropbox:rclone-test-sacohuh1wose" 2025/06/19 03:00:28 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2025/06/19 03:00:28 DEBUG : Creating backend with remote "/tmp/rclone2530411256" === RUN TestFixCase run.go:180: Remote "Dropbox root 'rclone-test-sacohuh1wose'", Local "Local file system at /tmp/rclone2530411256", Modify Window "1s" 2025/06/19 03:00:29 DEBUG : EXISTING: Uploading chunk 1/1 2025/06/19 03:00:29 DEBUG : EXISTING: Uploading chunk 2/1 2025/06/19 03:00:30 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Adding "EXISTING" to batch 2025/06/19 03:00:31 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Batch idle for 500ms so committing 2025/06/19 03:00:31 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Committing sync batch length 1 starting with: EXISTING 2025/06/19 03:00:31 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Committed sync batch length 1 starting with: EXISTING 2025/06/19 03:00:32 DEBUG : EXISTINGBUTDIFFERENT: Uploading chunk 1/1 2025/06/19 03:00:32 DEBUG : EXISTINGBUTDIFFERENT: Uploading chunk 2/1 2025/06/19 03:00:33 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Adding "EXISTINGBUTDIFFERENT" to batch 2025/06/19 03:00:33 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Batch idle for 500ms so committing 2025/06/19 03:00:33 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Committing sync batch length 1 starting with: EXISTINGBUTDIFFERENT 2025/06/19 03:00:34 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Committed sync batch length 1 starting with: EXISTINGBUTDIFFERENT 2025/06/19 03:00:35 DEBUG : SUBDIRA/subdirb/SUBDIRC/HELLO: Uploading chunk 1/1 2025/06/19 03:00:35 DEBUG : SUBDIRA/subdirb/SUBDIRC/HELLO: Uploading chunk 2/1 2025/06/19 03:00:35 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Adding "SUBDIRA/subdirb/SUBDIRC/HELLO" to batch 2025/06/19 03:00:36 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Batch idle for 500ms so committing 2025/06/19 03:00:36 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Committing sync batch length 1 starting with: SUBDIRA/subdirb/SUBDIRC/HELLO 2025/06/19 03:00:37 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Committed sync batch length 1 starting with: SUBDIRA/subdirb/SUBDIRC/HELLO 2025/06/19 03:00:37 DEBUG : SUBDIRA/subdirb/SUBDIRC/subdird/filewithoutcasedifferences: Uploading chunk 1/1 2025/06/19 03:00:38 DEBUG : SUBDIRA/subdirb/SUBDIRC/subdird/filewithoutcasedifferences: Uploading chunk 2/1 2025/06/19 03:00:38 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Adding "SUBDIRA/subdirb/SUBDIRC/subdird/filewithoutcasedifferences" to batch 2025/06/19 03:00:39 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Batch idle for 500ms so committing 2025/06/19 03:00:39 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Committing sync batch length 1 starting with: SUBDIRA/subdirb/SUBDIRC/subdird/filewithoutcasedifferences 2025/06/19 03:00:40 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Committed sync batch length 1 starting with: SUBDIRA/subdirb/SUBDIRC/subdird/filewithoutcasedifferences 2025/06/19 03:00:41 DEBUG : existing: Size and modification time the same (differ by -499.999999ms, within tolerance 1s) 2025/06/19 03:00:41 DEBUG : existing: Unchanged skipping 2025/06/19 03:00:41 DEBUG : existingbutdifferent: Sizes differ (src 5 vs dst 8) 2025/06/19 03:00:42 DEBUG : EXISTINGBUTDIFFERENT: moving to existingbutdifferent-rclone-move-wevenab5 2025/06/19 03:00:42 DEBUG : EXISTING: moving to existing-rclone-move-fuqobiz2 2025/06/19 03:00:42 DEBUG : pacer: low level retry 1/10 (error too_many_write_operations/...) 2025/06/19 03:00:42 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2025/06/19 03:00:42 DEBUG : pacer: low level retry 1/10 (error too_many_write_operations/..) 2025/06/19 03:00:42 DEBUG : pacer: Rate limited, increasing sleep to 40ms 2025/06/19 03:00:43 DEBUG : pacer: low level retry 2/10 (error too_many_write_operations/.) 2025/06/19 03:00:43 DEBUG : pacer: Rate limited, increasing sleep to 80ms 2025/06/19 03:00:43 DEBUG : pacer: Reducing sleep to 60ms 2025/06/19 03:00:43 INFO : EXISTINGBUTDIFFERENT: Moved (server-side) to: existingbutdifferent-rclone-move-wevenab5 2025/06/19 03:00:43 DEBUG : EXISTINGBUTDIFFERENT: moving to existingbutdifferent 2025/06/19 03:00:43 DEBUG : pacer: low level retry 2/10 (error too_many_write_operations/..) 2025/06/19 03:00:43 DEBUG : pacer: Rate limited, increasing sleep to 120ms 2025/06/19 03:00:43 DEBUG : pacer: low level retry 3/10 (error too_many_write_operations/) 2025/06/19 03:00:43 DEBUG : pacer: Rate limited, increasing sleep to 240ms 2025/06/19 03:00:43 DEBUG : pacer: low level retry 1/10 (error too_many_write_operations/.) 2025/06/19 03:00:43 DEBUG : pacer: Rate limited, increasing sleep to 480ms 2025/06/19 03:00:43 DEBUG : pacer: low level retry 3/10 (error too_many_write_operations/...) 2025/06/19 03:00:43 DEBUG : pacer: Rate limited, increasing sleep to 960ms 2025/06/19 03:00:44 DEBUG : pacer: low level retry 4/10 (error too_many_write_operations/..) 2025/06/19 03:00:44 DEBUG : pacer: Rate limited, increasing sleep to 1.92s 2025/06/19 03:00:44 DEBUG : pacer: low level retry 2/10 (error too_many_write_operations/..) 2025/06/19 03:00:44 DEBUG : pacer: Rate limited, increasing sleep to 2s 2025/06/19 03:00:46 DEBUG : pacer: Reducing sleep to 1.5s 2025/06/19 03:00:48 DEBUG : pacer: Reducing sleep to 1.125s 2025/06/19 03:00:48 INFO : EXISTING: Moved (server-side) to: existing-rclone-move-fuqobiz2 2025/06/19 03:00:48 DEBUG : EXISTING: moving to existing 2025/06/19 03:00:49 DEBUG : pacer: Reducing sleep to 843.75ms 2025/06/19 03:00:50 DEBUG : pacer: Reducing sleep to 632.8125ms 2025/06/19 03:00:50 INFO : existingbutdifferent-rclone-move-wevenab5: Moved (server-side) to: existingbutdifferent 2025/06/19 03:00:50 INFO : EXISTINGBUTDIFFERENT: Moved (server-side) to: existingbutdifferent 2025/06/19 03:00:50 INFO : EXISTINGBUTDIFFERENT: Fixed case by renaming to: existingbutdifferent 2025/06/19 03:00:51 DEBUG : pacer: low level retry 1/10 (error too_many_write_operations/) 2025/06/19 03:00:51 DEBUG : pacer: Rate limited, increasing sleep to 1.265625s 2025/06/19 03:00:51 DEBUG : pacer: Reducing sleep to 949.21875ms 2025/06/19 03:00:51 INFO : existing-rclone-move-fuqobiz2: Moved (server-side) to: existing 2025/06/19 03:00:51 INFO : EXISTING: Moved (server-side) to: existing 2025/06/19 03:00:51 INFO : EXISTING: Fixed case by renaming to: existing 2025/06/19 03:00:52 DEBUG : pacer: Reducing sleep to 711.914062ms 2025/06/19 03:00:52 DEBUG : existingbutdifferent: Uploading chunk 1/1 2025/06/19 03:00:54 DEBUG : pacer: Reducing sleep to 533.935546ms 2025/06/19 03:00:54 DEBUG : existingbutdifferent: Uploading chunk 2/1 2025/06/19 03:00:54 DEBUG : pacer: Reducing sleep to 400.451659ms 2025/06/19 03:00:54 INFO : SUBDIRA: Fixed case by renaming to: subdira 2025/06/19 03:00:55 DEBUG : pacer: Reducing sleep to 300.338744ms 2025/06/19 03:00:55 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Adding "existingbutdifferent" to batch 2025/06/19 03:00:55 DEBUG : pacer: Reducing sleep to 225.254058ms 2025/06/19 03:00:55 DEBUG : pacer: Reducing sleep to 168.940543ms 2025/06/19 03:00:55 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Batch idle for 500ms so committing 2025/06/19 03:00:55 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Committing sync batch length 1 starting with: existingbutdifferent 2025/06/19 03:00:56 DEBUG : pacer: low level retry 1/10 (error too_many_write_operations/...) 2025/06/19 03:00:56 DEBUG : pacer: Rate limited, increasing sleep to 337.881086ms 2025/06/19 03:00:57 DEBUG : pacer: Reducing sleep to 253.410814ms 2025/06/19 03:00:58 DEBUG : pacer: Reducing sleep to 190.05811ms 2025/06/19 03:00:59 DEBUG : pacer: Reducing sleep to 142.543582ms 2025/06/19 03:00:59 ERROR : Dropbox root 'rclone-test-sacohuh1wose': sync batch commit: failed to commit batch length 1: batch had 1 errors: last error: upload failed: too_many_write_operations 2025/06/19 03:00:59 ERROR : existingbutdifferent: Failed to copy: upload failed: batch upload failed: upload failed: too_many_write_operations 2025/06/19 03:01:00 DEBUG : pacer: Reducing sleep to 106.907686ms 2025/06/19 03:01:00 INFO : SUBDIRA/subdirb: Fixed case by renaming to: subdira/subdirb 2025/06/19 03:01:00 DEBUG : pacer: Reducing sleep to 80.180764ms 2025/06/19 03:01:00 DEBUG : pacer: Reducing sleep to 60.135573ms 2025/06/19 03:01:02 DEBUG : pacer: Reducing sleep to 45.101679ms 2025/06/19 03:01:02 DEBUG : pacer: Reducing sleep to 33.826259ms 2025/06/19 03:01:03 DEBUG : pacer: Reducing sleep to 25.369694ms 2025/06/19 03:01:03 INFO : SUBDIRA/subdirb/SUBDIRC: Fixed case by renaming to: subdira/subdirb/subdirc 2025/06/19 03:01:03 DEBUG : pacer: Reducing sleep to 19.02727ms 2025/06/19 03:01:03 DEBUG : subdira/subdirb/subdirc/hello: Sizes differ (src 5 vs dst 8) 2025/06/19 03:01:04 DEBUG : pacer: Reducing sleep to 14.270452ms 2025/06/19 03:01:04 DEBUG : pacer: low level retry 1/10 (error too_many_write_operations/) 2025/06/19 03:01:04 DEBUG : pacer: Rate limited, increasing sleep to 28.540904ms 2025/06/19 03:01:05 DEBUG : pacer: Reducing sleep to 21.405678ms 2025/06/19 03:01:05 DEBUG : pacer: Reducing sleep to 16.054258ms 2025/06/19 03:01:06 DEBUG : pacer: Reducing sleep to 12.040693ms 2025/06/19 03:01:06 DEBUG : SUBDIRA/subdirb/SUBDIRC/HELLO: moving to subdira/subdirb/subdirc/hello-rclone-move-zanezen3 2025/06/19 03:01:06 DEBUG : pacer: low level retry 1/10 (error too_many_write_operations/.) 2025/06/19 03:01:06 DEBUG : pacer: Rate limited, increasing sleep to 24.081386ms 2025/06/19 03:01:06 DEBUG : pacer: low level retry 1/10 (error too_many_write_operations/...) 2025/06/19 03:01:06 DEBUG : pacer: Rate limited, increasing sleep to 48.162772ms 2025/06/19 03:01:07 DEBUG : pacer: low level retry 2/10 (error too_many_write_operations/..) 2025/06/19 03:01:07 DEBUG : pacer: Rate limited, increasing sleep to 96.325544ms 2025/06/19 03:01:07 DEBUG : pacer: low level retry 3/10 (error too_many_write_operations/..) 2025/06/19 03:01:07 DEBUG : pacer: Rate limited, increasing sleep to 192.651088ms 2025/06/19 03:01:08 DEBUG : pacer: Reducing sleep to 144.488316ms 2025/06/19 03:01:08 INFO : SUBDIRA/subdirb/SUBDIRC/subdird: Fixed case by renaming to: subdira/subdirb/subdirc/subdird 2025/06/19 03:01:08 DEBUG : pacer: Reducing sleep to 108.366237ms 2025/06/19 03:01:08 DEBUG : subdira/subdirb/subdirc/subdird/filewithoutcasedifferences: Sizes differ (src 5 vs dst 8) 2025/06/19 03:01:08 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Waiting for checks to finish 2025/06/19 03:01:08 DEBUG : pacer: Reducing sleep to 81.274677ms 2025/06/19 03:01:08 INFO : SUBDIRA/subdirb/SUBDIRC/HELLO: Moved (server-side) to: subdira/subdirb/subdirc/hello-rclone-move-zanezen3 2025/06/19 03:01:08 DEBUG : SUBDIRA/subdirb/SUBDIRC/HELLO: moving to subdira/subdirb/subdirc/hello 2025/06/19 03:01:09 DEBUG : pacer: Reducing sleep to 60.956007ms 2025/06/19 03:01:09 DEBUG : SUBDIRA/subdirb/SUBDIRC/subdird/filewithoutcasedifferences: moving to subdira/subdirb/subdirc/subdird/filewithoutcasedifferences-rclone-move-fugasav8 2025/06/19 03:01:09 DEBUG : pacer: low level retry 1/10 (error too_many_write_operations/...) 2025/06/19 03:01:09 DEBUG : pacer: Rate limited, increasing sleep to 121.912014ms 2025/06/19 03:01:10 DEBUG : pacer: low level retry 2/10 (error too_many_write_operations/...) 2025/06/19 03:01:10 DEBUG : pacer: Rate limited, increasing sleep to 243.824028ms 2025/06/19 03:01:10 DEBUG : pacer: Reducing sleep to 182.868021ms 2025/06/19 03:01:10 INFO : subdira/subdirb/subdirc/hello-rclone-move-zanezen3: Moved (server-side) to: subdira/subdirb/subdirc/hello 2025/06/19 03:01:10 INFO : SUBDIRA/subdirb/SUBDIRC/HELLO: Moved (server-side) to: subdira/subdirb/subdirc/hello 2025/06/19 03:01:10 INFO : SUBDIRA/subdirb/SUBDIRC/HELLO: Fixed case by renaming to: subdira/subdirb/subdirc/hello 2025/06/19 03:01:10 DEBUG : pacer: low level retry 3/10 (error too_many_write_operations/...) 2025/06/19 03:01:10 DEBUG : pacer: Rate limited, increasing sleep to 365.736042ms 2025/06/19 03:01:10 DEBUG : pacer: Reducing sleep to 274.302031ms 2025/06/19 03:01:10 DEBUG : subdira/subdirb/subdirc/hello: Uploading chunk 1/1 2025/06/19 03:01:11 DEBUG : pacer: Reducing sleep to 205.726523ms 2025/06/19 03:01:11 DEBUG : subdira/subdirb/subdirc/hello: Uploading chunk 2/1 2025/06/19 03:01:11 DEBUG : pacer: Reducing sleep to 154.294892ms 2025/06/19 03:01:11 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Adding "subdira/subdirb/subdirc/hello" to batch 2025/06/19 03:01:11 DEBUG : pacer: Reducing sleep to 115.721169ms 2025/06/19 03:01:11 INFO : SUBDIRA/subdirb/SUBDIRC/subdird/filewithoutcasedifferences: Moved (server-side) to: subdira/subdirb/subdirc/subdird/filewithoutcasedifferences-rclone-move-fugasav8 2025/06/19 03:01:11 DEBUG : SUBDIRA/subdirb/SUBDIRC/subdird/filewithoutcasedifferences: moving to subdira/subdirb/subdirc/subdird/filewithoutcasedifferences 2025/06/19 03:01:12 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Batch idle for 500ms so committing 2025/06/19 03:01:12 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Committing sync batch length 1 starting with: subdira/subdirb/subdirc/hello 2025/06/19 03:01:13 DEBUG : pacer: Reducing sleep to 86.790876ms 2025/06/19 03:01:13 INFO : subdira/subdirb/subdirc/subdird/filewithoutcasedifferences-rclone-move-fugasav8: Moved (server-side) to: subdira/subdirb/subdirc/subdird/filewithoutcasedifferences 2025/06/19 03:01:13 INFO : SUBDIRA/subdirb/SUBDIRC/subdird/filewithoutcasedifferences: Moved (server-side) to: subdira/subdirb/subdirc/subdird/filewithoutcasedifferences 2025/06/19 03:01:13 INFO : SUBDIRA/subdirb/SUBDIRC/subdird/filewithoutcasedifferences: Fixed case by renaming to: subdira/subdirb/subdirc/subdird/filewithoutcasedifferences 2025/06/19 03:01:13 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Waiting for transfers to finish 2025/06/19 03:01:13 DEBUG : pacer: Reducing sleep to 65.093157ms 2025/06/19 03:01:13 DEBUG : subdira/subdirb/subdirc/subdird/filewithoutcasedifferences: Uploading chunk 1/1 2025/06/19 03:01:13 DEBUG : pacer: Reducing sleep to 48.819867ms 2025/06/19 03:01:13 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Committed sync batch length 1 starting with: subdira/subdirb/subdirc/hello 2025/06/19 03:01:13 DEBUG : subdira/subdirb/subdirc/hello: dropbox = 876764f8f7cb4581b40dd4a7a1be217b2c58ae386ca01c86ea0b30bba6d050ed OK 2025/06/19 03:01:13 INFO : subdira/subdirb/subdirc/hello: Copied (replaced existing) 2025/06/19 03:01:13 DEBUG : pacer: Reducing sleep to 36.6149ms 2025/06/19 03:01:13 DEBUG : subdira/subdirb/subdirc/subdird/filewithoutcasedifferences: Uploading chunk 2/1 2025/06/19 03:01:13 DEBUG : pacer: Reducing sleep to 27.461175ms 2025/06/19 03:01:13 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Adding "subdira/subdirb/subdirc/subdird/filewithoutcasedifferences" to batch 2025/06/19 03:01:14 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Batch idle for 500ms so committing 2025/06/19 03:01:14 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Committing sync batch length 1 starting with: subdira/subdirb/subdirc/subdird/filewithoutcasedifferences 2025/06/19 03:01:16 DEBUG : pacer: Reducing sleep to 20.595881ms 2025/06/19 03:01:16 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Committed sync batch length 1 starting with: subdira/subdirb/subdirc/subdird/filewithoutcasedifferences 2025/06/19 03:01:16 DEBUG : subdira/subdirb/subdirc/subdird/filewithoutcasedifferences: dropbox = 876764f8f7cb4581b40dd4a7a1be217b2c58ae386ca01c86ea0b30bba6d050ed OK 2025/06/19 03:01:16 INFO : subdira/subdirb/subdirc/subdird/filewithoutcasedifferences: Copied (replaced existing) 2025/06/19 03:01:16 ERROR : Dropbox root 'rclone-test-sacohuh1wose': not deleting files as there were IO errors 2025/06/19 03:01:16 ERROR : Dropbox root 'rclone-test-sacohuh1wose': not deleting directories as there were IO errors sync_test.go:2656: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2656 Error: Received unexpected error: upload failed: batch upload failed: upload failed: too_many_write_operations Test: TestFixCase 2025/06/19 03:01:16 DEBUG : pacer: Reducing sleep to 15.44691ms 2025/06/19 03:01:16 DEBUG : pacer: Reducing sleep to 11.585182ms 2025/06/19 03:01:17 DEBUG : pacer: Reducing sleep to 10ms --- FAIL: TestFixCase (57.74s) FAIL 2025/06/19 03:01:26 DEBUG : Dropbox root 'rclone-test-sacohuh1wose': Purge remote "./sync.test -test.v -test.timeout 1h0m0s -remote TestDropbox: -verbose -test.run '^TestFixCase$'" - Finished ERROR in 58.702227813s (try 3/5): exit status 1: Failed [TestFixCase]