do@lenny ~$ sudo rados bench -p scbench-hdd 10 write --no-cleanup hints = 1 Maintaining 16 concurrent writes of 4194304 bytes to objects of size 4194304 for up to 10 seconds or 0 objects Object prefix: benchmark_data_lenny_3637180 sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 0 0 0 0 0 0 - 0 1 16 17 1 3.99925 4 0.854905 0.854905 2 16 20 4 7.99866 12 1.70749 1.1992 3 16 24 8 10.6648 16 2.98984 1.90804 4 16 26 10 9.99819 8 3.75672 2.24896 5 16 29 13 10.3981 12 4.60426 2.55642 6 16 32 16 10.6648 12 1.08795 2.82301 7 16 34 18 10.2839 8 1.35716 2.947 8 16 40 24 11.9979 24 5.58327 3.60565 9 16 43 27 11.9979 12 1.13108 3.66818 10 16 47 31 12.3977 16 6.76096 3.84986 11 16 48 32 11.6343 4 5.62342 3.90528 12 16 48 32 10.6648 0 - 3.90528 13 16 48 32 9.8444 0 - 3.90528 14 15 48 33 9.42689 1.33333 5.38488 3.95012 Total time run: 14.4781 Total writes made: 48 Write size: 4194304 Object size: 4194304 Bandwidth (MB/sec): 13.2614 Stddev Bandwidth: 6.96687 Max bandwidth (MB/sec): 24 Min bandwidth (MB/sec): 0 Average IOPS: 3 Stddev IOPS: 1.77281 Max IOPS: 6 Min IOPS: 0 Average Latency(s): 4.79904 Stddev Latency(s): 2.39027 Max latency(s): 9.87363 Min latency(s): 0.854905 do@lenny ~$ sudo rados bench -p scbench-hdd 10 rand hints = 1 sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 0 12 12 0 0 0 - 0 1 16 19 3 11.9907 12 0.78026 0.756034 2 16 22 6 11.994 12 1.4872 1.10345 3 16 28 12 15.9936 24 2.78407 1.7731 4 16 31 15 14.995 12 3.40679 2.09867 5 16 37 21 16.7951 24 3.9861 2.44746 6 16 40 24 15.9956 12 3.92352 2.58089 7 16 46 30 17.1384 24 2.61075 2.64985 8 16 51 35 17.4956 20 1.70534 2.6997 9 16 54 38 16.8848 12 1.71139 2.67779 10 16 60 44 17.5958 24 1.41101 2.7275 11 15 61 46 16.7234 8 3.04846 2.81068 12 15 61 46 15.3298 0 - 2.81068 13 7 61 54 16.6116 16 2.80271 3.05892 Total time run: 13.3231 Total reads made: 61 Read size: 4194304 Object size: 4194304 Bandwidth (MB/sec): 18.3141 Average IOPS: 4 Stddev IOPS: 1.86396 Max IOPS: 6 Min IOPS: 0 Average Latency(s): 3.31305 Max latency(s): 8.39827 Min latency(s): 0.612968 do@lenny ~$ sudo rados bench -p scbench-hdd 10 seq hints = 1 sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 0 0 0 0 0 0 - 0 1 16 19 3 11.9949 12 0.746218 0.681344 2 16 24 8 15.9955 20 1.96857 1.21254 3 16 28 12 15.9962 16 2.74538 1.63604 4 16 34 18 17.996 24 1.93976 2.11562 5 16 38 22 17.5963 16 3.60988 2.31159 6 16 43 27 17.9962 20 2.51393 2.4823 7 16 48 32 18.2821 20 3.88385 2.64439 8 15 48 33 16.497 4 3.35054 2.66579 9 14 48 34 15.1084 4 3.5924 2.69305 10 6 48 42 16.7969 32 5.34022 3.01666 Total time run: 10.0544 Total reads made: 48 Read size: 4194304 Object size: 4194304 Bandwidth (MB/sec): 19.0961 Average IOPS: 4 Stddev IOPS: 2.14994 Max IOPS: 8 Min IOPS: 1 Average Latency(s): 3.21351 Max latency(s): 6.37929 Min latency(s): 0.626458 do@lenny ~$ sudo rados bench -p scbench-hdd 30 rand hints = 1 sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 0 0 0 0 0 0 - 0 1 16 19 3 11.9948 12 0.703546 0.689649 2 16 25 9 17.9902 24 1.91925 1.27537 3 16 28 12 15.9933 12 2.58048 1.59284 4 16 34 18 17.9935 24 2.41106 2.0864 5 16 40 24 19.1939 24 2.87461 2.26437 6 16 46 30 19.9941 24 1.17153 2.34959 7 16 49 33 18.8519 12 1.77618 2.39425 8 16 55 39 19.4948 24 1.7475 2.49689 9 16 61 45 19.995 24 2.29268 2.60587 10 16 65 49 19.5953 16 4.70785 2.67741 11 16 70 54 19.6317 20 0.846898 2.64328 12 16 75 59 19.6621 20 4.68773 2.6837 13 16 81 65 19.9954 24 0.591887 2.68116 14 16 84 68 19.4241 12 1.23455 2.68813 15 16 90 74 19.729 24 0.746046 2.70456 16 16 94 78 19.4958 16 0.400425 2.75191 17 16 100 84 19.7605 24 0.667928 2.72239 18 16 106 90 19.9958 24 1.25919 2.72179 19 16 111 95 19.9959 20 2.24736 2.75825 2020-03-26 19:59:45.994068 min lat: 0.400425 max lat: 6.62169 avg lat: 2.74983 sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 20 16 115 99 19.7959 16 0.736072 2.74983 21 16 121 105 19.9959 24 6.73841 2.82779 22 15 125 110 19.996 20 7.33018 2.85325 23 16 132 116 20.1699 24 2.1527 2.87804 24 16 137 121 20.1627 20 0.588809 2.87121 25 16 143 127 20.316 24 1.81338 2.90248 26 16 149 133 20.4575 24 1.49447 2.88299 27 16 154 138 20.4404 20 1.68318 2.87505 28 16 159 143 20.4245 20 0.628533 2.87339 29 16 165 149 20.5477 24 1.10452 2.86992 30 16 171 155 20.6626 24 0.547918 2.86919 31 16 172 156 20.1251 4 6.05471 2.88961 32 16 172 156 19.4963 0 - 2.88961 Total time run: 32.8608 Total reads made: 172 Read size: 4194304 Object size: 4194304 Bandwidth (MB/sec): 20.9368 Average IOPS: 5 Stddev IOPS: 1.5606 Max IOPS: 6 Min IOPS: 0 Average Latency(s): 3.02835 Max latency(s): 7.60182 Min latency(s): 0.400425 do@lenny ~$ sudo rados bench -p scbench-hdd 30 seq hints = 1 sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 0 0 0 0 0 0 - 0 1 16 19 3 11.9943 12 0.936671 0.832956 2 16 22 6 11.9958 12 1.67656 1.20164 3 16 26 10 13.3289 16 2.88472 1.71382 4 16 31 15 14.9955 20 3.89267 2.31528 5 16 35 19 15.1959 16 1.89476 2.48036 6 16 40 24 15.9959 20 4.37326 2.73131 7 16 46 30 17.1387 24 4.70735 2.85483 8 16 48 32 15.9964 8 4.15251 2.83948 9 13 48 35 15.5522 12 5.24734 2.93356 10 11 48 37 14.7969 8 4.84108 2.98921 Total time run: 10.4787 Total reads made: 48 Read size: 4194304 Object size: 4194304 Bandwidth (MB/sec): 18.3229 Average IOPS: 4 Stddev IOPS: 1.33749 Max IOPS: 6 Min IOPS: 2 Average Latency(s): 3.33472 Max latency(s): 6.0753 Min latency(s): 0.661349