Write heavy application with +/- 10000k IO rate at peak time

We have a write heavy application that captures huge amounts of data per second. We are doing around 800 queries per second on our MySQL database that we are running on a 2048MB VPS (dedicated to MySQL). Data is also replicated to a equally sized MySQL replication client.

During peek hours our I/O rate goes up to between 9,000k and 10,000k and believe it or not that is pretty normal and not due to swapping or any other problem. We just have too much data to capture for storage and the replication adds even further load due to the binary logs being created.

Question 1: While our application runs well I'm worried that we may be overusing our slice of the server and effecting the contention ratio of other customers on the server. How will I know if this is the case?

Question 2: Does the IO graph only show the usage of my own VPS or that of the IO rate on the actual RAID HDD?

3 Replies

The IO graph is only for your own IO, not other people's. As for if your IO load is acceptable, Linode is kind of vague on that, so you'd have to ask them directly if they're OK with those levels. If memory serves, I think Jeb said that they only object if you're frequently going into the 5 digit IOs for extended periods, but even that wasn't a hard rule, so you'd really have to check with them.

If IO becomes a problem, your IO rate will go down and requests will block, same as on any other system. Time spent blocking on IO will show up in your CPU stats as "IO Wait"… this'll be the "wa" value in vmstat or top, the "iowait" line in munin, etc.

There's fair queuing in place such that other "normal" users shouldn't be too impacted by high IO neighbors, so I wouldn't worry about it too much.

Serial writes might actually be a best-case scenario; the underlying hardware can fake a write better than it can fake a read :-)

Thanks for your input guys. As to the CPU waiting due to IO blocking, it does happen but at fairly low rate. I did a vmstat today (system is only running at 60% its usual workload during this time of the day due to a public holiday).

Result was the following:

$ vmstat 1 20
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  0      0  56184  24044 1813048    0    0     1     0    0    1  1  0 99  0
 0  0      0  58268  24032 1810444    0    0     0  8428 2141 1209  3  2 95  0
 0  0      0  57892  24032 1810880    0    0     0  3580 1492 1170  1  0 98  0
 0  0      0  57892  24032 1810672    0    0     0  6972 1707 1273  4  1 88  7
 0  0      0  57488  24032 1810912    0    0     0  2644 1003  743  1  0 98  0
 0  0      0  57488  24032 1810912    0    0     0     0   28   13  0  0 100  0
 0  0      0  57488  24032 1810960    0    0     0    64   41   36  0  0 100  0
 0  0      0  57488  24032 1810960    0    0     0    20   53   45  0  0 100  0
 0  0      0  57488  24032 1810960    0    0     0    20   59   49  0  0 100  0
 0  0      0  57488  24032 1810960    0    0     0    16   44   39  0  0 95  5
 0  0      0  57788  24032 1810960    0    0     0  3980  228  143  0  0 97  2
 0  0      0  57788  24032 1810960    0    0     0 24276 1241 1088  1  1 92  6
 0  0      0  57788  24032 1810960    0    0     0     0   26   13  0  0 100  0
 0  0      0  57788  24032 1810960    0    0     0     0   25   15  0  0 100  0
 0  0      0  57876  24032 1810976    0    0     0     0   27   20  0  0 100  0
 0  0      0  57876  24032 1810976    0    0     0   108  784  596  1  0 99  0
 0  0      0  57360  24032 1811204    0    0     0   148 1364 1088  1  0 98  0
 0  0      0  57236  24032 1811428    0    0     0  7028 1892 1446  3  1 94  2
 0  0      0  56720  24036 1811816    0    0     0   100 1469 1156  1  0 98  0
 0  0      0  56720  24036 1811816    0    0     0     0   31   18  0  0 100  0

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct