Python: Cannot allocate memory

Hi all,

I've just changed my django app to use multiprocessing and Pool. On my dev machine (OSX) this work fine. When I deploy to my Linode however, the underlying os.fork() method throws:

OSError: [Errno 12] Cannot allocate memory

Anyone know much about this? My Linode is configured with Ubuntu 11.04. It has 512mb RAM, but this doesn't seem to get used according to top. However, the output from free -m is:

total used free shared buffers cached

Mem: 497 488 9 0 1 42

-/+ buffers/cache: 443 53

Swap: 255 255 0

Which looks rather bad. Anyone got any suggestions? I just don't think all my memory is gone! According to top Apache is holding 17.5% and a few other processes have around 10% in total.

Ta.

2 Replies

Here's some more info. I rebooted the server and reloaded the page that does the processing a few times, logging the memory availability after each reload. It seems that somewhere in apache/python/django it is not freeing up the memory after each load:

total used free shared buffers cached

Mem: 497 213 283 0 66 51

-/+ buffers/cache: 96 401

Swap: 255 0 255

total used free shared buffers cached

Mem: 497 348 149 0 66 82

-/+ buffers/cache: 199 298

Swap: 255 0 255

total used free shared buffers cached

Mem: 497 462 35 0 66 84

-/+ buffers/cache: 310 187

Swap: 255 0 255

total used free shared buffers cached

Mem: 497 486 10 0 11 53

-/+ buffers/cache: 421 76

Swap: 255 6 249

total used free shared buffers cached

Mem: 497 490 7 0 0 19

-/+ buffers/cache: 470 27

Swap: 255 155 100

total used free shared buffers cached

Mem: 497 474 22 0 1 25

-/+ buffers/cache: 448 48

Swap: 255 254 1

@blastStu:

It seems that somewhere in apache/python/django it is not freeing up the memory after each load:

I saw this behavior using multiprocessing pools on my Django website http://zunzun.com, and I kill the process after each use and mod_wsgi respawns a new process for me automatically. That worked, so I stopped investigating.

James

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct