Separate Python installations on Ubuntu

I'm fairly new to Python but I've been using it (through Django) to create a number of sites in the past few months. I like it and I want to keep it.

Something that has dawned on me while I've been installing packages through easy_install, pip, apt, etc is the almost overwhelming lack of control I appear to have over the foundations of my applications. If Ubuntu updates its python version, I'd need to re-get all the packages I'm using…

It's not something that would be massively chaotic if it happened today (for five sites) but a year down the line when I've hopefully got a few dozen more, it could cause a week of headaches.

But am I worrying for nothing? Or should I look to maintain the environment myself rather than relying on package managers?

If it's the second - how?