Barley any free memory, but top doesn't indicate as such

Fairly new to Linux. Have always worked on managed dedicated boxes with WHM or Plesk, but I decided to have a go at tackling things myself.

I'm running a small Drupal site that sees ~600 visitors a day. It seems my IO have been averaging at around 2-3k, but it spiked to 7k earlier today. I ran the cmd "free" and this is the output:

total used free shared buffers cached

Mem: 435296 429280 6016 0 15336 16980

-/+ buffers/cache: 396964 38332

Swap: 262140 156372 105768

So it seems that the system is always swapping. I run the cmd "top" but there is nothing using more than 1% of memory. When I load a page on the site, apache/httpd comes in and eats up about 7-11%. But how does that explain that I have virtually no free memory?

Here is my my.cnf file

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

# To allow mysqld to connect to a MySQL Cluster management daemon, uncomment
# these lines and adjust the connectstring as needed.
#ndbcluster
#ndb-connectstring="nodeid=4;host=localhost:1186"

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

[ndbd]
# If you are running a MySQL Cluster storage daemon (ndbd) on this machine,
# adjust its connection to the management daemon here.
# Note: ndbd init script requires this to include nodeid!
connect-string="nodeid=2;host=localhost:1186"

[ndb_mgm]
# connection string for MySQL Cluster management tool
connect-string="host=localhost:1186"

I could copy php.ini but it is large. Mem limit is set to 128mb (drupal is a memory hog), but I don't see how that could be the issue?

Hope you have patience for someone trying to learn. Thank you.