Question for the geeks

My assignment here asks about data security; it poses the question of how to store data on a server in a secure manner. This perplexes me somewhat. Say we use some fancy-schmancy data encryption algorithm to encrypt all the data files on the server. Then we need a key to decrypt them back again when we need them, and... where do we store that? It doesn't seem any more secure than just leaving the data plaintext; the key file could just as well be stolen by some attacker as any other file. (Or if we hard-code the key into the program, then the object code could be stolen and reverse-engineered.)

So: is it a stupid question? Is there any way of securing data on a server, or should we concentrate on not giving the attacker access to the file system in the first place?