"Too small", congratulations.. you found a memory leak.
If memory is allocated, and the program loses track of it.. (i.e: poorly managing pointers, it remains until the program is terminated..)
Because nothing is using it, makes sense that the kernel would swap it off to secondary storage.