Resolution won't have anything to do with the number of cores you use or have. It's entirely a function of the display, graphics hardware, and the OS's video driver. Since you're apparently using the VESA driver, you're limited to VESA resolutions.
Your Nvidia GTX 260 is apparently not supported by the
nv(4) driver, which is the only other possible solution for Nvidia graphics on OpenBSD. The graphics card is not in OpenBSD's table of known PCI devices, either:
Code:
vga1 at pci2 dev 0 function 0 vendor "NVIDIA", unknown product 0x05e2 rev 0xa1