The iPhone OS has always supported multitasking; otherwise you wouldn't be able to receive a call while listening to music! That's also how you can continue listening to music from your iPod app while playing many games. But Apple has limited it to their own applications in the past. (Unless, of course, you did a jailbreak on your phone and ran an app like backgrounder.)
To be fair, there were some good reasons why Apple hasn't enabled multitasking of third-party apps in the past. First, apps that can run in the background without the user's knowledge are a potential opening for malware. Second, once you start loading multiple tasks at once you're going to see a real hit in battery life--which was a sticking point for many iPhone users already. Third, once you have multiple apps running, you have to give the user a new interface to switch between them--likely with additional utilities to allow the user to balance resource usage. Fourth, multiple tasks slow the system down and the processors in the iPhone won't stand up to too great a load.
In general, the lack of multitasking hasn't been a serious issue for most (other than Pandora users). The size of the screen doesn't lead a user to be thinking in terms of multiple windows anyway. But the iPad runs the iPhone OS, and with its larger screen and faster processor there will be much more demand for multitasking. It may be called the iPhone OS, but this upgrade is pointed primarily at iPad users.