All the cache on the processor will have latencies, yes, but nowhere near as much as RAM, because the cache is running ondie and therefore at the same clockspeed as the RAM is.
The Cache is important because when the CPU calls up data to be processed, it stores the current "work unit" in cache. The latest processors from Intel have gotten to the point where they are waiting for data more than actual processing, so by increasing the cache, the processor will be able to work with more data at once. However there are other factors such as the pipeline length, architecture (68k or RISC, etc.).
I don't really think the performance boost by adding cache will really be that great. The whole idea from Intel has been that more mhz=more power. Well now that AMD just put out its 64bit chips, they have to push with something else (rather than raw clock speed), so as a last ditch effort, they bumped up the cache and put out another version of the P4 3.2ghz chip and called it the P4 Emergency edition. Also of note is that the new 64bit compatible procs from AMD have 2MB cache (or at least one of them does).
BTW, if anything is wrong here, feel free to correct it.