The danger of "q=$1"; or, why you shouldn't ignore Drupal 404 errors.
Drupal makes your life very easy, but that's no excuse for being lazy. While profiling scripts today, I ran into something that, for whatever reason, never clicked before. I was running Xdebug and KCachegrind on some complex pages to get an idea where I could optimize. Xdebug has a profiler built in that creates files digestible by something like KCachegrind. With profiling on, you get a profile dump file for every PHP script you run. You can in-turn open that file with KCachegrind to get some helpful analysis.
Profiling is a great way to expose those problems that I often ignore...
Using Syslog Module and Splunk to Rock Your Reporting World
Drupal's Syslog module is one of the core modules that I find is often overlooked. On a high traffic site, it's generally a good idea to reduce the db traffic as much as possible. Besides, since the database logging doesn't rotate/archive logs, it is only marginally helpful. So, while you need watchdog, it can quickly bite you in the... sorry, bad pun, but you get the idea. The good news is that we can still log all of these helpful messages in a flat file using Syslog and do some insanely cool reporting using Splunk.