Extreme network slowness
After experiencing extreme network slowness, we tracked it down to packet loss going from the CS network to the VCC (but not the other direction). We will be replacing the transcievers on both ends of the link as soon as we can coordinate a time and hope that this will fix many of the networking issues we've been having.
What was planned to be a gbic swap turned out to be extensive troubleshooting and new fiber pairs going to the VCC. The new link seems to be much more reliable and we're hoping that it will eliminate many of the cronic network problems we've been having.
Around 4pm on July 14th an error was introduced into the firewall which interrupted networking temporarily, but the network seems to be stable.