Systems

If you ever have to work with systems from several different vendors, you quickly learn the pains of having to understand the nuances of the equipment and the interactions with their support organization.  It is mostly a frustrating experience as most of the vendors have little insight into the functionality of their equipment in a large scale deployment where upgrading/changing software or hardware components on a whim just isn't feasible.

I'm always amused with the request to perform a service impacting operation to collect information.  Another humorous request is for direct access to production equipment.  Of course this can be done with VPN access, but allowing an external support team into your production environment without knowledge of the overall network/system operation. We actually saw a vendor change a setting to our production environment in the middle of the day to collect some data.  Something that we would only touch in a maintenance window as it could have adverse effects.  The follow up meetings surrounding this instance were not fun.

So you learn over time who in the vendor's support organization is most familiar with your implementation and hopefully the bounds of access are strictly honored and that no changes are made without prior consent.