Without proper caching, having a large number of poorly optimized plugins can really impact both the server, and the page generation times.