Keep your Joomla! Lean
One thing that people love about many open-source CMSs like Joomla!, Wordpress or Drupal is the huge amount of components which can be used to augment the functionality of your site. However, people forget that one crucial thing:
Once you have a 3rd party component installed, few will have people have the inclination, know-how, ability or time to update. More so, many people wouldn't even know that they should be upgrading hence the increased risk.
There are more reasons why you should keep your CMS be it Joomla or otherwise, lean
A website with few components does not require much mainteance. The only thing you'll need to keep updated is the core. And obviously your content. But that's our point, focus your effort on the content you need to provide to your audience, not on effort required to make sure your website is still up and runnin.
3rd party components are likely to create conflicts and clashes which might mean going back to the author's of the software to resolve - if a resolution is available. Keep Joomla lean ensures fewer clashes between 3rd party code.
Easy Updates and Upgrades
Updates and upgrades are always a headache - or at least a concern. Will everything keep running smoothly after I'm done from this upgrade or update? If you have fewer components to upgrade and update, your life will be so much easier.
Each component you install increases the execution time, load time and response time of your website. And you need to keep this as low as possible especially if your website gets a lot of traffic. Keep to the core Joomla and will be as fast as it can possibly be.
The good add-ons and components usually don't come cheap. If your budget is limited, then sticking to the core as much as possible will decrease your costs. All the maintenance required will also increase your costs since you will need to keep abreast of what needs to be updated and what doesn't.