A while back, Jeff Atwood spoke about the The Power of Defaults, arguing a good point that when creating a piece of software, the default options you choose as the designer have the power to make or break your product. This is true, however he took the argument further, revisiting an older post titled The Problem With Configurability, effectively arguing that you should design your software right the first time with a strong set of defaults, and do not allow over-configuration.
In other words, don't make users think. Or at least don't make them think about anything outside the narrow focus of their immediate goal. Furthermore, making everything configurable really means the designer isn't doing his job.
This is where our opinions diverge.
At a previous job of mine, I was tasked with (re)designing and building the entire assessment module of an LMS. Everything from the interface for teachers to create assessment pieces, to the interface for students to complete it, to the reports, to selective release locks (eg: only release assessment B after assessment A has a score of 20 or more) and even versioning for entire exam papers. Prior to this, the system already had an assessment module, however it was not powerful enough for what our users wanted.
In the old system, because the user couldn't customise things at all, any divergence from the default that they wanted meant a lot of hacky code was needed just to achieve relatively simple requests, of which there were many. The new system gave that power to the user, but no one wanted to find out how to achieve what they wanted and instead threw up their hands and kept requesting changes to the system which they themselves could make. They wanted the default to satisfy them, however (and here is my point finally), a default, no matter how well chosen, can't be all things to everyone. It will be juuust slightly wrong for some people, an annoyance to others, and the antithesis of what others want. Unless you are building for just one person, the default is just a starting point.
After each support call, I'd be cursing the "stupid user" who didn't know to look in whatever menu to solve their problem, and I'd suspect the user would be there cursing the stubborn developer who made it too difficult for them to achieve their task. The answer, as with most things in life, ends up somewhere in the middle of the two. Depending on what sort of users you've got, you'll need to find a balance between "massively powerful, but incredibly difficult to use" and "very simple, but underpowered". An example of the former is a CMS called TYPO3, who, upon their beginners' page, go to great pains to stress that their software is hard to use and is incredibly complex. On the other end of the scale is pretty much any piece of software that Apple has made.
If you are trying to get a very wide userbase including people with very low levels of skill, motivation or patience, invest more time into your defaults. If your audience is someone who has (or should have) the time and motivation to get that much more from your software, spend your time making it flexible to their needs. I say "should have", because in certain circumstances, users have unfair expectations of software. If the software has been built to do every single little task which they've asked, and some things they will probably ask for, they have to expect that the learning curve is going to get a little steeper, and it's up to them to put in a bit more effort. It is a shared responsibility.
Personally, I get some weird sadistic satisfaction from being able to control every single part of how a program works. UEStudio, my text-editor of choice, has no less than 54 tab pages in its settings, and after gradually tweaking it over time, it is perfect for me. Occasionally, I've taken a look other editors: ones with features that would probably assist me greatly, however they haven't allowed the level of tweaking that I require, and for me at least, that is a show-stopper.