Hofstadter's law tells us that it always takes us longer than we expect, even when we take it into account.
But why is that? Why are our estimates of time biased in one direction and not another? Because it is not just that people are randomly wrong about their estimates, they consistently underestimate how much time any decently large project will take to complete.
Here are some reasons I can think of:
§1. Project managers and marketers have a bias towards optimism, because they need to convince other people that a project is worth funding, which means they will tend to overpromise. Because funding is drawn from a finite pool, there is an incentive towards overpromising, because someone who is overly conservative will tend to lose out to those who overpromise. This is one of the many ways in which the short term has a way of dominating the long term, that the display of virtues is rewarded over actual virtues, because people make long-term decisions informed by what they see in front of them.
Reason number 1) is undeniable when it comes to people who are not working on a project and who need to sell others on it, but what about someone working on a solo project and who vastly underestimates the time it takes? It's unclear whether they get anything from overpromising, though you could say that they are overpromising to themselves in order to get the project started?
But here are some other reasons.
§2. It's well known that newbies have fantasies, this is simply called the Dunning Kruger effect, which is yet another phenomenon which is accurate but which doesn't have very satisfying explanations. We could say that newbies vastly overestimate their abilities simply because their discernment in the specific field they are working in is so poor that they believe that whatever limits they have represents the horizon of what's possible. Like a fish swimming in a tiny pond, thinking that the ocean is the same but slightly bigger, because it doesn't have the experience required to see its limitations.
The self, by itself, does not understand limits, much less the Reality which is beyond them, and so it thinks that whatever ideas it can think of are those of a real genius. Modern society makes this all the more pronounced because people at work tend to be surrounded by others similar to them, only differing in a few degrees of aptitude, but never being qualitatively different. Which is to say, people are surrounded by others who are slightly better or slightly worse at the task they are paid for, but will rarely meet someone who lives life in a fundamentally different way, because such a person would never cross path with them to begin with.
All of this discussion was to highlight that echo chambers are a major part of what upholds the Dunning Kruger effect, and that the mind, by itself, becomes an echo chamber, thinking that its horizons are the boundaries of Reality itself. An important aspect of learning, especially when one is a total newbie, is to deliberately seek knowledge and experience of others who have been there before us.
It's shocking how many people have fantasies of opening a restaurant for instance, but never do the bare minimum of research to see the kind of difficulty required to have a successful one. Some people might respond by saying that they know it is "difficult" to run a restaurant, but they rarely know what exactly is difficult. A restaurant can fail because of the quality of the food it serves of course, but more often than not, it seems to me that the disasters occur at the management and financial level. Vastly underestimating the money required to pay for the place, being unable to organize effectively the cooks and the waiters, being unable to attract new customers, all of those have to do with running the restaurant, not the food itself.
Thus what we can see from this example is that newbies are also totally unaware of the skillset required to accomplish an endeavor, because they lack the experience to see the dimensions alongside which one needs to be good to succeed.
Someone might be very intelligent, but can struggle to sustain their attention on the same project over a long period of time, and yet blame their lack of success on a lack of intelligence! The general problem is that very few people are able to recognize the precise reason why they fail at things, because most are lopsided and only view the world through their narrow set of aptitudes. This is why artists who fail to build an audience think that they need to improve their art, instead of recognizing the two parts of art-making and audience-building as largely separate, even if they influence one another, because obviously bad art attracts few people. And this is also why intelligent cowards think that they need to adopt more intelligent approaches to attract women, instead of learning to deal with their fear and put themselves out there, because their ego is aligned with maintaining their lopsidedness, not seeing the Truth.
To restate what has been said about the Dunning Kruger effect, we can say that poor discernment, which is the default state unless one consciously seeks knowledge and experience, is limited by its lack of horizons, and thinks of that as the entirety of Reality. Thus people underestimate how much time it takes to finish their projects because they didn't even know of all of the skills required to build it properly.
It doesn't help that mastery often looks very effortless on the outside, because it avoids all of the main traps and mistakes that people make. In reality, mastery is the result of a lengthy and arduoud process of refinements in the face of mistakes. It's only because they have made so many of them that they can smoothly sail around them, because knowledge of the territory doesn't happen at the safety of one's home, but in direct confrontation with the unknown and the difficulties it present to us.
§3. A project is refined through decisions, but many of those couldn't have been known beforehand, because those decisions respond to other decisions. Every finished project is the remnant left behind by several potential projects, because many choices had to be aborted, some of which couldn't be salvaged in any way. Thus the progress on a project is not linear, we can go down avenues which end up being unfruitful, but which could nonetheless be valuable as learning experiences.
The ability to scrap bad ideas while they're still young is a trait that experienced creatives have that lesser experienced ones don't. Newbies cling dearly to their ideas, because they have a lot of personal attachment to them. To see the idea scrapped is a bit like losing a part of them, which is why the sunk cost fallacy exists: it's a form of unprocessed grief, not for a person but for one's creative vision, which is why it cannot be reasoned out of them, because it is beyond mere ideas. More experienced people can see that an idea can sound cool without being all that great down the line, very likely because they themselves have seen it play in their own life.
§4. Another psychological effect that accounts for why we underestimate the time for any project is that our minds do not think of the emotional labor required to do difficult things. We look at the past as a series of events, the results of what we did, but not so much of the resistance that we experienced, the confusion that we felt, the fear that we felt. A great many deal of projects would be completed much faster if one never felt demotivated, especially when you take into account the momentum that you have to rebuild when you get stuck, but alas we are not machines that can tirelessly work on the same tasks again and again.
Being able to deal with the erratic nature of our motivation, and the occasional disruptions that invite themselves into our life is a necessary part of any long-term endeavor, but alas we have a tendency to focus too much on what we want to make, as opposed to the resistance that prevents us from making it happen.
§5. For small tasks, the ones that can be done in a single day, we can make precise estimates because the self which does the planning and the self which has to execute on it are essentially the same, even if just like the sky, our self is ever-changing.
At the timescale of weeks, months and years however, we are much more exposed to our changing self, which is why we often feel like we are burdened by ideas and projects which we no longer resonate with. This is why people with a wildly changing sense of self are so bad with long term commitments. Either their attention scatters in a hundred directions, never completing any of them, or they try to impose one fixed agenda over their fluctuating self, which leads to deadened passion and blocked inspiration.
This is why creativity is such a paradoxical affair: the ones with the most stable sense of self, whose element is earth, are the most likely to carry out an intention to its resolution, but they are also the ones with the most boring and trivial ideas to begin with. On the other hand, those whose minds constantly beams with exciting ideas rarely carry them out to their fullest, because they struggle to commit their energy to a single thing, their attention being like the wind, lightweight but constantly changing in direction.
All of the points I have mentioned, besides the first one which has more to do with marketing, align with the general idea that the idea that we have of Life, and Life itself, are entirely different. Our mind is prodigiously useful at simplifying Reality and presenting it in discrete bits we can analyze independently of one another, but this ability to dissect is also very misleading if we take it to be Reality itself. What we want, what we plan for and what we expect are all products of our mind, but Life is beyond all of those.
One view on this problem of planning is that it is unfortunate that our mind cannot grasp the difficulties which will await us, but in a sense it is useful because it tends to lean more towards action than towards planning. Isn't that what a good life is at the end of the day? Action that we find meaningful and that makes us whole, through the struggles of self-mastery and the humility which comes from hardship?
I think the real problem with Hofstadter's law is when the mistakes of one's mind are blamed on other people, when the marketer overpromises and finds a way to blame the engineering team. On the other hand, being overly optimistic about your own abilities and failing short of them doesn't strike me as very deeply problematic, it's just life and the way that lessons often come through pain and readjusting our course.
The beauty of a story comes from how we experience it through time, because it is what allows qualities to reveal themselves. Try to summarize a great story and you often end up with a disappointing nothing, a few gestures in the air trying to replicate the vividness that you felt while experiencing it.
Likewise we can bring descriptions about Life, but it is living itself which is most important. Experience is deeper than knowledge, which is why I think that the major concerns with Hofstadter's law is not how one might not commit that mistake ever again in their life, which is impossible, but rather how one might mitigate its most disastrous consequences, for instance going bankrupt because of a terrible financial decision, and how one might live a meaningful life even if one's plans do not pan out as they expect.
Because often, the most beautiful things in life come precisely from what we do not plan for. The people we meet and who influence us, the surprising beauty of the flowers we never paid attention to, that one casual conversation that gives us clarity about what we have been dealing with lately, and getting the sense that everything is okay in life just from looking at the clouds gently drifting in the sky. We can always get better at planning, and only an irresponsible fool does not take their future seriously, but ultimately the mind is in service of experience, the future is only meaningful because the present is, and as a result we should live accordingly, because it is not the end of the world to underestimate the difficulty of a project which is dear to us, but it is a grave mistake to not even try to pursue it in the first place.
Go back to the list of blog posts
Bias Optimism Time Creativity Projectmanagement
2026-03-19