Apple's Open Source effort with the Darwin project has been a failure. Apple failed to build a community around Darwin for the seven years since its original release because it was not a business direction, but rather a marketing hub. Culturally, Apple did not understand and does not understand what it means to be an open source or to build a working community.
When Apple started sending Mac OS X Server, it included some GPL projects as well as many other open source projects. A few engineers working on these projects were able to convince some marketers and some managers that since they had to free some of the sources, why not be good citizens and make modifications to all the open source software? From there, there was a short marketing leap from unlocking the low-level components of Mac OS X as an open source, and one major recurring project to free up the entire base of Mac OS X as a stand-alone OS. During the dot com days, open source was good. Everyone wanted a piece of the open source pie, and this was Apple's way of getting it. This was great all the way around, the marketing department winning, the engineers winning and the open source world (appearing to be) winning. Just throwing a source out there and calling it good is not enough. Darwin 0.3 was just a source and did not attract much attention. Several Apple engineers worked, and spent much more time and effort as they needed to build a standalone operating system based on the sources. They came up with a build system that was compatible with Apple's internal build system, built up an addiction graph for the projects and eventually released an OS. This was something concrete that they could show. This is what Apple offers the open source community, a new, free operating system.
With something to show to their efforts, especially an Apple engineer went over to focus on and attract developers for their new operating system. This was great, they now had something to build a community around, they just needed to get the word out. The word definitely came out, and people started to come and have a look. Darwin was brand new, this was the first release, and it was not as polished as the established operating system and would certainly not be mistaken for a business product. It is not easy to take a collection of non-building codes and make an operating system out of it, but a small number of Apple engineers did so, often in their own time, and it showed. To install the operating system, you needed a Mac with OS 9 to dump a backup file on a partition. The source required click registration to obtain, and when you finally downloaded it, it was impossible to build. However, some people made it through all the obstacles, realized that it was a completely new project, and were thrilled with the ability to run a free UNIX-based OS on their Mac. Some were just happy to be part of a whole new project and get into the ground level. Everyone involved could feel the potential of the project. Apple was a big company with many engineers, and if they came behind an open source bet, this could be huge.
However, the potential was slow to realize. Darwin 1
However, it was a problem. Already in a year, Apple began alienating some of its users and contributors. Apple developed a reputation for not contributing, especially new features or additions. Small bug fixes that were easily reviewed can be accepted, but medium-sized features, such as SysV IPC support, did not gain traction. People were upset about spending all that time working on their project, which would ultimately benefit Apple, and having ignored all their efforts. None of these contributions received "sorry, we are not interested" or a confirmation from Apple engineers. These contributions were almost universally met with silence, as if they went to /dev/null.??1919900900. During this period, Apple ended up hiring over half of its most active contributors, accounting for about 3-4 people. This drained the contributing pool significantly and effectively mitigated those employed. When these contributors became employees, they signed NDA agreements, intellectual property agreements, and were otherwise controlled by much of the unlimited participation they liked before hiring. They discovered the cause of the lack of communication, such as being a secret culture. Even being perceived as someone who can leak information, even accidentally, was a responsibility, and it was best not just to communicate with those outside Apple, or even outside your own group. In addition, they discovered the reason why no one wanted to receive large patches: The one who sent them to the building was personally responsible for the function, and everyone had more than enough to work on the functions they were given by managers, let alone people in their leader hierarchy. Engineers were judged on how they met management goals, and management's goals did not include anything related to Darwin or an external community.
As time went on, Darwin did not improve. It had the same crappy shell script installer, poor terminal support and poor hardware support on systems not supported by Mac OS X. The biggest problems with Darwin did not seem to get any attention, the source started to disappear, first from leaf projects, then from increasingly important projects. The source that was there became increasingly difficult to build as "integration" took place, and previously independent projects began to have more and more dependence, many of which were closed-source components. At that time, virtually anyone interested in Darwin had enough to overcome these obstacles either from Apple or abandoned the community. The Darwin lists were almost completely populated by people who ran Mac OS X and had questions about UNIX issues.
For what it was worth, Apple recognized this problem and tried to act to fix it. Some people were hired specifically to create a better Darwin community. For over a year, this problem was discussed, solutions were proposed and rejected, and countless meetings were held. All the time, less source was available and what was there was more difficult to build. Finally, an independent organization was decided to promote Darwin and its development was in order. It was clear that Apple itself was unable to create and maintain the community it wanted. The end result was the OpenDarwin project, jointly created by Apple and ISC. Apple supplied the hardware, ISC provided connectivity and power. Much of OpenDarwin's original content came from a site called darwinfo.org, which I ran until I joined Apple and was asked to remove a lot of functionality.
OpenDarwin was starting a confusing start, but it was going to be the solution to all our problems. OpenDarwin started without any clear goal. Was OpenDarwin to pack Apple's standalone OS and expand it in ways Apple was not willing to spend resources on? Was OpenDarwin offering the necessary tools and environment to allow people to hack their Mac OS X systems? Or was OpenDarwin just there to provide hosting and resources for projects built on or around Darwin and Mac OS X? Over time, this became a process of elimination. Providing tools and an environment to allow people to hack the Mac OS X systems was largely unsustainable, as any system we invented would be blown away by the next software update or Dev Tools / XCode update. Packing Apple's standalone OS became increasingly difficult and time consuming. It did not help that most of OpenDarwin's first members were the former Darwin contributors who now worked for Apple at Darwin and did not have time to devote to such projects. Not to mention having just worked at Darwin all day, and wanted a distraction when you're out of the clock. Even just maintaining the infrastructure to host projects was burdensome. The OpenDarwin project was able to attract a marginal society, but few in the community had the time or the dedication to make any meaningful contribution. Those who did, ended up taking everything on and quickly being burned out. The OpenDarwin project continues to exist in a state of limbo.
With the creation of the OpenDarwin Project, Apple was able to push open source to the back burner. They were able to check open source agenda with hardware donations instead of culturally changing and real interaction. Source was squeezed out when binary forms for Mac OS X were made available to the public, the need to obtain buildable source was reduced, since it was OpenDarwin's problem, and engineers were generally happy to have a minor thing to be harassed about.  Just One Problem: Over the years, the original goal of creating a standalone stand-alone operating system and a community around it had failed. There was no community, and the standalone freestanding operating system had been composed of binary packages and the source no one could build. Even worse, instead of acknowledging the problem, Apple and most of the world simply forgot about Darwin.
So, how did it fail? What was the problem? The problem was that Darwin was a marketing plow and nothing more. Marketing meant that by throwing some source files over the wall, they could increase their market share and revenue. And they were right. You go to almost any technical conference these days, and it's an amazing number of Mac laptops in the crowd. Before Mac OS X, people would be laughed at by the building to have a Mac, but now, with Mac OS X and the open source magic, it was suddenly OK or even cool. Manually and culturally, Apple had no intentions to get involved in any society outside its own walls. There were some well-intentioned engineers and even a few low-level managers who were fooled by their own marketing department, and that was all. These people had given significant amounts of their spare time under the illusion that they were doing something right, something good, but eventually they were simply used by their own company's marketing department.
23. February 2006