Ken Kocienda gives an inside look into Apple’s software development process during his 15 years at Apple. He was there during the release of Safari, iPhone, and iPad.
The book is a must read for anyone who develops products or is interested in Apple. (The book is also great vindication for those who believe that A/B split testing doesn’t play a role in a lot of excellent and innovative design, and is a way overused tool today).
This post will
Start with the key concepts of Creative Selection and the importance of demos
Look at taste, judgment, and design
End with summary of development of Safari, the iPhone Keyboard, and general comments on working at Apple.
Listen to this post on YouTube below, or on your podcast app under: Gregory Schmidt
Creative Selection
The title ‘Creative Selection’ does a great job summarizing the book. It is a process of continuous incremental Darwinian evolution that moves an idea towards a final product.
Creative selection is a process of
creating a “demo”
receiving specific feedback
producing another demo, which will then receive further feedback
Each round of demos & feedback selects for the best features at that time. Each demo also has the potential to introduce new variation and a “fresh divergence” with the decision to pursue a new approach at the next demo.
This is how Apple built software. One demo, after another. Improving each time.
Ken highlights what they did not do. They did not have long coffee breaks or jam sessions debating their ideas abstractly. They did not have long white boarding ideas sessions. “They did not shuffle papers around waiting for an epiphany and … inspiration that would get them to a final product”
In short, they did not try to “flip Thomas Edison’s ratio of inspiration to perspiration”.
"Genius is one percent inspiration and ninety-nine percent perspiration.”
- Thomas Edison
(In the book Ken actually calculates this ratio of inspiration to perspiration for the Safari web browser project. The ratio holds up, and if anything, underestimates the required perspiration to get the job done. In Edison’s case, his insight to use bamboo as filament in the lightbulb was the easy part. He then had to test over 1200 types of bamboo to find the optimal one.)
Demos
The major ‘takeaway’ from Ken’s book, is the importance of demos in his work at Apple.
Without a demo it is very difficult to discuss and compare two ideas or receive feedback. Ken provides the example of asking two people to ‘think of a cute dog’. This is a pretty straightforward and simple concept to think of. However, without a concrete demo it is very difficult to compare two people’s ideas on what a cute dog is.
But, once each person produces a picture of a cute dog, a proper discussion can take place. Clear commentary and feedback can be given. And now, we can have progress.
It is tough to discuss who has an idea of a cuter dog without a demo. Now imaging trying to discuss and make progress on more complex problems. But with demos this is possible, and multiple designs and their merits can be debated, discussed, and concrete decisions made to choose the best ideas at the time to move forward.
The continued production of demos keeps people focused and prevents abstract distractions from getting in the way.
No single step in this process results in the final product. But the incremental improvement upon the previous demos is the trick.
Five purposes of demos are:
Highlight potential
Explore concepts
Show progress
Prompt discussion
Drive the discussion for making software
Other points on demos at Apple
Good demos moved upward to demos for higher levels of management at Apple. Eventually demos would get to Scott, who made the final decision on if the software was ready to demo to Steve.
Everyone present at the demo could contribute their feedback.
Demos at lower levels may be for feedback from peers, but generally demos are held with a clear ‘sense of command’. Each demo had a ‘decider’, who made the final call on the next steps. The outcome may be decide (‘yes, it is good’), change (‘make these changes and come back’), or reject/dislike (‘none of these are good enough, come back with something new next time’)
Demos allow the creator to gauge people’s genuine reaction to using the product.
Demos are build like Hollywood backlots. Some parts of them are facades and don’t actually work. Other parts of the demo are interacted with, and require reality. But in the end, the person interacting with the demo, doesn’t know what is a thin backlot and is able to suspend disbelief to believe the demo is real.
Use the demo to “start approximating your end goal as quick as possible”
Demos “advance the story”
“Demos make us react”
“Demos are the catalyst for creative decisions”
“Most demos - almost all of them - fail”. Get over the apprehension of failure very early. Because that is the norm for demos.
More demos = more feedback
Demos ‘makes discussion of things that are complicated easy. They make it easier to make decisions’
- Ken Kocienda
Demoing for Steve Jobs
Steve reviewed demos in a boardroom called ‘Diplomacy’. It had a sign above the entrance ‘Know Thyself’.
Ken describes demoing for Steve like ‘going to see the Oracle’ - who will speak and decide. He made the final decision on what Apple would commit to.
Steve was known to change his mind over time, and contract previous opinions of his. Much has been written on Steve Jobs, I liked Kocienda’s take that although his mood and response were generally unpredictable, his passion was predictable - “he wanted it to be great”. He “respected Steve’s taste, but not his temperament”.
From Ken’s description, Steve Jobs would study what was placed in front of him to review with great intensity. Carefully surveying it from all angles. Then he may ask a question - asking someone else their opinion on it, and wait to see their answer. “These were tests”. If you passed them, he knew he could trust you to help make software better. One earned the ability to work closer to Steve by continuously providing good feedback.
Steve would make the final call and decide to commit, give specific changes, or ask to see something entirely new next time.
Tools for Selection
Kocienda highlights seven factors that were a summary of their creative process at Apple. These were never officially identified, but the list was created in retrospect. These seven elements part of the design process
1. Inspiration - from great past work
2. Collaboration - working with others
3. Craft - skill to achieve high quality results
4. Diligence - the hard work to get things done, don’t take shortcuts
5. Decisiveness - don’t delay or procrastinate, be decisive in favour of simplicity
6. Taste - a refined sense of judgment
7. Empathy - see world from other people’s perspective
Simplicity: what to select for?
Steve always chose a design ethos in favour of simplicity.
A decision that added complexity back into the design was generally a bad idea. Complexity often resulted in more complexity for the user but then also for the developer and designer. It made everyone’s life more difficult.
‘Software should speak for itself. Nobody will be there the first time a new user tries it to teach them.’
Design is how it works
Most people make the mistake of thinking design is what it looks like. People think it’s this veneer – that the designers are handed this box and told, “Make it look good!”
That’s not what we think design is. It’s not just what it looks like and feels like.
Design is how it works.
- Steve Jobs
Objects should explain themselves….
The appearance of product should tell you what it is and how to use it…
Shallow beauty in products does not serve people…
- Ken Kocienda
Taste & Judgment
The principle that design is how it works gives “taste a purpose beyond self indulgence”. It “removes the arbitrariness of taste”.
Taste is a critical part of design at Apple.
“Taste is developing a refined sense of judgment and finding the balance that produces a pleasing and integrated whole”
- Ken Kocienda
He describes the process pretty much identical to a process I have stumbled upon myself over the years.
Judgment needs to become a “a knee jerk reaction”. This refined like-response results in “the ability for form opinions with your gut, that you can then justify with your head”.
One needs to build “trust in a refined personal like-response”. This takes time and requires study of past works in order to assemble a catalogue of things one likes, in order to act as a measuring stick to compare new ideas against.
Why is the development of taste & judgment critical? Because you “have to keep making progress. You can’t open oneself to infinite possibilities at each step”.
Balance
Ultimately each small judgement about taste in the pursuit of design (how something works) must come together in an integrated whole. When this is done poorly, the product becomes off balance.
This ability to integrate multiple judgments together is based on a balance of taste and empathy for the user. Understanding their needs an leaning in favour of simplicity over complexity.
A/B Testing vs Heuristics
Many of the decisions that go into a product are ultimately a series of heuristics - subjective decisions based on taste. These heuristics are integrated with some algorithms.
How to centre the photo on the page, is based on an algorithm. How long an animation should be or how large an icon should be is based on a heuristic. I agree that too many designers these days try to turn heuristics into algorithms - numerical outcomes that can be ‘tested for’.
Sometimes it is quick to decide on the right heuristic. The process of demos provided the opportunity to “put their faith in a sense of taste” while making small decisions each time.
Other times deciding the right heuristic took time. For instance, the iPhone keyboard used an algorithm to auto-correct words. But precisely when the algorithm should kick-in was based on a heuristic. This threshold was based on people’s feedback from those who were using the software during its development. (note how Apple did not decide this heuristic based on a single metric arrived at during an ‘a/b split test’ but instead from people’s reaction to the product)
This is very different than how other companies and designers operate who are obssed with split testing.
“At Apple, we never would have dreamed of doing that, and we never staged any A/ B tests for any of the software on the iPhone. When it came to choosing a color, we picked one. We used our good taste—and our knowledge of how to make software accessible to people with visual difficulties related to color perception—and we moved on.”
- Ken Kocienda
The designer Douglass Bowman recalls why he left Google over this:
When I joined Google as its first visual designer, the company was already seven years old. Seven years is a long time to run a company without a classically trained designer. Google had plenty of designers on staff then, but most of them had backgrounds in CS or HCI. And none of them were in high-up, respected leadership positions. Without a person at (or near) the helm who thoroughly understands the principles and elements of Design, a company eventually runs out of reasons for design decisions. With every new design decision, critics cry foul. Without conviction, doubt creeps in. Instincts fail. “Is this the right move?” When a company is filled with engineers, it turns to engineering to solve problems. Reduce each decision to a simple logic problem. Remove all subjectivity and just look at the data. Data in your favor? Ok, launch it. Data shows negative effects? Back to the drawing board. And that data eventually becomes a crutch for every decision, paralyzing the company and preventing it from making any daring design decisions.
Yes, it’s true that a team at Google couldn’t decide between two blues, so they’re testing 41 shades between each blue to see which one performs better. I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I can’t operate in an environment like that. I’ve grown tired of debating such minuscule design decisions. There are more exciting design problems in this world to tackle...
…But I won’t miss a design philosophy that lives or dies strictly by the sword of data
As you can see, the inability to have a developed and refined sense of judgment in pursuit of taste and balance prevents one from efficiently designing and developing products.
It also prevents one from understanding how the design comes together as a whole - which is the most important part and part which lends itself least to split testing. Split testing can’t test multiple integrated wholes.
The process of designing based on popularity, doesn’t result in excellent design.
I fully agree with this sentiment. In this age it sometimes feels taboo to even suggest that a designer can make a decision without split testing ideas, (sometimes these sensible people are referred to as “UX cowboys”), I am reassured that during Ken’s time at Apple’s approach was exactly that. And they developed magnificent products.
Instead, Apple chose to focus on frequent demos. With each demo being an opportunity for those present to use their sense of judgment and taste to make suggestions, and move forward incrementally.
In the end, Apple worked to select for the “the intersection of technology and the liberal arts”. Just as a poet, painter, and author doesn’t disassemble their work to split test it, Apple focusing on the integrated whole of their product.
“We’ve always tried to be at the intersection of technology and liberal arts, to be able to get the best of both, to make extremely advanced products from a technology point of view, but also have them be intuitive, easy to use, fun to use, so that they really fit the users – the users don’t have to come to them, they come to the user.”
- Steve jobs
Safari
Ken was part of the original three man group in charge of setting up a web browser on the Mac. This was to replace Internet Explorer that had been on the Mac since 1997.
It is a great story of how they tried to use open source software - first Mozzila with 1.5 million lines of code (50,000 pages when printed at 30 lines a page). This was impossible to work with. Richard Stallman achieved success using Konqueror - which had only 120,000 lines of code. One-tenth of Mozilla.
Part of the reason Mozzila had ballooned to be so large was all the boilerplate code in Mozzila that was required in order for them to try and design software with individual components that could be easily connected to each other like lego blocks.
Super Human Progress: Richard got a demo of the Konqueror browser up and running within two days. Whereas Ken and Don Milten had been working on the problem for over 6 weeks using Mozzila. Ken uses this example to illustrate how unlike the physical world (such as Olympics) where physical limits are placed on human performance, there are certain times in software development when one individual can have improvements 10x or 30x over that of other experts in their field. These cases it seems often are early in a project, when there are not limitations in place. And a critical insight into an early stage concept or organizing principle can provide a tremendous improvement over the work of others.
Page Load Time: In order to focus efforts on Safari’s development- browser speed, was identified as the single most important feature. Therefore a Page Load Time tool was created to track this metric. Every single change to the code was measured against the Page Load Time. Only those changes that improved it would be accepted. In cases where new code was essential to a new feature, in order for it to be accepted, other code had to be optimized so that in the end there was a net neutral effect on speed performance. The selection of this single real world metric added great clarity to software optimization. Which is often a process that starts too early in software development and without a clear focus on improving ‘real world’ performance.
Insertion Point Problem: One of the interesting facts, is that the hardest technical problem he ever had at Apple was getting the “insertion point” at the end of a line in an HTML email text editor to work properly. It seems like something that is incredibly simple, yet was surprisingly complicated.
Alternative names for Safari browser were Alexander, Freedom, and Thunder. From the way Ken described the process, it seemed names were proposed by people working at Apple, rather then an external PR firm. (Perhaps a separate firm was involved, but Ken didn’t mention this.) Don Melton recalls the naming story on his blog.
The iPhone Keyboard
The development of the iPhone keyboard is remarkably long and complicated story. Ken goes into great detail, and reading the original book is highly recommended.
The process started when it was discovered that the demo’d version of the iPhone keyboard wasn’t very good and lack of a good keyboard had product killing potential.
This led to the Keyboard Derby where staff competed to build keyboards. Ken eventually won the design and spent the next two years perfecting the idea. Multiple demos with different number of letters per key. Different placement of buttons on the screen. Addition of algorithms to convert the garbage someone types in and convert it to the letters they meant to type. Different letter weighting score patterns. Pattern screw algorithms. Addition of touch feedback in having the keys enlarge after each press. Read the book - its worth it.
Some other random iPhone facts
QWERTY was selected as the preferred keyboard arrangement because it reflected the best combination of taste + empathy + craft. Other keyboard designs evoked too much reaction when people saw them because they were different, whereas showing a keyboard that people were used to evoked the most normal reaction and allowed people to use it best from the start.
Although the original iPhone only made use of two-point multi-touch gestures, it was capable of handling 11 simultaneous gesture points at once. Though, clearly this was never used as it would require all ten fingers plus your nose.
The size of the App icons on the home-screen was determined by a game. Early developers in the office installed a game that would render squares of different sizes on different parts of the screen. The user would click the square and a new one would appear. This was both the iPhone’s first game, and also provided a ton of information on the optimal App icon size based on where it was positioned on the screen. 57px was the final optimal number.
Autocorrect will never correct a word to mean a slur or a defamation
When people touch the screen, they actually do not touch with the part of their fingertip they think is touching the screen, but a part lower down on their finger. The software auto-corrects for this.
The iPhone keyboard typing sounds are based on the ‘clack’ of a pencil on a desk. Ken was inspired by the idea that when Ben Burke was designing the blaster shot sounds in Star Wars he produced these sounds from a hammer striking a guide-wire for an antenna.
The original iPhone patent is a great 300+ page read - US7479949B2 Touch screen device, method, and graphical user interface for determining commands by applying heuristics
Some figures from the book, Creative Selection:
Working At Apple
“Every day at Apple was like going to school, a design-focused, high-tech, product-creation university, an immersion program where the next exam was always around the corner.”
Because of the high level of secrecy on everything, this often mean trying to recruit people without being able to tell them what they will be doing.
When Steve returned to Apple he removed the separation between research and development and software implementation by disbanding the “‘the advanced technology group”. The research required would be done by the same people who also had to implement the software.
DRI - Directly Responsible Individual - person who would sign up and was ultimately responsible for a product or software.
They worked in small groups, and this encouraged regular interaction and efficient communication. There were only 7-9 people on the Safari Browser project, and 25 people listed as inventors on the iPhone.
Not everything was fancy and new. The Diplomacy room that Steve reviewed software demos in was remarkably unimpressive and in fact drab and old looking. Some mockup designers even in 2008 preferred to use Adobe Director - a software tool from the 1990s - because of how quick they were with the software.
No detached high level managers making key decisions. When high level managers did make decisions, they were involved in the project.
Steve Jobs would start preparing for keynotes about 3-4 weeks prior to the event, continuously practicing on stage a hall and getting feedback.
Random notes on software development
‘Make progress by skipping past problems you can’t solve and move into the ones you can’
The traditional term for using your own products in development is “dogfooding”. Ken prefers the term, “living-on”. Either way, it is an essential part of knowing if what you are building is any good.
Convergence is the period of getting ready to launch a product and fix all the bugs. Their Radar software tracks the bugs during the time, and the goal is for the line to move downward.
Getting to ‘zero bugs’ during software’s convergence period is never possible. Getting to a reasonable level to ship is the goal. To emphasize this, Netscape aimed for “zero boobs” - a testament to the fact there will always be some bugs when shipping
Heisenbug - is when software’s behaviour is mysterious yet the bug causing it cannot be found.
A Seagull Manager is a “top executive who is rarely around. Who flies in occasionally and unexpectedly from who knows where. Lands on your beach. Squawks noisily. Flaps its wings all over the place. Launches itself back into the air. Circles overhead. Drops a big poop on everyone. And then flies away. Leaving the rest of the team to clean up the mess, figure out what it all meant, and wonder what to do about the inevitable follow up visit.”
K48 - codename for iPad
Project Purple - codename for iPhone
Some people mentioned
Scott Forstall - able to barrage people with series of rapid fire questions
Greg Christie - human interface team, who ultimately suggested one letter per key on the iPhone
Bas Ording - designer, invented inertial scrolling on iPhone
Don Milten - worked on original Safari browser. When at Netscape he helped them to remove all the profanity from the codebase before Netscape when public.
Richard Williamson - had a software company in teens, made bold claims of how easy things were to do, and backed them up by delivering on them in record time
Imran Chaudhri - led the Human Interface Team that developed the innovative iPhone interactions
Interested In More?
Watch 6 hours of interviews of interview with Kenneth Kocienda and Richard Williamson on their careers at Apple, recorded by the Computer History Museum in 2017. Part 1. Part 2.
Read the original book. It has 4.5/5 stars on Amazon.com
I listened to this book on Audible and took notes while listening at the gym. I have tried to be reasonably accurate in the quotations, and place in quotes around phrases that I thought are near direct quotations from the text. I, and auto-correct, likely have made errors in this process. Please notify me of any errors or unattributed quotations so I can fix them.
Early iPhone prototypes (Walabee) that required connection to the computer to run. Photo from Ken Kocienda, Tweeted June 29. 2017
