Nov 17, 2018
Q&A with Steven Sinofsky at Twitter HQ
Developing cross-functional teams
Introduction
Steven Sinofsky is a Board Partner with Andreessen Horowitz, advisor to companies, and investor.
A respected technologist and business leader, Steven Sinofsky began his career at Microsoft in 1989. He started as a software design engineer in development tools. Working in product management, he grew to be one of the company’s senior execs on Microsoft Office overseeing six major releases of the full range of Office apps and servers. Most recently he served as President of the Windows division through 2012. In this role, he oversaw many of Microsoft’s largest products, including the development of Windows 7, Windows 8, and Surface, as well as Windows Services such as Outlook.com, SkyDrive and Identity.
He has co-authored a book, “One Strategy: Organization, Planning, and Decision Making” based on internal blog posts he wrote throughout the time he spent with the Windows team.
It is great to have Steven with us to speak with the Data Science team. At Twitter we have a centralized Data Science organization. A significant part of this organization is Product Data Science—a team of data scientists working closely and in sync with each cross-functional product team. They ensure data-aware decision-making through the entire product development cycle. The work entails overseeing metrics, opportunity sizing, experimentation, and machine learning efforts (model design, tuning, and validation) for each product.
I have prepared some questions for Steven. Most are inspired from thoughts he shared in his book, blog posts, and most certainly, in Tweets.
Strategy
Pardis. What caught my attention in your book is the term strategic integrity, referring to strategic alignment at a company. In cross-functional organizations, who is responsible to ensure strategic integrity? Who should be interpreting strategy into detailed actions?
Steven. I’m happy you picked up on that term. It’s actually pretty funny, when we were writing the book, that is what I wanted the title to be. The problem was that the book publisher couldn’t say that whole phrase without falling asleep—“strategic integrity” just sounds boring. But it was very important to me, so I’m glad that you picked up on it.
The gist of what I found worked very well for us for many years was that you try not to separate strategy and execution. You try to spend enough energy with the team so that there is an understanding of not just what we’re doing but why. And far more often that we like, the reason a strategy lacks integrity is that too quickly in the process of going from the retreat and the ideas to the “what are we doing”, the goals are lost.
Every strategy has two components to it that are super important. One of them is that you have to be able to see it. Not every feature of a release or a product is visible but everybody should be able to watch a visualization of the product and understand where their work will contribute to that visualization. And that becomes sort of—in a favorite Silicon Valley phrase—the north star. The first step to having a strategy is showing it and it’s not like a quick demo you show once but something you come back to and measure yourself by in terms of success. Which brings us to the second point. The easiest way for people to understand it is to write something like a press release. It follows a formula and you see them in PR Business Wire all the time. This way, you have a visualization of the strategy and then a press release which has to be just 500 words. Now everybody knows what’s supposed to be in the release.
It turns out if you had to work through those two tools, you had to think through a lot of stuff. And so when it comes time to figure out how everybody is going to execute, they can see what they have to get done and what has already been communicated to everybody. That’s the foundation of integrity. It’s the frame of the building. The execution is the details at all the levels of construction.
Deadlines
Pardis. Are deadlines good? And in a cross-functional organization, who sets them? Related to that, who defines the MVP that needs to ship? Every company will take on new bets that they would like to quickly release and assess.
Steven. That’s an interesting question and there is an easy answer, which is, that deadlines are what the team wants it to be. The team is going to say how long something is going to take and if the feature is cool, it doesn’t matter if they miss the deadline and negotiate a new deadline. This is very typical of how companies work when they’re early in life. Whether they are tiny seed-stage or even recently public companies. The difference is that in the march to product-market fit, you should basically just do whatever you want to do, because there is no one out there who cares what you do. You’re figuring out what they want.
Things are different when you develop a customer base. This is when you need to start to have grown-up conversations about 1) what are we doing for our existing customers, 2) what are we doing to acquire new customers, and 3) what are we doing for the people who are watching what we’re doing and evaluating if what we’re doing is good. This last group are people we often forget about. There are a lot of people who watch what you do, and whether they use your product or will doesn’t matter because they have a big voice and opinions. So as a company matures you need to start thinking about all of these groups and the impact that the work has on each. This is where you can get too risk-averse. Of course the flip side is that you can do what startups do and only increase your risk. The most famous example of this is “move fast and break things”. That was a really cool idea when there weren’t a billion people using a service and then all of a sudden the notion of “break things” really doesn’t sound very cool. It sounds neat to get features all the time but if 1 out of n break the service or 1 out of n you have to back out, it’s not really cool anymore.
Now a big part about coming up with a deadline is that it is as much about the start as it is about the end date. Because the start date for work should also be the start date for which the end date has integrity. Think about it this way: you can say six of us are going to build this thing and we’re starting tomorrow. But if you haven’t yet figured out 1) the impact of your work on each group enumerated above, 2) how it relates to all the other code and all the other features, and 3) how it affects all the roadmaps—you’re actually not ready to start. So it doesn’t help anybody to declare the start date. Because whatever end date you have isn’t going to work. You’re going to get a certain amount of the way through and you’re going to think wow this is more than we thought, we have to go work with this other team, etc. Therefore I would say that probably equally important to the end date is the start date. Everything before the visualization we talked about is not the start date. The start date is the day after you have that visualization. Because then you know what you’re going to build and the impact it is going to have. And your ability to predict the end date is now much more mechanical. You could mess up and be late but you’re not going to discover something in the middle that you hadn’t thought of before. Not enough companies think through the start date as much as they just start and then announce an end date. Part of what causes things to slip or look random is that they just weren’t ready to announce the start in the first place.
Status meetings
Pardis. In a cross-functional organization, how do you not get caught up in 4–5 status meetings a week? Sometimes preparing for status updates in a week takes away from the work itself.
Steven. The weird part about meetings of course is that—now I haven’t actually done a survey but—nobody likes them. The only person who likes any given meeting is the person who summoned everybody to a room to have the meeting. And then I will estimate, statistically significantly for a room of data scientists, that two-thirds of the time the person who summoned the meeting is still dissatisfied with the meeting to the point where, after doing that a number of times, will invariably decide to stop everything, and have a meeting to discuss how to have meetings. And that is almost always how meetings progress.
My first rule of meetings is to just not have them and then work backwards. And I don’t mean cliches like we have no meetings on Wednesdays, I mean don’t have meetings until you know what you’re going to accomplish. And the interesting thing about it is that what happens is invariably two things: 1) if you focus on having meetings that go back to the strategy and the plan it turns out most of the meetings you don’t need, because you know exactly what people are working on; and 2) if you have meetings that are tied to the rhythm of the work you’re doing, you have a far fewer number of meetings than the meetings where you think you need to know more. And the rhythm of the work is very different from the rhythm of the calendar. Things that you just say are weekly or daily or biweekly, they’re never connected to anything except the calendar. If you say you have a weekly meeting for engineering but engineering only tracks work every two weeks, then every other one of those meetings is annoying, because everyone is going to say that they’re getting the rest done next week.
Couple no meetings with going through a process to determine what we were going to build and when we’re going to start and when we’re going to finish. When we do that, the status meetings are held at the milestones where everyone provides an update. It turns out you can manage thousands of people in this way. Because everyone was focused on the overall goal, things didn’t change that much. So all other meetings are for people getting together to decide over the details of the work and who is going to work on it. So my answer to you is: don’t have status meetings. Because you have a visualization of what you’re going to do and you wrote down what you’re going to do and everybody signed up to do it. And the status meeting is redundant with all of that. There’s a leap of faith there and I get it.
Staffing
Pardis. How do we plan for staffing a cross-functional team? What are some best practices in engineer/PM/designer/data scientist ratios on teams?
Steven. This is a very controversial topic for people. So your on your own in how you interpret it. :) What I find in most companies is that they are way under-invested in product management and have the right number of engineers; and probably under-invested in the whole idea of quality and operations. I’m going to focus on the product part. The reason is that there is a feeling at early stage companies that product stuff that isn’t coding is overhead—it’s not capital-efficient to have product managers at an early-stage company, which is really true. But after you get to about 30 engineers, you can’t manage that efficiently without product management. The engineers are just unable to spend time connecting the dots across all their systems.
This is generally the path that people take: first they divide the product into a frontend and backend. Of course computer science people love that because you have an API, and you have a user interface that’s separate from the data, and it all sounds great—except in all systems for all of software history, the best, most innovative features come from breaking the abstraction layers between the frontend and the backend. Everything interesting that anybody ever does in software comes when the frontend and backend people get together to create new abstractions and find new ways of working together. The minute you separate them, especially without product management, then everything gets resolved by, “this is our API leave us alone”, or “this is the UI we want to do and we’re doing it.” And it happens all over the place. The way to get this resolved is by having product managers (program managers at MS) to break through that notion.
The numbers I offer people for how we worked effectively on products like Word and Excel generally blow people’s minds as either “very inefficient” or “what did they all do all day.” Take the team that built the UI for Office, that’s 10 different products, 5000 icons and commands, 3000 menu items, and 1500 dialog boxes. It had, at its peak, about 40 engineers and 25 program (product) managers and a dozen designers. Start Excel and look at the surface area that needs to be designed and compare it to most any other product you use — and that is just Excel (there’s also Word, PowerPoint, Outlook, SharePoint, and all the web versions of those). Yet oddly, most products, even much smaller products, have significantly more designers and design resources. There’s an efficiency gap in how the resources are applied and put to use.
What I discovered that I should add Design and Product into the ratio that isn’t coding because it turns out that a lot of Design in the valley is product management, except it’s not the most efficient product management. The same way that a lot of design done by product management is not the most efficient Design. There is a gap in skillset.
The thing about both Design and Data is that these are disciplines that generally work best when they work across the whole of what you ship. Especially if you take Data, data doesn’t care about frontend or backend or it doesn’t care about this part of Twitter or that part. They care about users and the holistic view of what’s going on. The way I think about the ratios as you asked, is what’s the right amount of research needed for a product.
We created a Research and Planning team for Office. At its peak Office had around 750 engineers and around 50 people in the Research and Planning organization. But it worked across all of the product. They were experts in all of the data that we had. You didn’t assign people to an account forever because when there’s no work there’s no work.
Pardis. This is interesting because over our lifespan, we have been going back and forth around the Data team organization, charter, and ratios. Should Data be a centralized organization? Should every team hire their own data scientist? Should every product team work with a data scientist.
Having a data scientist working with each product has been working well for us. Having said that we also see a lot of value in having that overall birds-eye-view of all the product, as you mentioned. This is because sub-products can have cannibalization effects over each other. We achieve this through collaboration with other data scientists in our centralized Data team.
Steven. You definitely need to think about it holistically as growth in one part of the product can come at the expense of another. I need to add that one of the reasons that I believe that design and research work better as a centralized organization was primarily for the career path and learning of those people. If you have all the designers in one place, they know one another’s language and know who talk to. Even if you embed them, keeping the manager structure as such that they work for other designers is great. A design manager knows where to balance workload and where to deal with the interns, and all of the things come from having a community. This applies to Data too.
However, no matter what you do, almost always there will be a group that complains and almost always the biggest of all the sub-products will threaten to build their own version of this team. It’s human nature. It’s always a big group and they always try this kind of org gymnastics and it’s really up to cross-functional leadership to attenuate that kind of reaction. Because ultimately, the company is better off by having a core group of amazing people in each discipline.
Internal blog
Pardis. At Microsoft, you shared your vision and ideas for the organization on the internal SharePoint site. You found it to be a powerful communication tool, and I quote, “to provide […] perspective and to discuss a broad set of issues in depth, from organization to motivation, and from culture to budgets, especially when used in conjunction with traditional tools such as 1:1s, team meetings, office hours, and so on.”
Do you recommend an internal blog for internal discussions and ideas? What can go wrong?
Steven. If I had to do it all over again, I would wish for a tool like Twitter internally. Writing a lot of blog posts is truly exhausting. If you added up all the blog posts I did it internally over the course of running Windows it would come to around 750,000 words. But there was no other way. You have a team scattered across the globe. There are 10,000 people and you can’t meet with all of them. You need a format that can be consumed later so you can the benefits of repetition. For example people want to know how fast you get promoted on the team. The old way of doing that is that you would connect them with someone in HR and People teams who would provide some generic guidelines. Alternatively as the person accountable for promotions, you write a blog post for how promotions work in your particular org, you’d say, “it’s the same system that the company uses but here is more detail.” When you write it down, it’s easy to share later. You avoid the risk of people not absorbing it or making up their own version of what you said verbally a year ago. And topics cover processes for managing the team, industry dynamics (how people are perceiving the products), and of course the product (how do we decide on things—I must have written 10 posts on the ratio question).
Twitter thread is of course my favorite way to do things now. When you start writing a blog post you get very fluffy and frothy in your language as if you were writing a speech. So this is how I do my Twitter threads: I have an Excel spreadsheet, and it has character counts in column B. What it’s forced me to do is to adhere to what I learned in my 8th grade on the 5-paragraph essay. It’s 1 through 25 rows with 280 limits on each, down the whole column. I write the whole thing on my iPad with a keyboard while waiting at the airport and hit Tweet when I’m in the air. I did get the snarky “you should start a blog” reply on my threads and I’m like, “you’re telling me this, I have like 10M blog words out there, and threading is now a feature, and they figured out how to render it well.” I am pretty convinced that I am the only person who does threads with bullets.
Innovation after product-market fit
Pardis. How do you ensure innovation? Who is in charge of innovation at a company?
No one is an expert in everything and a product is complex and made up of many moving parts. How do you ensure you can empower experts at every step?
Steven. Well you obviously get a Chief Innovation Officer and make sure they’re in charge of innovation. That’s a very common big company mistake, especially in consumer products. For some reason people think you can have innovation in a separate place from everybody doing everything else. At a big company anything innovative that you want to do is risky. At a big company, your notion of risk is skewed. If you were able to leave the company and evaluate the risk, you realize that the most risky thing that you can dream up at a company is at most a 2. And so you’re far from thinking about what you can do that’s a 10. Event though in your head you’re thinking ‘Omg, what if we did this, it’s going to be crazy!!”, except it’s not a 10, it’s that your notion of risk is all messed up. Why? Because of all the success around you.
The important thing to understand is that it’s going to be way way more uncomfortable than you really believe. If you’re comfortable and you think that your biggest risk is that they’re not going to finish on time, then it’s just not risky. The biggest lesson for me working at Microsoft was learning the ultimate example of product-market fit. Windows 95 was a ground-breaking product in 1995. Hundreds of millions of people got on the internet using America Online. It turns out it would be 12 years until Windows 7 shipped and everything in between was not the world’s best product. It was Windows XP which came 5 years after Windows 95. The problem was that as soon as it was released it was crushed by viruses and malware that the team had to quickly figure out how to fix. They were going to do a 9-month project that turned into 26 months and it wasn’t until Windows XP Service Pack 2 that things got fixed and that wasn’t until 2003. How good was Windows product-market fit? So good that it survived 12 years of not having a good product. And yet everybody was very busy the whole time. The lesson was this: if you have product-market fit, you can do a lot of bad stuff. The other best example of product-market fit that I have ever seen is Twitter. Which other product romanticized the fail whale and survived long enough to talk about it? And here we are today with 10X the number of users and for the most part you don’t ever see the fail whale anymore. That’s kind of amazing.
Pardis. So somehow related to that, did you ever have to stop listening to existing users for a chance to build for non-existing ones? Where does growth come from?
Steven. The truth is once you have product-market fit, it’s unwise to listen too closely to your users. Because your users are so vested in incrementally improving the product that you’re going to incrementally improve your product to death.
Look at the star to heart change, the world was going to end. It turns out it didn’t really matter because you had product-market fit. And what those users were asking for was completely disconnected from the growth challenges that Twitter was facing. And 280 was armageddon. Threads were the same way. Your best customers don’t want you to change the product. It’s the weirdest disconnect. Everything I ever worked on was like that. Steven King writing a book in Word has very different needs than everybody else doing a one-page meeting agenda. For Outlook, administrative assistants who schedule meetings and book conference rooms in 8 time zones have very different needs than the normal people only getting spam everyday.
The problem is that the more you listen to your best users the more you get pigeonholed into a product that doesn’t expand its use space. The easiest way to overcome this is to think of it as a beast you need to feed. Every product cycle, however long it is, you literally allocate resources to please those people: more keyboard shortcuts, more customization. Your best users are never going to take you to a risky bet that changes your product trajectory in a step-function kind of way.
