With Windows 7 beta testers expressing frustration about not getting enough information on the upcoming operating system, Microsoft’s Windows 7 engineering chief is trying to address some of the concerns.
In an impassioned Feb. 25 blog post, Steven Sinofsky, senior vice president for the Windows and Windows Live Engineering Group at Microsoft, responded to Windows 7 beta tester concerns that they have not received adequate feedback on issues being reported to Microsoft by the testers.
“The Windows 7 beta represents a new level of feedback in this regard in terms of the overall volume,” Sinofsky explained. Earlier in the post, he said:
“What we can say for certain, is that we are listening to each and every comment, blog post, news story, MS Connect report, Send Feedback item, and of course all the data and telemetry.“
In the process of making his point, Sinofsky pointed to the fact that Microsoft has released a reliability update for Internet Explorer 8 running on Windows 7.
He also said:
“As we’re developing the features for Windows 7 we work closely with PC makers, enterprise customers, and all types of customers across small business, education, enthusiasts, product reviewers and industry “thought leaders,” and so on. We shape the overall “blueprint” of the release based on this wide variety of input. As we have design prototypes or code running, we have much more targeted and specific feedback by using tools such as usability tests, concept tests, benchmark studies, and other techniques to validate the implementation of this blueprint. Our goal with this level of feedback is for it to be representative of the broad set of Windows customers, even if we don’t have a 1:1 interaction with each and every customer.“
Moreover, Sinofsky provided specific details on some of what Microsoft has done to address beta user concerns:
“– During a peak week in January we were receiving one Send Feedback report every 15 seconds for an entire week, and to date we’ve received well over 500,000 of these reports. That averages to over 500 reports for each and every developer to look through! And we’re only through 6 weeks of using the Windows 7 beta, even though for many Windows 7 already seems like an old friend.- To date, with the wide usage of the Windows 7 Beta we have received … hundreds of Connect (the MSDN/Technet enrolled beta customers) bug reports and have fixes in the pipeline for the highest percentage of those reported bugs than in any previous Windows development cycle.- To date, we have fixes in the pipeline for nearly 2,000 bugs in Windows code (not in third party drivers or applications) that caused crashes or hangs. While many Beta customers have said they are very happy with the quality of Windows 7, we are working to make it even better by making sure we are fixing the issues experienced by such broad and significant usage.- To date, we have recorded over 10,000,000 device installations and over 75 percent of these were able to use drivers provided in box (that is no download necessary). The remaining devices were almost all served by downloading drivers from Windows Update and by direct links to the manufacturer’s web site. We’ve recorded the usage of over 2.8M unique plug-and-play device identifiers.- On a personal note, I’ve received and answered almost 2,000 email messages from folks all around the world, just since this blog started in August. I really appreciate the discussion we’re having and am doing my best to keep up with all the mail.“
Sinofsky also defined how Microsoft views the term “bug”:
“Let’s talk a bit about “bugs”. Up front it is worth making sure we’re on the same page when we use the much overloaded term bug. For us a bug is any time the software does something that someone one wasn’t expecting it to do. A bug can be a cosmetic issue, a consistency issue, a crash, a hang, a failure to succeed, a confusing user experience, a compatibility issue, a missing feature, or any one of dozens of different ways that the software can behave in a way that isn’t expected. A bug for us is not an emotional term, but just shorthand for an entry in our database representing feedback on the product. Bugs can be reported by a human or by the various forms of telemetry built into Windows 7. This broad definition allows us to track and catalog everything experienced in the product and do so in a uniform manner.“
Testers Respond
Overall, Sinofsky provided a detailed account of how Microsoft incorporates feedback from users and how the company makes decisions as to what to address and in what order. For instance, he said:
“The challenge of how to incorporate all the feedback at this stage in the cycle is significant. It is emotional for us at Microsoft and the source of both considerable pride and also some consternation. We often say “no matter what happens, someone always said it would.” By that we mean, on any given issue you can be assured that all sides will be represented by passionate and informed views of how to resolve it, often in direct opposition to each other plus every view in the middle. That means for the vast majority of issues there is no right or wrong in an absolute sense, only a good decision within the context of a given situation. We see this quite a bit in the debates about how features should work-multiple solutions proposed and debate takes place in comments on a blog (people even do whole blogs about how things should work). But ultimately on the Windows development team we have to make a call as we’re seeing a lot of people are looking forward to us finishing Windows 7, which means we need to stop changing the product and ship it. We might not always make the right call and we’ll admit if we don’t make the right call, even if we find changing the behavior is not possible.“
One of the testers expressing frustration of the type Sinofsky was responding to is Chris Holmes, who authors the GeekSmack blog. In a recent post, Holmes wrote, “I know I drifted off into a bit of a rant … and hopefully this feedback is taken seriously by the Windows team (hey, there’s a chance, it’s not like we’re dealing with the Windows Live team, those people don’t know the meaning of the word feedback).”
Yet, in another post, Holmes noted that Sinofsky personally responded to one of his queries. Holmes wrote, “I honestly was not expecting a reply as I understand that Steven is undoubtedly a very busy man. But when I checked my in-box a few hours later, not only did he respond, but he did a very good job explaining his position and why the interim builds were not a good option for this release cycle.”
And a commenter to Sinofsky’s own post said:
“The frustrating thing about all my feedback on Microsoft Connect is that most of the time I got one reply: “You should create a DCR. This is not a bug.” My response: “How can I create a DCR?” And I got no answer[…]And there are sites like the Windows 7 Taskforce. Where users discuss and most of the time come to a consensus. There is 1 issue that “will be fixed” and 4 things that are fixed. One feature request was marked as “fixed” when in fact it’s not. Out of over 500 entries. Most of these things wouldn’t be hard to implement: Adding 1px borders around elements, changing bitmaps, changing colors and so on have NO impact on stability and don’t have to be localized. I don’t get why MSFT doesn’t change those things. If only to make Windows look more polished.“
Meanwhile, Sinofsky summarized his argument: “The work of acting on feedback responsibly and managing the development of Windows through all phases of the process is something we are very sincere about. Internally, we’ve talked a lot about being a learning organization and how we’re always learning how to do a better job, improve the work we do, and in the process work to make Windows even better.”
