The development of Windows 8 seemed shrouded in a fog of secrecy and arrogance for many long time Windows users. Head of Windows at the time, Steven Sinofsky, had a vision for the future of Windows. Former Microsoft CEO Steve Ballmer allowed Sinofsky and his team carte blanche authority over the company’s future. Windows 8, unfortunately, did not meet Microsoft’s lofty expectations and during the process, alienated old, new and potential Windows users. Microsoft’s Apple-esque approach of dictating feature sets and development landed during an odd period in personal computing and substantially cost the company the favor of OEM’s, enterprise customers, and previous Windows users.
Suffering from the black eye that was Windows 8, Microsoft pivoted and decided to go back to a formula that saw Windows adoption reach levels of 400+million copies at one time. The new-old formula is about feedback and user participation. Microsoft has opened its software development on its various platforms to ‘Insider Previews’, “Beta Programs’ and feedback driven initiatives. As much as Ballmer was known for his “Developers! Developers! Developers!” chant, Microsoft is quickly becoming synonymous with “Feedback! Feedback! Feedback!”
As for the development of Windows, Samer Sawaya, a Senior Program Manager Lead on Windows has posted an informative and lengthy blog about how User Feedback helped shape the next version of Windows. Samer is part of the Data and Fundamentals Team headed by legendary Twitter informant, Gabe Aul. While most questions are typically directed at Gabe during the days of build releases of Windows 10, it’s Samer and a team of dedicated developers who sift through questions and put answers into motion.
“The goal of our team is to connect teams across Windows directly to the customer feedback you provide, and to make it very simple to leverage your feedback within the code we’re writing—to make it better. Therefore, we track how much feedback we receive, how much we convert into bug reports and work items, and how many of the work items we complete. We monitor these key metrics continuously to ensure we’re acting on your feedback,” according to Samer.
Some of the key takeaways from the in-depth piece are:
Engineers use your feedback
With over 3 million pieces of Windows Insider feedback passing through the Windows Feedback app since October, the Windows team has devised a way to address most of the feedback. Engineers utilize a familiar interface to access User Initiated Feedback (UIF). The new interface is based on Visual Studio Online and gives engineers the opportunity to access the complete set of raw feedback given by Insiders. Equipped with parameters and filters on specific values, engineers can quickly sift through millions of pieces of feedback gathered. Engineers are also alerted via automated emails about highlighted feedback issues specific to their areas that they have not addressed.
Why the team uses an upvoting system
According to Samer, “We use an algorithm to turn feedback with a lot of upvotes automatically into product work items and determine to which team the feedback belongs. The work item lands on the appropriate team’s list of work to do, and they look very closely at the feedback and evaluate how to address or respond to it.” However, the sheer number isn’t the deciding factor always. Many times, new, emerging issues or issues regarding a particular experience related to hardware take precedent over the avalanche of upvoted issues. Another use case for upvotes is to help engineers find and validate harder to replicate issues.
The team uses a cool typography graphs to spot themes
Internally, the team uses a web reporting tool called Gestalt to explore feedback via text analysis. Gestalt can cluster phrases that appear on particular flights together to form a visual cloud representation of broad themes. Engineers can then filter the word cloud in a variety of ways like build number, mobile vs. PC, device model, date filed, language, and Fast Ring vs. Slow Ring to address feedback.
The Insider Hub pop-ups mean something
If you feel annoyed by the constant pop-up survey questions that appear after most interactions in Windows 10, rest assured they are inching toward a goal. The reasons for the pop-ups include:
- It helps us get “in the moment” feedback from you since you just performed the action for which we’re interested in your feedback.
- By using both rating scales and free-form text input, we can easily compare progress on a featured flight over flight to gauge if we are improving.
- We can ask more granular targeted questions that improve specific aspects of the experience.
An example of the successful workflow of the pop-ups explains how PDF support was quickly added to Build 10041. The print team created one of those Windows Insider Hubs with a mini survey. After asking about the printing experience on Windows 10, many used the prompted Insider Hub pop-ups to comment on PDF support. The print team decided there was enough write-in comments about PDF support to increase the priority of doing the print to PDF feature.
Samer also highlights the diversity of Windows and its user feedback-driven crowd. Apparently, there are only four countries in the world that currently do not have a Windows Insider running Windows 10. While impressive, the sheer number of users and language allows the Windows team to see an overview of languages being employed in Windows 10. The Windows team also uses the technologies from Bing Translator to automatically translate the millions of multilingual feedback received.
Lastly, Samer hammers home the importance of creating a two-way conversation between Insiders and engineers. To achieve this, the Windows team is looking to continually add to the Insider hub in the future. One of the recent additions to the Insider Hub consists of feedback tags. “The tags will at first indicate simply that we received and processed your feedback. You’ll notice the RECEIVED tag in the Windows Feedback app. This is our way of letting you know that your feedback is in our systems, available for engineers to look at, ready to be promoted to a work item, and is actively contributing to the word clouds and trending charts that we use to assess our external flights,” according to Samer.
Another way the Windows 10 engineers would like to continue the feedback conversation is with its Community Champions program. The Windows 10 team has set aside 80 engineers dedicated to maintaining conversations the with Windows Insiders forums. Community Champions are real people, real engineers that are trying to understand the issues many of us Insiders are running into from time to time. As Insiders fiddle around with the latest Windows 10 build, they should keep this simple idea in mind, “your feedback has made Windows better.” Similar to the Windows 7 marketing material of “I made Windows 7,” Windows 10 was developed using the voice of users. The Windows 10 team would like to continue the open dialog well past the July 29th ship date also. So Insider should keep an eye out for new developments via the Insider Hub.
Now that we’ve helped build it, for Microsoft’s sake, let’s hope the vast majority want to use it.
