Susan Bradley, an 18 year Microsoft MVP focused on Windows patching and patch management, has sent an open letter to Microsoft executives Satya Nadella, Carlos Picoto, and Scott Guthrie about the frustration Windows 10 users have when dealing with installing new updates. This letter includes the results of a survey taken by over 1,000 consultants and over 800 consumers regarding their experience with Windows 10 updates.
Being a Microsoft MVP in Consumer Security, I have known Susan for quite some time and can tell you that she is somewhat of a legend among those who regularly support Microsoft products. When I saw her open letter mentioned at AskWoody.com and posted at ComputerWorld, I read through many of the survey comments and decided to reach out to her to find out more about why she wrote the letter.
"It's due to increasing frustration with patching and patch management issues. I see consultants turning off updates completely as they see that as the only way they can have stable systems I see Server admins saying that Server 2016 is fine except when you want a system that doesn't reboot. I see Surface users get their machines cratered with 1803 side effects with SSD drives." Susan told me via email. "I see more and more people say that waiting to install updates is what one must do. All of these are unacceptable. We can't continue on with the status quo."
While there has been no public response to her letter, Susan has told Bleeping Computer that she had been contacted by a Microsoft Customer manager regarding the letter to tell her that they were looking into her concerns.
Thankfully, Susan gave me permission to repost her open letter to Microsoft executives below. Anyone who supports or routinely finds problems with Windows 10 updates, will definitely find it an interesting read. If you want to see a full size image the survey results, you can click on the images below.
Mr. Satya Nadella, Mr. Carlos Picoto, and Mr. Scott Guthrie
Today, as Windows 10 turns three years old, I am writing to you to ensure that you are aware of the dissatisfaction your customers have with the updates released Windows desktops and servers in the recent months. The quality of updates released in the month of July, in particular, have placed customers in a quandary: Install updates and face issues with applications, or not install updates and leave machines subject to attack.
In the month of July 2018 alone there are 47 knowledge base bulletins with known issues. Some of these were stop issues, but most concerning were the .NET side effects with your own software: SharePoint, BizTalk, and even Exchange servers were impacted by these July 10th updates.
I am a Moderator on a community listserve that focuses on the topic of Patch Management, patchmanagement.org. Recently many of the participants on the listserve have expressed their concerns and dissatisfaction with the quality of updates as well as the timing of updates.
I recently asked the list members to answer several questions about patching on Windows 7 to Windows 10. The full results of this unscientific survey can be read here. I urge you to take the time to read the responses. It showcases that your customers who are in charge of patching and maintaining systems are not happy with the quality of updates, the cadence of feature releases and feel that it cannot go on as is.
Question 1 I asked on a scale of 1 to 5, 5 being the highest how satisfied are you with the quality of Windows updates in general?
Many respondents were not satisfied with Windows updating in general.
Question 2 I asked specifically about satisfaction of patching of Windows 10 specifically:
Many respondents were not happy with the quality of Windows 10 updates.
Question 3 I asked if Windows 10 feature updates were useful to the respondents’ business needs.
Many respondents indicated that the feature updates were either not useful at all or rarely useful to their business needs.
In question 4, I asked regarding the cadence of feature releases.
Most of the survey respondents did not want feature releases as often as they are being released now.
In question 5 I asked if Windows 10 meeting your business needs?
Most of the survey respondents answered that it was meeting their needs.
Finally, I asked an open-ended question as to what could be changed in Windows 10 to make it better for your business. You can read the response to Question 6 here.
I also did a similar survey for consumers. The results of the survey targeted to consumers were similar to the results from the consultants and patching administrators. The majority thought that the feature updates occurred too many times during the year, that they were overall not happy with the quality of updates from Microsoft. The full survey results from your Microsoft consumer customers can be found here.
I urge you to take the time to look at both the results from Patching administrators, and also consumers and home users in detail. You will see similar trends in both surveys.
Insider process is not identifying issues
It appears that there is a break down in the testing process. The Windows 10 insider process is not able to identify issues on released products. When your own products break with these releases, it is clear that current testing processes are not good enough.
It is concerning when issues with Microsoft’s own software releases have detrimental side effects with other Microsoft software. Case in point the recent .NET 4.7.2 and Azure AD connect that causes side effects and issues with high CPU.
At one time you had a program called the Security Update Validation Program that allowed firms with special nondisclosure agreements to test security updates ahead of their release. I urge you to increase this program and include a broader testing process. While your MSRC communication says that for best practice one needs to install updates immediately, but the reality is that the prudent patcher is waiting at least a week, if not more, before they install updates. I hope you find this trend as concerning and disturbing as I do.
Feature releases are causing patch fatigue
I am disturbed when I see users and consultants talk about taking drastic measures to take back control of updating and rebooting. Some are disabling windows update as a drastic measure to ensure that updates do not reboot systems when they are not wanted. It’s clear that your team also acknowledge that unexpected updates are problematic. But your customers deserve better than “promising” results. They deserve a stable platform that reboots only when they want it to. The operating system needs to do a better job of communicating to the end user and especially to the patching administrator when a machine will receive an update. The addition of the Windows Update for Business settings that often conflict with other group policy settings cause confusion, not clarity.
While it’s commendable that you’ve listened to feedback and made changes to Windows update during these three years, the fact is that these changes in each version release have caused confusion, and in some cases behavior that was not expected at all. Dual scan is one such change that caused confusion, and as a side effect caused administrators to have updates installed when they did not want them. The lack of clear communication regarding update changes leads to this confusion. Administrators are having to follow various blogs and sites and even twitter channels to be able to understand the changes. The lack of basic documentation of windows update error codes, the fact that it took several feature releases to make changes to the unreadable Windows update log, the fact that it took several feature releases before acknowledging the problem of symbol publishing showcases that the changes in windows updating have had a major impact in the servicing and handling of Windows 10. I personally know of several large enterprises are not on the current Semi Annual channel release of 1803 and are in fact several feature releases behind. The constant change and churn is not helping firms in their deployment strategies.
Patch communication needs work.
Starting in January of this year with the release of Spectre/Meltdown patches, there have been numerous instances where patching communication has been wrong, registry entries detailed in Knowledge base articles regarding registry key application was initially incorrect and later updated, or vendor updates had to be stopped and in general patching communication has been lacking. We in the patching community understand that the coordination with other vendors means that this communication process was not easy, but needless to say, communication and follow up in regards to side effect and known issues needs to be faster and more communicative. On a regular basis, it is difficult to identify if there are known issues with an update and if our firms will be directly impacted. Often the patching known issues refer to undefined “third party software” and we often must ask each other in the patching community If we were impacted and what vendors we were using. Clarity in documenting known issues would be greatly appreciated.
Impact on Azure.
When one downloads a Windows 10 virtual machine in Azure and deploys it, is often built from a release from several months ago. These patching side effects we see in the traditional operating system channels, impact patching on Azure as well. Recently a RDP patch that was released in March and ultimately implemented fully in June impacted Azure virtual machines. The fact that you had to release a Knowledge base article to instruct customers to go around this issue showcases that delays in patching Azure, and the lack of clear patching communication causes ripple effects to your cloud platforms.
I ask you to take time out of your very busy schedule to review these survey results and see the customer dissatisfaction. Many of your customers are not happy. We need action to fix these issues with patch quality.
As both a user of Microsoft software, and a shareholder of Microsoft, please take this feedback as it’s intended: We want Microsoft software to be such that we can indeed install all updates and patches immediately without reservation. As it stands right now, we do not trust the software and the patching quality enough to do so.
I thank you in advance for the opportunity to share with you your customers’ views.
Moderator at Patchmanagement.org
Writer on the topic of patches for Askwoody.com
July 29, 2018