
7月19日凌晨4點,,當邁克爾·阿默(Michael Armer)的手機被打爆時,,他“感到驚慌失措”。阿默是RingCentral的首席信息安全官,,他收到了關于一場令人震驚的計算機故障的通知,,這場故障像多米諾骨牌一樣導致機場、銀行和醫(yī)院的技術系統紛紛癱瘓,。
混亂的范圍引發(fā)了人們對重大網絡安全漏洞或國家支持的攻擊的擔憂,。阿默說:“這足以讓你血脈僨張?!?
事實證明,,這次大規(guī)模的計算機故障并非邪惡黑客所為,而是安全公司CrowdStrike在例行軟件更新中出現故障的結果,。阿默在談到CrowdStrike的更新故障時說:“我們都很幸運,,這與他們的標準化和自動化軟件部署有關?!?/p>
不過,,在慶幸這次破壞不是網絡攻擊的同時,這次事件也凸顯了現代社會所依賴的技術的脆弱性和可怕的互聯性,,以及當今錯綜復雜的軟件更新系統所帶來的危險程度,。安全專家表示,即使是規(guī)模最大的企業(yè)也會讓員工不堪重負,,并迫使他們不斷進行風險權衡的艱難抉擇,。
補丁的問題
當檢測到威脅時,CrowdStrike等安全軟件會提供“補丁”或軟件更新,。鑒于探測公司系統并設計新攻擊路線的黑客數量之多,,對補丁的需求是持續(xù)不斷的,有時甚至一天數次,。各企業(yè)行動迅速,,通常會自動進行這些更新,以確保其防護盾沒有漏洞,。
問題是,,新軟件就像未經測試的藥物——每一行新代碼都可能有漏洞或缺陷,從而導致問題,、意想不到的副作用,,以及與其他軟件的危險交互。在理想情況下,公司會花時間測試每個軟件更新,,然后再將其部署到所有計算機上,。
紐約一家頂級律師事務所的首席信息安全官表示:“這確實是一個棘手的難題,你無法跟上黑客的數量,。有時,,你必須發(fā)布安全補丁,原因是它很關鍵,,而供應商一直在緊盯著你,,你根本無法對它進行[測試]。有時24小時內會有幾次更新,,這樣你就會陷入反復測試的怪圈,,永遠無法完成測試?!?/p>
對于許多內部安全團隊來說,,這意味著要在速度和風險之間取得平衡。軟件供應鏈平臺捷蛙科技(JFrog)的首席信息安全官保羅·戴維斯(Paul Davis)表示:“防病毒產品每天都要推出多次更新,,因為在某種程度上,,我們已將其逼入絕境。它們檢測軟件或惡意活動的反應速度越快,,就越有優(yōu)勢,。因此,在這種情況下,,每天進行多次測試的要求變得非常繁重,。”
他說,,真正的挑戰(zhàn)是如何保護企業(yè)應對可能在數小時甚至數分鐘內傳播的網絡安全威脅,,同時確保這些軟件更新經過測試?!拔覀儽仨殰y試軟件的基本功能,,但我們依靠這些自動更新來確保安全,這幾乎就像是一種經過計算的風險,?!?
對每臺受影響的計算機進行現場心肺復蘇術
這家位于紐約的律師事務所使用了來自不同供應商的30多種獨立安全工具,這些工具可在筆記本電腦,、臺式機或服務器上運行,。通常情況下,如果更新導致問題,,軟件供應商會部署一個修復程序,,企業(yè)可以在同一天內迅速將其推送到數千臺計算機上,。
但由于CrowdStrike漏洞的性質,這是無法實現的,。該漏洞導致運行微軟(Microsoft)Windows系統的電腦死機,并顯示可怕的“藍屏死機”,。受影響的系統需要一個接一個地恢復正常,。
紐約律師事務所的首席信息安全官解釋說:“你必須親自走到每臺電腦前,關掉電源,,然后再開機,,當屏幕亮起時,你必須按F3鍵進入所謂的安全模式,,然后去刪除在某個位置存放文件,。這簡直就是一場噩夢?!?/p>
然而,,一些首席信息安全官將大部分責任歸咎于微軟,而不是Crowdstrike,,甚至盡可能避免使用Windows,。一家中型人工智能公司的首席信息安全官表示:“在硅谷,科技公司傾向于避免使用Windows,?!庇捎谟懻摪踩δ芫徑獯胧┑拿舾行裕竽涿?。他說,,這是因為Windows核心架構的設計導致了惡意軟件、間諜軟件以及今天因Crowdstrike漏洞更新而出現的驅動程序不穩(wěn)定,。
他說:“CrowdStrike無疑需要進行流程改進,,但在2024年不應該出現內核(核心架構)被第三方破壞穩(wěn)定性的情況。從安全的角度來看,,微軟今年的表現很糟糕,,必須贏得生態(tài)系統的信任?!蔽④洓]有回應置評請求,,只是指出了關于此次故障的聲明。
CrowdStrike首席執(zhí)行官喬治·庫爾茨(George Kurtz)7月19日在網上發(fā)表聲明,,為這一事件道歉,,他說此次事件涉及到“Windows主機的內容更新”,并指出Mac和Linux主機不受影響,。
“CrowdStrike的全體員工都明白這一事件的嚴重性和影響,。我們迅速查明了問題所在,,并部署了修復程序,從而能夠專注于恢復客戶系統,,這是我們的首要任務,。”
事后分析
捷蛙科技的戴維斯反駁了一般企業(yè)可以不使用 Windows 的觀點,。他說:“Windows仍然是占主導地位的操作系統,。當你加入一家公司時,(通常)會提供給你一臺Windows電腦或Mac電腦,?!?/p>
身份安全公司Silverfort的首席信息安全官約翰·保羅·坎寧安(John Paul Cunningham)表示,7月19日的宕機事件應該給企業(yè)敲響警鐘,,讓企業(yè)對自動軟件更新更加謹慎,。在坎寧安看來,并非所有的威脅都是一樣的,,企業(yè)需要更加謹慎,,不要總是默認進行自動更新。
他表示:“像CrowdStrike這樣的公司經常建議進行自動更新,,但是這一前提是使用最新版本的產品更安全,。”但他說,,公司可以在推送之前花更多時間測試,,即使這需要多花點功夫?!爸灰踩珗F隊知道有更新,,他們就可以手動推送,而更新本身仍是自動進行的,?!?/p>
RingCentral的阿默表示,對于大多數網絡安全領導者來說,,如何在風險和速度之間以及各大操作系統之間取得平衡,,需要進行一些事后分析和決策。
雖然進行軟件更新很重要,,但他指出,,公司也應慶幸7月19日的宕機沒有帶來更糟糕的后果。他說:“我個人很慶幸,,這不是一次國家支持的攻擊,。”(財富中文網)
譯者:中慧言-王芳
7月19日凌晨4點,,當邁克爾·阿默(Michael Armer)的手機被打爆時,,他“感到驚慌失措”,。阿默是RingCentral的首席信息安全官,他收到了關于一場令人震驚的計算機故障的通知,,這場故障像多米諾骨牌一樣導致機場,、銀行和醫(yī)院的技術系統紛紛癱瘓。
混亂的范圍引發(fā)了人們對重大網絡安全漏洞或國家支持的攻擊的擔憂,。阿默說:“這足以讓你血脈僨張,。”
事實證明,,這次大規(guī)模的計算機故障并非邪惡黑客所為,而是安全公司CrowdStrike在例行軟件更新中出現故障的結果,。阿默在談到CrowdStrike的更新故障時說:“我們都很幸運,,這與他們的標準化和自動化軟件部署有關?!?/p>
不過,,在慶幸這次破壞不是網絡攻擊的同時,這次事件也凸顯了現代社會所依賴的技術的脆弱性和可怕的互聯性,,以及當今錯綜復雜的軟件更新系統所帶來的危險程度,。安全專家表示,即使是規(guī)模最大的企業(yè)也會讓員工不堪重負,,并迫使他們不斷進行風險權衡的艱難抉擇,。
補丁的問題
當檢測到威脅時,CrowdStrike等安全軟件會提供“補丁”或軟件更新,。鑒于探測公司系統并設計新攻擊路線的黑客數量之多,,對補丁的需求是持續(xù)不斷的,有時甚至一天數次,。各企業(yè)行動迅速,,通常會自動進行這些更新,以確保其防護盾沒有漏洞,。
問題是,,新軟件就像未經測試的藥物——每一行新代碼都可能有漏洞或缺陷,從而導致問題,、意想不到的副作用,,以及與其他軟件的危險交互。在理想情況下,,公司會花時間測試每個軟件更新,,然后再將其部署到所有計算機上。
紐約一家頂級律師事務所的首席信息安全官表示:“這確實是一個棘手的難題,,你無法跟上黑客的數量,。有時,,你必須發(fā)布安全補丁,原因是它很關鍵,,而供應商一直在緊盯著你,,你根本無法對它進行[測試]。有時24小時內會有幾次更新,,這樣你就會陷入反復測試的怪圈,,永遠無法完成測試?!?/p>
對于許多內部安全團隊來說,,這意味著要在速度和風險之間取得平衡。軟件供應鏈平臺捷蛙科技(JFrog)的首席信息安全官保羅·戴維斯(Paul Davis)表示:“防病毒產品每天都要推出多次更新,,因為在某種程度上,,我們已將其逼入絕境。它們檢測軟件或惡意活動的反應速度越快,,就越有優(yōu)勢,。因此,在這種情況下,,每天進行多次測試的要求變得非常繁重,。”
他說,,真正的挑戰(zhàn)是如何保護企業(yè)應對可能在數小時甚至數分鐘內傳播的網絡安全威脅,,同時確保這些軟件更新經過測試?!拔覀儽仨殰y試軟件的基本功能,,但我們依靠這些自動更新來確保安全,這幾乎就像是一種經過計算的風險,?!?
對每臺受影響的計算機進行現場心肺復蘇術
這家位于紐約的律師事務所使用了來自不同供應商的30多種獨立安全工具,這些工具可在筆記本電腦,、臺式機或服務器上運行,。通常情況下,如果更新導致問題,,軟件供應商會部署一個修復程序,,企業(yè)可以在同一天內迅速將其推送到數千臺計算機上。
但由于CrowdStrike漏洞的性質,,這是無法實現的,。該漏洞導致運行微軟(Microsoft)Windows系統的電腦死機,并顯示可怕的“藍屏死機”,。受影響的系統需要一個接一個地恢復正常,。
紐約律師事務所的首席信息安全官解釋說:“你必須親自走到每臺電腦前,,關掉電源,然后再開機,,當屏幕亮起時,,你必須按F3鍵進入所謂的安全模式,然后去刪除在某個位置存放文件,。這簡直就是一場噩夢,。”
然而,,一些首席信息安全官將大部分責任歸咎于微軟,,而不是Crowdstrike,甚至盡可能避免使用Windows,。一家中型人工智能公司的首席信息安全官表示:“在硅谷,,科技公司傾向于避免使用Windows?!庇捎谟懻摪踩δ芫徑獯胧┑拿舾行?,他要求匿名,。他說,,這是因為Windows核心架構的設計導致了惡意軟件、間諜軟件以及今天因Crowdstrike漏洞更新而出現的驅動程序不穩(wěn)定,。
他說:“CrowdStrike無疑需要進行流程改進,,但在2024年不應該出現內核(核心架構)被第三方破壞穩(wěn)定性的情況。從安全的角度來看,,微軟今年的表現很糟糕,,必須贏得生態(tài)系統的信任?!蔽④洓]有回應置評請求,,只是指出了關于此次故障的聲明。
CrowdStrike首席執(zhí)行官喬治·庫爾茨(George Kurtz)7月19日在網上發(fā)表聲明,,為這一事件道歉,,他說此次事件涉及到“Windows主機的內容更新”,并指出Mac和Linux主機不受影響,。
“CrowdStrike的全體員工都明白這一事件的嚴重性和影響,。我們迅速查明了問題所在,并部署了修復程序,,從而能夠專注于恢復客戶系統,,這是我們的首要任務?!?/p>
事后分析
捷蛙科技的戴維斯反駁了一般企業(yè)可以不使用 Windows 的觀點,。他說:“Windows仍然是占主導地位的操作系統,。當你加入一家公司時,(通常)會提供給你一臺Windows電腦或Mac電腦,?!?/p>
身份安全公司Silverfort的首席信息安全官約翰·保羅·坎寧安(John Paul Cunningham)表示,7月19日的宕機事件應該給企業(yè)敲響警鐘,,讓企業(yè)對自動軟件更新更加謹慎,。在坎寧安看來,并非所有的威脅都是一樣的,,企業(yè)需要更加謹慎,,不要總是默認進行自動更新。
他表示:“像CrowdStrike這樣的公司經常建議進行自動更新,,但是這一前提是使用最新版本的產品更安全,。”但他說,,公司可以在推送之前花更多時間測試,,即使這需要多花點功夫?!爸灰踩珗F隊知道有更新,,他們就可以手動推送,而更新本身仍是自動進行的,?!?/p>
RingCentral的阿默表示,對于大多數網絡安全領導者來說,,如何在風險和速度之間以及各大操作系統之間取得平衡,,需要進行一些事后分析和決策。
雖然進行軟件更新很重要,,但他指出,,公司也應慶幸7月19日的宕機沒有帶來更糟糕的后果。他說:“我個人很慶幸,,這不是一次國家支持的攻擊,。”(財富中文網)
譯者:中慧言-王芳
When Michael Armer’s phone started blowing up at 4 a.m. Friday morning, he “freaked out.” Armer, the chief information security officer at RingCentral, was receiving notifications about a stunning computer outage that was knocking down airport, bank, and hospital tech systems like dominos.
The?scope of the chaos raised fears of a major cybersecurity breach or a state-sponsored attack. “That’s enough to get your blood flowing really quickly,” Armer said.
It turns out that the massive computer outage was not the work of nefarious hackers. It was the result of a glitch in a routine software update by security company CrowdStrike. “We were all very fortunate that this was related to one of their standardized and automated software deployments,” Armer says of the CrowdStrike update snafu.
But along with the relief that the disruption was not a cyber attack, the incident has highlighted the fragility and frightening interconnectedness of the technology?modern society depends on — and the extent of the danger posed by today’s convoluted system of software updates which security experts say stretches staff thin at even the largest organizations and forces a constant balancing act of risky trade-offs.
The problem with patches
Security software like CrowdStrike provide “patches,” or software updates, when threats are detected. Given the number of hackers probing companies’ systems and devising new lines of attack, the need for patches is constant — sometimes as many as several times a day. Organizations move quickly and often automate these updates to ensure that there are no holes in their protective shields.
The problem is that new software is like an untested pharmaceutical drug – each new line of code could have a bug or defect that causes problems, unexpected side effects, and dangerous interactions with other software. In an ideal situation, a company would take the time to test each software update before deploying it to all their computers.
“It’s a really difficult conundrum, you cannot keep up with the number,” said a CISO at a top law firm in New York City. “Sometimes you have to put out a security patch because it’s critical and you’ve got vendors breathing down your neck and there’s no way to [test] it,” he said. “Sometimes there are several updates within a 24-hour period so you’d be caught in a recursive circle of testing where you would just never be done.”
For many in-house security teams, that means striking a balance between speed and risk. “The antivirus products are pushing up multiple updates per day because in some ways we’ve pushed them into a corner,” said Paul Davis, field CISO at software supply chain platform JFrog. “The faster that they can respond to detect a piece of software or malicious activity, the better they are. So that being the case, then the requirement to test multiple times a day becomes onerous.”
The real challenge, he said, is how to protect the organization that is responding to cybersecurity threats which can spread in hours, or even minutes, and at the same time make sure those software updates are tested. “We have to test the basic functionality of the software, but we rely on these automated updates to be safe, and it’s almost like a calculated risk.”
Hands-on CPR for each affected computer
The New York City law firm uses more than 30 separate security tools from a variety of vendors that run on laptops, desktops or servers. Normally, if an update causes problems, the software vendor will deploy a fix that an organization can quickly push to thousands of computers within the same day.
But because of the nature of the CrowdStrike flaw however, that wasn’t possible. The flaw essentially caused computers running Microsoft Windows to freeze up and display the dreaded “blue screen of death.” Affected systems needed to be brought back to life, one by one.
“You have to physically walk over to every computer and power it down and then bring it up, and when the screen comes up, you have to hit F3 to go into what they call Safe Mode and then go and delete a file somewhere,” the New York law firm CISO explained. “It’s just a nightmare.”
Some CISOs, however, put the bulk of the blame on Microsoft, not on Crowdstrike– and even avoid Windows altogether if they can. “In Silicon Valley, tech companies tend to avoid Windows,” said the CISO of a medium-sized AI company, who requested anonymity due to the sensitivity of discussing security mitigations. He said that it is because of the design of Windows in its core architecture that leads to malware, spyware and the driver instability that occurred today as a result of the Crowdstrike flawed update.
“CrowdStrike has clear process improvements to make, obviously, but it should not be possible in 2024 to have a kernel [core architecture] which is destabilized by a third party,” he said. “Microsoft has had a bad year, from a security perspective, and they have to win the trust of the ecosystem back.” Microsoft did not respond to a request for comment other than pointing to its existing statement about the outage.
In a statement posted online Friday, CrowdStrike CEO George Kurtz apologized for the incident, which he said involved a “content update for Windows hosts,” noting that Mac and Linux hosts were not affected.
“All of CrowdStrike understands the gravity and impact of the situation. We quickly identified the issue and deployed a fix, allowing us to focus diligently on restoring customer systems as our highest priority.”
Post-game analysis
JFrog’s Davis pushed back on the idea that a typical organization could get away with not using Windows. “Windows is still the predominant operating system,” he said. “When you join a company, you’re [usually] offered either a Windows machine or a Mac machine.”
John Paul Cunningham, CISO at identity security company Silverfort, said that Friday’s outage should be a wake-up for call for organizations, and make companies more leery of automated software updates. In Cunningham’s view, all threats are not created equal and companies can exercise more discretion by not always defaulting to the automated updates.
“Companies like CrowdStrike often suggest doing auto updates with this premise that staying on the most current release of the product is more secure,” he said. But companies can take more time to test it before pushing it out, he said, even if it takes a little more work. “As long as the security team knows there is an update, they can push it out manually–the update itself is still automatic.”
The bottom line is that for most cybersecurity leaders, figuring out how to strike a balance—between risk and speed, and between operating systems—will require some post-game analysis and decision-making, said RingCentral’s Armer.
And while getting a grip on software updates is important, he noted that companies should also be thankful Friday’s outage was not even worse. “I personally am thankful that it wasn’t a state-sponsored attack,” he said.