Insecure Storage Buckets Expose 1.8 Billion Online Posts Scraped for U.S. Military

A Pentagon contractor left three storage buckets publicly accessible on Amazon’s S3 service, exposing more than 1.8 billion online posts collected since 2009. The messages, posted by people from around the world, were likely collected as part of an intelligence-gathering operation for the U.S. military.

The breach was discovered by researchers from UpGuard, a company that has identified many instances of misconfigured cloud-based services over the past year. According to them, the storage buckets allowed any Amazon Web Services (AWS) global authenticated user—a type of account that can be obtained with a free signup—to access and download their contents.

The UpGuard researchers found details on the servers indicating that the software used to collect the data was created by a now-defunct government contractor called VendorX. The buckets’ names were centcom-backup, centcom-archive and pacom-archive, names that likely refer to the U.S. Central Command (CENTCOM) and the U.S. Pacific Command (PACOM).

Information in the archives also hinted that the content was related to Outpost, a project that some former VendorX employees described on their LinkedIn pages as “a multi-lingual social analytics platform” built exclusively for CENTOM and used “to positively influence change in high-risk youth in unstable regions of the world.”

The scraped content included online posts in many languages from news sites, comment sections, web forums and social media sites, made by users from specific regions of the world that are of interest to the U.S. military. The content in the centcom archive was more focused on the Middle East and South Asia while the posts in the pacom archive were more focused on Southeast and East Asia and Australia. However, content posted online by U.S. citizens were also included in the archives.

“While a cursory examination of the data reveals loose correlations of some of the scraped data to regional US security concerns, such as with posts concerning Iraqi and Pakistani politics, the apparently benign nature of the vast number of captured global posts, as well as the origination of many of them from within the US, raises serious concerns about the extent and legality of known Pentagon surveillance against US citizens,” the UpGuard researchers said in a blog post.

This is not the first time when misconfigured Amazon S3 buckets operated by third-party vendors has led to the exposure of potentially sensitive data from other companies.

In July, the UpGuard researchers found an S3 storage bucket created by a vendor called NICE Systems that contained the names, addresses and account details of 14 million Verizon customers. In September, another misconfigured S3 repository exposed 9,400 résumés of veterans and people with government security clearances who applied to work for a private security company called TigerSwan. The bucket had been created by a recruitment agency called TalentPen, which had a contract with TigerSwan.

“This cloud leak is a striking illustration of just how damaging third-party vendor risk can be, capable of affecting even the highest echelons of the Pentagon,” the UpGuard researchers said about the latest CENTCOM and PACOM leak.

Last week, researchers from another security company, Kromtech Alliance, found two publicly exposed Amazon S3 buckets with data from the Australian Broadcasting Corporation. The data included 1,800 daily MySQL database backups from 2015 to present, emails, logins, hashed passwords and secret access keys and login details for another repository.

Earlier this month, Amazon introduced new security features for its S3 storage service that include default encryption and permission checks that displays a prominent indicator next to each S3 bucket that’s publicly accessible. Kromtech has also developed a free tool called the S3 Inspector that allows companies to check the permissions on their S3 buckets and determine if they’re exposed publicly or not.

Lucian Constantin

Lucian has been covering computer security and the hacker culture for almost a decade, his work appearing in many technology publications including PCWorld, Computerworld, Network World, CIO, CSO, Forbes and The Inquirer. He has a bachelor’s degree in political science, but has been passionate about computers and cybersecurity from an early age. Before he chose a career in journalism, Lucian worked as a system and network administrator. He enjoys attending security conferences and delving into interesting research papers. You can reach him at lucian@constantinsecurity.com or @lconstantin on Twitter. For encrypted email, his PGP key’s fingerprint is: 7A66 4901 5CDA 844E 8C6D 04D5 2BB4 6332 FC52 6D42

lucian-constantin has 58 posts and counting.See all posts by lucian-constantin