Qualys Discovers Two Major Vulnerabilities – Techstrong TV
Bharat Jogi, Director of Security and Threat Research at Qualys, discusses the discovery of two recent vulnerabilities, the 12 year old Linux vulnerability in PolicyKit and easy-to-exploit vulnerability in Snap, a universal application packaging and distribution system developed for Ubuntu. The discussion will include an overview of the vulnerability and include a technical discussion of methods used to uncover the vulnerability. The video is below followed by a transcript of the conversation.
Alan Shimel: Hey, everyone, welcome to another Techstrong TV interview. My guest for this interview is Mr. Bharat Jogi. Bharat, of course, is with our friends at Qualys. Hey, Bharat, welcome to Techstrong TV. How are you?
Bharat Jogi: I’m doing great, Alan. Thank you for inviting me to the show.
Alan Shimel: Oh, it’s our pleasure to have you on. So, Bharat, look, our audience is very familiar with Qualys, one of the leaders in the cybersec infosec space. But they’re probably not as familiar with you, so why don’t you, if you don’t mind, give us a little bit of your personal story. What’s your position at Qualys, what do you do?
Bharat Jogi: Sure, Alan. My name is Bharat Jogi. I’m the director for vulnerability and threat research at Qualys. Basically, what my team does on a daily basis is we are constantly involved into identifying, reviewing source code for open source technologies, and trying to find more vulnerabilities that could really impact customers, and a lot of the infosec community in general.
So, that’s what my team does on a daily basis. Along with that, obviously, we keep up with all the tsunami of vulnerabilities that get release on a daily basis. So essentially, that’s what my team does. We are pretty much spread out across the geography and we try to find the best security researchers and best talent that we can add to the team.
Alan Shimel: Absolutely. You know, this is one of the big evolutionary developments that I’ve seen over, let’s say, the last ten years, which is it used to be companies like Qualys, for instance, that had great vulnerability scanning and patching and management solutions, they didn’t do the underlying research themselves, discovering these vulnerabilities.
They counted on this sort of obscure – not obscure, but this far-flung network of security researchers that would find vulnerabilities, and then there was the whole process of getting a vulnerability into MITRE or into the CVC and having a CVC number assigned to it, and a criticality, and all of that stuff.
Then a company like a Qualys would say, okay, we need to have a scan for this vulnerability, you want the right patches for it, how to mitigate it until the patches are put forth.
Alan Shimel: Right, but now, we’ve jumped the shark a little bit where companies like Qualys, they’ve got teams like yours out here, finding vulnerabilities, first instance, if you will – what we used to call zero days, and stuff like that.
Bharat Jogi: Yes.
Alan Shimel: You’re finding the vulnerabilities themselves. How long has the research team been in operation there, do you know?
Bharat Jogi: So, I think the Qualys research team for vulnerabilities has been in operation since the day Qualys was born. Essentially, that’s how Qualys really came in. They were researchers who were doing vulnerability research, and from there, it started to expand and became a product.
So, I would say the research team, the whole research team, has been ever since Qualys was born. That is what we call it, that research has always been in the Qualys DNA. I still recall when I joined Qualys back in 2009, we were just a handful of people trying to do as much as we can.
But along the way, I think our mission, our core values, was always the same. We always want to find quality vulnerabilities before the bad actors can find it, and basically help our customers and the infosec community in general to be ahead of the bad guys.
So, I recall back in 2009 we were just a handful of people doing that. From there, we have moved, today we have a team of almost 45 people doing this on a daily basis. So yeah, it’s been a fun journey, and along this way we picked up as many as we can.
I’m very proud that today, we have always concentrated on finding quality bugs over quantity, and that is what if you see some of our reports that we publish around security advisories, especially, that we publish, like real infosec researchers really do appreciate them. Because we tend to have this old-school style of doing stuff, and we give as much detail that is needed, which I think a lot of companies today are not giving that kind of visibility to the researchers, that hey, this is a vulnerability, this is how we found it.
This is our way of giving back to the community. We have certainly learned a lot from the infosec community. We obviously review a lot of these findings that other researchers do. This is our way of giving back to the infosec community.
Alan Shimel: Absolutely. Look, on top of all this, you guys are doing great work. I mean, there’s a lot of threat research out here in the world today, and the Qualys team is doing more than their fair share. Recently, your team has identified at least two that I wanna talk about today – two very significant vulnerabilities, bugs, whatever you wanna call them, that have a pretty big effect, pretty widespread effect.
Wanted to talk about them a little bit, and in talking about them, Bharat, without giving away company secrets, let’s educate the audience about how a research team finds these bugs, how did your team come across it and what the bugs are. I think people would be fascinated to know that.
Bharat Jogi: Sure. So, in 2022, we found two critical vulnerabilities. The first disclosure that we did was what we dubbed as PwnKit. Now, this is a vulnerability that affects all Linux distributions, any of the default configuration. So, and this bug was actually introduced back in 2009.
So, all these years, the last 12, 13 years, this bug was hiding in the plain sight, and basically it affected all Linux distributions in 34 configurations, and basically gave an attacker full root privileges on any Linux distribution.
Obviously, the attack surface, the threat surface, for this vulnerability is extremely huge. Basically, the most important thing to note about this vulnerability was that we were aware at the time that we did the disclosure, we obviously engaged in the responsible disclosure, engaging all the vendors, and so on and so forth.
We were aware that this vulnerability would get exploited very soon. Like, we did not draw any exploits, et cetera, but we were aware. As soon as we did the disclosure, within a matter of hours, researchers were able to review that vulnerability and create an exploit for that.
So, I think the first time I saw something on Twitter, somebody posting about it, was within, like, three hours. So, within three hours, that vulnerability got noticed. And we essentially made it a point that, you know, when there’s something like this, that we are dropping, we are informing the community that hey, please take this thing seriously, because we definitely believe exploits for this could be created.
So, that was the first vulnerability that we disclosed in 2022. The second one that we disclosed was in February. We named it Oh, Snap! More Lemmings. That was, again, a privilege escalation vulnerability, giving attackers a full root privilege access on a Linux distribution.
Now, this vulnerability was not as ubiquitous as the PwnKit, because the Oh, Snap vulnerability only affected Ubuntu distributions in their default configurations. Obviously, if you have Snap on other distributions, they also become vulnerable, but by default, it affects only Ubuntu distribution.
But Ubuntu is a fairly popular distribution. I think from the numbers that we saw, there are, like, 40 million Ubuntu distributions in use. So, yeah, overall, very proud of the vulnerabilities that we disclosed.
I’m very happy that it was also very well received by the infosec community, given that we tend to provide as much as details around the vulnerability. Our advisories are very much detailed, so that the folks in the infosec community, and the researchers, can also understand our core processes and also learn something about the learning that we do.
Alan Shimel: Absolutely, and congratulations to the team. Let’s look at that a little bit. The first bug you said has been around since at least 2009.
Bharat Jogi: Yeah.
Alan Shimel: But no one had put out an exploit that we know of until after you had done your responsible disclosures. How do we know, though? I mean, were there – how do we know there wasn’t already an exploit out there that maybe it’s a government agency, maybe it’s an enemy or an adversarial government agency or gang or something, right?
It just seems to me if this is a bug that’s been around since 2009, how do we – we can’t be sure that there hasn’t already been an exploit on these things.
Bharat Jogi: Yeah. I mean, definitely. But what happens is, typically, in my experience, when something is getting actively exploited, some way or another this information would surface. Now, obviously, this is not to say that we would always get that, but I tend to find from the experience that I have dealing with this kind of exploit and these vulnerabilities, that if some things are actively being exploited, some way or another, we live in this hyperconnected world.
We have various logs, et cetera, that are being monitored, so, if an exploit is being actively exploited and it’s being used in tamperings at a large scale, people would tend to – somehow, this information would surface, and people would get their eyes to it.
This is not to say that none of the – there will obviously be exploits happening very covertly, and they have not surfaced. But I like to believe, given the world that we live in, that if there is active – at least if the exploits are being leveraged in any wide-scale attack, they would surface sooner, if not later.
Alan Shimel: You would hope, and you’d think, right?
Bharat Jogi: Yeah, and that’s all we can hope for, yeah.
Alan Shimel: Right. So, it’s only March whatever it is – March 1, March 2 – it’s a busy year already. Certainly finding one major vulnerability a month, I hope that it’s not that bad a year for us, right? But let’s talk about how does the team – what’d you say, there’s about 40 people on the team now?
Bharat Jogi: Yes, 45.
Alan Shimel: A team of 45 people – how do you find these vulnerabilities? Because we’re looking for vulnerabilities where there is no exploit that we know of, necessarily, right?
Bharat Jogi: Yes.
Alan Shimel: So, it’s not like we could observe someone, an incident, and say oh, this is how they got in. We’re looking for truly zero days – ones where people don’t know of. How do you find them?
Bharat Jogi: So, most of the vulnerabilities that we have found, has been through thorough source code audit. It’s a, I would say, a slow process, but it is a sure fire way of finding very quality bugs. We employ other techniques, like fuzzing, et cetera, in order to discover more vulnerabilities, but our best research, our best vulnerabilities, and the most complicated vulnerabilities that we have found are coming from doing the tons of source code audit.
That’s what the research team primarily focuses on. They do a ton of source code audit of the open source technologies, and try to uncover quality bugs. So, just to give an example, obviously source code audit is something that we continuously do, but often, sometimes, we get into this mode where we want to push the boundaries.
So, I’ll take an example of the vulnerability that we found in 2017. We called it Stack Clash. So, it’s not that it was that vulnerability that we found. Obviously, again, affected all Linux distributions, so on and so forth. But, you know, folks knew about this vulnerability in theory that yes, something like Stack Clash could happen.
But up until the point that we disclosed the Stack Clash vulnerability, those attack techniques were always deep in the code. So, we often take those opportunities to go above and beyond, and see – to push the limits.
To say, yes, this is something that the industry knew for many years, but let’s take that, and let’s see what we can add to it, and if we can create a practical means of exploiting the vulnerability, which are considered just theoretical until that point. So, we want to push those boundaries as well. That’s the same thing that –
Alan Shimel: Excellent.
Bharat Jogi: – with, you know, Oh, Snap vulnerability – Snap is something which we were aware as a very defensive style of coding. It uses all the different, latest exploit-thwarting technologies, so that even if you can find the vulnerability, you wouldn’t be able to exploit it. But our researchers basically decided to push the boundaries, and we were able to successfully exploit vulnerabilities in Snap.
Alan Shimel: Very good. Very good. Bharat, I’m going to imagine Qualys customers are probably getting a threat research feed from the Qualys team, keeping them up-to-date on the latest research that you guys are doing.
For the general public, maybe, who are not yet a Qualys customer but wants to check in on Qualys research, how would they go about it? Is there a Web page or a section of the Qualys site that is open to not just paying Qualys customers, but to the public at large?
Bharat Jogi: Yeah. So, we have a page currently on Qualys.com, and we essentially put all our research material there. Whatever vulnerabilities that we found, how we found it, to some extent. Also, after the reasonable time of disclosure of the vulnerability, we also create exploit code for the vulnerabilities that we find.
So, we publish that. We tend to do as much as – as I mentioned earlier, the goal of research is not just so that it is to integrate it into the product. But we also – this is our way of giving back to all the learnings that we have found from the infosec community.
So, this is our way to giving it back to the infosec community. So, for the Research.Qualys.com, you see all the research that Qualys has done. Also on Blogs.Qualys.com, we provided a ton of information of how you might be aware of the guidance with the Russia and Ukraine crisis.
So, we provide a lot of material around those on Blogs.Qualys.com. Also, what we do on a monthly basis, I, along with some of our senior product management from the different product management teams, we do a monthly review on vulnerabilities and patches, where we basically discuss all the vulnerabilities that were disclosed and what customers can essentially do and actions for them. So, we do try to engage in a lot of this outreach and sharing as much information as we have with the community.
Alan Shimel: Got it. Good stuff. Bharat, I told you this was gonna be 15 minutes; we’re over 20. I apologize. But you know what, I wanted people to get a flavor. We all see this research or that research, or found the bug, blah, blah. I don’t know, for people who aren’t necessarily in this end of the business, if they really understand what a threat research team is about, what they do day-to-day, how do they find these bugs, and what do they do once they do. So, thanks for coming on and enlightening us.
Bharat Jogi: Sure, Alan, it was a pleasure being here.
Alan Shimel: Keep up the great work. I gotta be honest with you – I hope you don’t continue finding a major bug every month, right? Let’s hope things are tightening up. But stay on the job, keep doing what you’re doing.
Bharat Jogi: Thanks a lot, Alan. Thanks a lot for inviting me to the show.
Alan Shimel: My pleasure. Bharat Jogi, Qualys threat research team, here on Techstrong TV. We’re gonna take a break. We’ll be right back with another guest.
