SBN

Analyzing the Effects of COVID-19 on mPulse Traffic

The events surrounding the COVID-19 pandemic, and in particular various social-distancing measures (quarantine, shelter in place, etc.), have had an impact on people’s lives and routines across the world. As internet usage is a large part of those routines, we wanted to see how usage changed as the events of the pandemic unfolded. mPulse — our performance measurement tool — collects anonymous real user monitoring (RUM) data from a large number of websites spanning a wide range of verticals including media, online retail, travel, finance, gaming, groceries, pharmacies, food, and more. This study deliberately ignores web performance data and focuses only on traffic volume, as we wanted to study changes in behavior rather than user experience.

We’ve found that in every part of the world, major local events have had a significant impact — either positive or negative — on web traffic. Whether it’s people looking for information about an event, shopping online to avoid going to a store, or working more from home due to business closures, there exists a “visible” causal relationship between events and traffic. In many cases where the data did not match a known COVID-19 event, we found that there were other local cultural or political events that had an effect instead.

DevOps Connect:DevSecOps @ RSAC 2022

The following report will go into the results in detail, split by region using a sample of our data.

For all groups, we calculate traffic baseline separately for each day of the week by using the arithmetic mean of traffic for that day for six weeks, ranging from the second to seventh weeks of 2020, i.e., we have a separate baseline for Mondays, Tuesdays, Wednesdays, and so on.

Summary

Internet traffic varies based on several different local events. Looking at common traffic trends across multiple regions helps surface events.

In almost every part of the world, lockdowns or social distancing measures led to a move of traffic from office to home or mobile, while major announcements led to an increase in traffic.

We also see that certain holidays and national events lead to a drop in traffic. Lunar New Year in several Asian countries, Holi in India, Super Bowl Sunday in the U.S., Holy Week, and Good Friday, in particular, in Mexico all had a drop in traffic.

Other local events result in significant spikes. The death of Kobe Bryant, the anniversary of Charlie Hebdo in France, Three Kings Day in Spain, and Leap Day in China all resulted in traffic way above what was expected for that day.

Valentine’s Day weekend appears to be cultural. In many countries it resulted in a drop in traffic, but in China and South Korea it resulted in a spike.

People moving to work or study from home can have a significant impact on the population of some states. Virginia saw a big drop in traffic when the University of Virginia moved its classes online, and New York saw traffic move to New Jersey and Connecticut.

Lastly, as restrictions have loosened, we have seen traffic patterns change again. China’s traffic started picking up when lockdowns started being lifted. Singapore saw a second wave show up in late March, which resulted in traffic finally moving homeward.

Overall Data Summary

Page Views   Sample of 85 billion+

Countries      18

Date Range   Tuesday, December 31, 2019 – Thursday, April 23, 2020

Selected Regions by First Confirmed Report

  1. Mainland China (Wuhan): Tue, Dec 31
  2. Japan: Thu, Jan 16
  3. United States: Mon, Jan 20
  4. South Korea: Mon, Jan 20
  5. Taiwan: Tue, Jan 21
  6. Hong Kong: Thu, Jan 23
  7. Singapore: Thu, Jan 23
  8. France: Fri, Jan 24
  9. Canada: Mon, Jan 27
  10. Germany: Mon, Jan 27
  11. India: Thu, Jan 30
  12. United Kingdom: Fri, Jan 31
  13. Spain: Fri, Jan 31
  14. Italy: Fri, Jan 31
  15. Mexico: Fri, Feb 28

Icon Legend

The following icons will be used through all charts to show various events:

  • ① — First Case reported
  • ⛔️ — Travel Restrictions
  • ☮︎ — Peace Memorial Day
  • 🏠 — Stay-at-home or shelter-in-place advisory/order
  • 🔒 — Lockdown ordered
  • 🌍😷 or 🌎😷 or 🌏😷 — WHO declared pandemic
  • 🌘 & 🌔 — Start and end of Lunar New Year
  • 🏮 — Lantern Festival
  • 🎩 — U.S. President’s Day
  • 🏈 — U.S. Super Bowl Sunday
  • 💝 — Valentine’s Day
  • National Flag — National Day

Some icons may be combined to show multiple events on the same day.

Analysis

China

Let’s start by looking at the data for China.

For the mainland, on the face of it, we see that traffic spiked around the time that the virus was identified as a new type of SARS coronavirus. This report had no apparent effect on traffic anywhere else in the world. We then see a drop in traffic for Lunar New Year, which ranges from January 25 to February 2; however, the annual Chunyun (travel home for the New Year) started 15 days prior (January 10), so traffic was expected to drop earlier as people traveled. There were additional spikes around February 12 and 28.

Traffic started to move back to normal only after the Lantern Festival on February 8. Lastly, we see traffic increase again after the lockdown was lifted on April 7.

Tellis_China.png

Types of Sites

Looking at the types of sites being visited, we see that the majority of traffic is to online retail sites. The three spikes on January 9 (before the start of Chunyun), Valentine’s Day, and Peace Memorial Day/Leap Day were all tied to people shopping online. Traffic during the Lunar New Year was down across all sites.

Once the extended holiday ends, we see a consistent increase in media websites (news, television, and radio) and a steady flow of traffic over baseline for professional software and services. These include things like ISPs, antivirus software, productivity software, etc. — that is, things that people need to work from home.

Tellis_China2.png

Traffic Source

If we look at how traffic moves among home, mobile, and office, we see some distinct events.

For the weekend starting January 10, there was a slight movement of traffic from home to mobile. It’s likely that people were traveling for Chunyun. We see a similar movement in South Korea and Taiwan. The traditional weekend pattern only resumed after March 6.

The start of Lunar New Year on January 25 showed a reduction in office traffic, which only started to pick back up on April 7 when the lockdown was lifted.

Tellis_China3.png

China Conclusion

In general, the majority of China’s traffic was to retail sites, and followed expected shopping events and vacation timelines. The Lunar New Year holiday pushed more traffic to mobile, and that only returned to normal in early March.

Other Nearby Regions

The chart for Hong Kong has a much clearer move in traffic location, starting when the first case was confirmed.

Singapore appeared to be business as usual until late in March when their second wave hit, then traffic moved homeward.

In Taiwan, it appears that people tended to be out on weekends, with traffic shifting from home to mobile.

Tellis_China4.png

India

We see a small spike in traffic when the first case was confirmed in India, and an increase in traffic after the WHO called it a pandemic. There was a spike after a curfew was announced, and traffic dropped after the country was put in lockdown. We see that traffic moved mostly to mobile at this point.

It’s also interesting to note the drop in traffic leading up to Holi, which is a big festival in India.

Tellis_India.pngLooking at the breakdown by vertical in India, we see that media sites picked up as soon as the curfew was announced, while retail and travel sites dropped at the same time. Additionally, financial sites saw a huge spike when the first case was announced, and again when the curfew was announced.

Tellis_India2.png

The U.K.

We see almost no reaction to COVID events from the U.K. until a lockdown was announced on March 22. Traffic jumped up 50% at that point, and also moved away from mobile and office to home. The increase in traffic was completely due to media sites.

Tellis_UK.png

France, Germany, and Spain

We see similar trends in France, Germany, and Spain, with traffic increasing around the announcement of social distancing measures. In Spain there appears to be a spike after the Day of Andalucía, which coincided with a massive spread of the virus to Andalucía. Traffic moved from mobile to home in France and Spain, but had the opposite effect in Germany. We also see a flattening out of the weekend trend, indicating that traffic habits were similar between weekdays and weekends.

We see a spike in traffic in France and Spain around Three Kings Day. This coincided with the fifth anniversary of the Charlie Hebdo attack in France; however, in both countries the spike was associated with retail traffic.

Post lockdown, traffic in France and Spain was predominantly media-related, whereas in Germany, it was retail.

 Tellis_FR,IT,DE.png

Italy

Italy’s data appears to be a bit different from elsewhere. We see very little deviation from the baseline and very little change in traffic location. There is a slight flattening of the weekday-weekend trend after the pandemic was declared on March 11.

 Tellis_Italy.png

Looking at the breakdown, we see spikes in retail traffic just before La Befana on January 6, which is a big Christmas gifting day in Italy. The next spike was on February 29, which we cannot currently explain because it’s considered an unlucky day in Italy. Software and services saw below-normal traffic soon after the first case was announced, really dropping after the country was locked down. Travel sites also dropped around this time, while media traffic increased after the pandemic declaration.

The results we see in the aggregate chart above are due to various industries canceling each other out.

Tellis_Italy2.png

Canada

Canada had its first case on January 27, but we saw very little change in traffic patterns until the WHO declared a pandemic. Traffic increased at this point, and moved homeward.

Tellis_Canada.png

Mexico

Mexico had its first case on February 28, which showed a spike in traffic. After that, traffic started an upward trend, slowing only for major holidays, International Women’s Day (there was a women’s strike in Mexico on this day), Benito Juárez’s birth memorial, and Holy Week, particularly Good Friday.

While there was no formal lockdown, traffic shifted from mobile to home after the pandemic declaration.

 Tellis_Mexico.png

United States of America

We see very little deviation from the norm when the first U.S. case was announced and when travel restrictions were announced; however, traffic drops on Super Bowl Sunday, when most people were either watching TV or streaming video. The first real COVID-related traffic increase happened around the time the country went into lockdown.

Tellis_USA.png

Looking at the U.S. in more detail, we see that almost half of U.S. traffic was online shopping, and this showed an uptick after the U.S. locked down. Media sites showed an uptick after the WHO declared COVID-19 a pandemic, with additional spikes starting from March 26-29 as the U.S. moved to #1 in the number of confirmed cases, and again on April 3 when the CDC recommended wearing face masks in public.

We also see wildly changing patterns in grocery and pharmacy sites. Traffic was below normal between Martin Luther King Jr. Day and Super Bowl Sunday, and then rose sharply after the WHO declared a pandemic, with another spike just before the U.S. locked down.

Tellis_USA2.png

Regions by First Confirmed Report

We see a big spike of first confirmed cases on March 6. Most of these were related to the Grand Princess cruise ship that had 21 cases on board.

All cases in January and early February were individuals who returned from a foreign trip.

The February 28 case in Oregon is interesting because it was the first case in someone who hadn’t traveled abroad, indicating community transmission.

Tellis_graph.png

Ups and Downs

The individual traffic charts across U.S. states showed similar patterns to what we’ve already seen. The highest peak in each state showed some clusters.

Prior to the week of April 13, April 4 was a common peak across 13 states. This appears to be in response to the CDC’s guidelines on April 3 that everyone should wear a mask when going outside. Most states then showed a second bigger spike after April 15 (read on for more details about this).

In the chart below, the size of each bubble represents the number of states that had a peak that day while the vertical axis indicates the magnitude of the peak away from baseline.

Tellis_graph2.png

Verticals by Region

The following heat maps show us how each state’s traffic changed vs. the baseline across the top four verticals (retail, media, professional software and services, and groceries). Red indicates traffic below baseline while blue indicates traffic above baseline.

Retail

Retail traffic had a strange dip (for a Monday) on March 16, but really started to take off after April. This was likely related to online sales. Earth Day sales, spring sales, and Mother’s Day sales all starting soon after Easter.

Tellis_Retail.png

Media

Media took off on March 12 (pandemic declaration) across all states. Traffic was consistently high after this date, so weekend traffic appeared to be abnormally higher than baseline. The spike in media traffic on January 27 across all 50 states and D.C. coincided with the helicopter crash that killed Kobe Bryant.

Tellis_Media.png

Professional Software and Services

D.C. was the only region that saw an increase in hits to professional software and services.

Tellis_ProfSvcs.png

Groceries and Pharmacy

Visits to online groceries and pharmacies took off after the pandemic declaration except for California, Hawaii, New York, Oregon, and Washington, which picked up around February 28. California, Indiana, Michigan, and the tri-state area of Connecticut, New Jersey, and New York showed the largest move to online grocery and pharmacy shopping.

Tellis_Groceries.png

Americans Working from Home

Between March 13 and 20, Americans across the U.S. shifted their browsing patterns from the office to home or mobile. The Dakotas were the only states where traffic moved predominantly to mobile devices.

We see 15 states moved to a “predominantly at home” state on March 13 and 14, while the rest followed from March 17 onward. While not all states issued formal stay-at-home orders, in several states we see that the largest counties issued their own stay-at-home or shelter-in-place orders, which had largely the same effect.

What’s interesting to note is that in almost every state, traffic moved away from the office before any stay-at-home order was issued — in some cases, as much as two weeks earlier.

This table is ordered by date of lockdown announcement. Blue bars indicate traffic moving homeward before an official stay-at-home order or recommendation while red bars indicate the opposite.

Tellis_WFH.png

Conclusion

We saw an increase in traffic across the board as different COVID-19 related events unfolded. However, the most notable effect we saw was the move of traffic from office to home and mobile, an increase in traffic to online grocery and pharmacy sites, and a temporary increase in traffic to sites that provide software and services to make working from home easier. In some parts of the world, people checked on their finances when major COVID-19 related events happened (first case, quarantine, etc.).

We saw an increase in traffic to media sites whenever various announcements were made, regardless of whether these were COVID-19 related or not, and an increase in traffic to retail sites just before or on major gifting days like Three Kings Day and Valentine’s Day.

In the U.S., the CDC recommendation for wearing face masks led to a surge in people either trying to buy face masks, or looking at articles on how to make their own. Toward the latter half of April, we saw behavior moving back to somewhat “normal” patterns, albeit with higher volume, with many online shoppers taking advantage of spring sales.

 

Appendix

See Also

Future Work

It would be interesting to next study the performance impact of traffic moving toward slower home-based connections. Will that result in slower overall performance, or will consumers start investing more in faster home internet connections? We could also look at which news topics were most popular on different dates.

References

Methodology

  1. We look only at page view data in mPulse, not SPA or XHRs.
  2. We do not consider streaming media traffic.
  3. We look at traffic from December 31, 2019, to April 23, 2020.
  4. We calculate traffic baseline separately for each day of the week by using the arithmetic mean of traffic for that day for six weeks ranging from the second to seventh weeks of 2020.
  5. We split traffic into the categories of home, office, mobile, cloud, or school, based on the following criteria:
    • If Akamai classifies their internet connection as “hosted” or we find the term “Cloud” in the ISP or organization, then we treat the source location as a network Cloud. We exclude this data from the report. For most regions, this is less than 0.01%.
    • If MaxMind classifies their internet connection as “Corporate,” then we treat the source location as Office. It’s possible that many people using a full VPN from home will fall into this category.
    • If MaxMind classifies their internet connection as “Cellular,” or Akamai classifies it as “mobile,” then we treat the source location as Mobile.
    • If the term “University” or “College” shows up in the ISP or organization name, we treat the source location as School.
    • If the organization that owns the IP address is the same as the ISP or if one of them is a substring of the other, or they sound similar, then we treat the source location as Home (i.e., directly connected to ISP).
    • If both the organization and ISP are null, we mark the location as Unknown. There is a very small percentage of traffic in this category (less than 0.01%).
    • If none of the above is true, we mark the location as Maybe Office, although a cursory verification shows that in some parts of the world, this may be Home.
  6. For daily traffic, we compare it to the calculated baseline for that day (i.e., Monday-Sunday) using the formula:
    Tellis_FormulaThis gives us a number that hovers around the x axis. Points above the x axis show an increase in traffic while points below show a decrease in traffic.
  7. All timestamps are normalized for the local time zone where the traffic came from. For large countries with multiple time zones (e.g., the U.S.), we chose a time zone with a large population. For the U.S., this was America/New York, which covers all the eastern states and then some.
  8. We include major regional events as annotations in the chart, as in many cases these also have an impact on web traffic.
  9. We do not show global aggregate numbers, as the baseline traffic from the U.S. is significantly larger than many other nations, and overall results will be skewed toward the U.S.
  10. For this study, D.C. is treated the same as the 50 U.S. states, so we effectively count 51 states.

 

*** This is a Security Bloggers Network syndicated blog from The Akamai Blog authored by Philip Tellis. Read the original post at: http://feedproxy.google.com/~r/TheAkamaiBlog/~3/kmhUcuwSDvA/analyzing-the-effects-of-covid-19-on-mpulse-traffic.html

DevSecOps Poll

Step 1 of 6

What is the biggest roadblock implementing DevSecOps practices?