How much of the web runs on renewables today?

As part of the work I’m doing on the Planet Friendly Web, I’m trying to get access to data that I can base the guide on. In some cases this involves creating datasets from existing data. Here I share some findings from a dataset I generated along the way.

For example, to get a figure on how much of the web runs on renewable power, I started with a dataset of the top 1 million domains by traffic from Alexa.com, then run the list against the Green Web Foundation’s own API, which maintains a list of which domains run on renewable power.

To do this, involves making something like 100k API requests, so I created a screenscraper to carry out the job, and take care of retries, failed requests and so on. You can see it here on github.

I’ve uploaded the dataset created to datbase, partly as an experiment in making it available in a decentralised way, but also partly try out the workflow for publishing data.

So, now we have some data, let’s see what we can do with it, right?

Doing some analysis and some interesting findings

I have an earlier exploration of the data in a notebook on github, but when working with this data, I ‘m bit embarrassed to say I forgot how to use the Dataframe filters to slice the data quickly.

So instead, I’ve used Open Refine. You could probably store this in a Google spreadsheet too, as 100k rows is big, not but THAT big.

Anyway, what do we see?

There’s a few interesting findings just from faceting data like below in Openrefine,  and sorting by count along a few dimensions:

Screen Shot 2018-05-15 at 12.07.36.png

If you’re not familiar with OpenRefine, I’ll summarise what’s visible in this view:

  • Youtube.com is now more popular than google.com. Who knew?
  • The top three websites in the world run on renewable power. Huzzah!
  • Based on the greenweb foundation’s data, around 7% of the web the most popular domains on the net run on renewable power.
  • Hetzner AG, a German hosting company hosts more domains running on green power than Google does.
  • Amazon doesn’t appear here at all as a green provider.

After a slow start, I understood Amazon to be a HUGE player here, and while they have a nice shiny page showing off their windfarms and how much renewable power they use , they also run a load of their servers on coal. That they don’t appear may be an artefact of the Green Web Foundation going by an organisation’s entire power mix, to decide whether a company is running on green power or not.

I think need to check with Rene at the Green Web Foundation to see.

Fancy playing too? Come hang out on slack

This shows some pretty superficial analysis, but there’s already some interesting nuggets here.

If working with this data sounds interesting to you, let me know in the comments – I’m looking for collaborators on the Planet Friendly Web Guide.

Alternatively, come hang out in the sustainableux.com slack channel, where there’s a nice little community growing around sustainable web design.

If you prefer email

It turns out there’s a W3C Sustainable web design group. Here’s my post to the mailing list, if you’d prefer to communicate there via email.

 

 

3 thoughts on “How much of the web runs on renewables today?

  1. Regarding Amazon, we’ve got some of our servers there in the Canada Zone, which is one of their 5 carbon-neutral data centers. I think their list of IP addresses is here https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html#subscribe-notifications

    Unfortunately it takes time to process and maintain information like this in the Green Web Foundation databases. Not sure that they have the resources to parse this effectively. This really is something that should be supported by some of the internet giants.

    Like

  2. For what it’s worth, I really agree with you, Mike.

    I don’t fully understand how the Green Web Foundation does the look ups, but I do know you’re able to register as a provider at the link below, and how the site is green.

    https://admin.thegreenwebfoundation.org/

    I think there might be some clever look-up stuff going on, to follow a supply chain of resellers, so resellers of hetzner are listed as green, for example,.

    I’m not sure it the register resolves to say… individual AWS regions though.

    Anyway, for what it’s worth, I’ve put in a funding bid for a Mozilla Open Web Felllowship to basically work on this full time for the next year. Building this stuff out is exactly what I want to be able to focus on, so there’s a chance to bring this more mainstream.

    Like

    1. That’s great. Thanks for the link to the admin pages for the Green Web Foundation. I set up something for our hosting. Still wish there was an easier way to do this.

      Like

Comments are closed.