We’ve been exploring a few ways in which we could use carbon.txt within our own platform for provider verification. In this post, Fershad and Tim introduce one such use case – linking domains to verified providers in order to deliver more accurate results from our Green Web Check API.
Our Green Web Check API receives around 7 million requests per day from folks checking if web domains are hosted on verified hosting providers in our Green Web Dataset. The API powers our own Green Web Checker, as well as other platforms like Ecograder, Website Carbon, WebPageTest, and many more.
Historically, we’ve returned green results through associating domains to verified providers using IP address matching or Autonomous System Numbers (ASNs) resolution. This works in most cases, but not all. When these approaches fail, Green Web Checks either return a “no evidence found” result for a site that should be green, or show the wrong provider organisation on the results page and on the green web hosting badge.
There are two scenarios where these failures most commonly occur for a website being checked:
- CDNs – we are unable to identify the true host IP address for the domain due to it being blocked by a content delivery network (CDN), or,
- Resellers – the hosting provider is unable to share IP address ranges with us, which is often the case with providers that are resellers.
These are the cause of a lot of our inbound support queries, and a frustration for hosting providers and their customers. For a long time, we’ve been looking for ways to move away from IP matching which could allow us to address these scenarios in a graceful way within our checks.
Introducing domain linking with carbon.txt
If you’ve been following our work for the last couple of years, you’ve probably seen us reference a project called carbon.txt a few times. Carbon.txt allows any website to make their sustainability claims easier to discover and use on the web. It works by hosting a carbon.txt file containing structured, machine-readable links to sustainability data in a known location – for example at yourdomain.com/carbon.txt.
For the last few months, we’ve been working on a way to allow hosting providers that are verified in our dataset to use carbon.txt as a means to claim ownership of their own domain, and to link customer domains to their listing in our dataset. This eliminates the need for IP/ASN based lookups when checking if a domain is hosted by a verified green provider.
How it works
These instructions are for hosting providers who are currently verified in our dataset.
Linking domains to a hosting provider using carbon.txt is a multi-step process. Put in the most simplest way possible, the following steps need to happen:
- A hosting provider creates a carbon.txt file for their organisation.
- They upload that carbon.txt file to their primary domain.
- They login to our Provider Portal and verify that the carbon.txt file exists at that domain.
These steps establish the provider’s ownership of that domain. In order for them to then link other domains (e.g. customer websites) to their provider listing they can then use HTTP Headers or DNS TXT records which we provide in our Provider Portal to establish that connection. When our Green Web Check runs, it will look for these signals first to determine the host of a website, before falling back to IP matching.
These steps are a simplified explanation of the process. We have a detailed step-by-step guide for you to follow in the FAQ How can verified hosting providers use carbon.txt to improve the accuracy of Green Web Check results?
I’m a customer of a verified provider. How can I make sure my domain is linked to them?
If you are a customer of a verified provider and want to ensure that your site’s host always appears correctly in the Green Web Check, then you will need to ask your provider to first follow the steps above to setup and verify a carbon.txt file for their domain.
After that’s done, they can share the generated DNS TXT record for their listing with you. You can add this to the DNS records for your site, and then visit our Green Web Check to test your site.
Note: If you do not see the expected results, click the Refresh check link on the results page to ensure that the incorrect result is wiped from our caches.
Does this solve all the cases where a Green Web Check might show the wrong provider?
No.
This is one solution for a few specific cases where the Green Web Check might return incorrect results. Through our user research phase, we learnt that it is not always possible for providers to modify HTTP Headers or set DNS records for the sites they manage for customers. We are still looking for other methods which would allow providers to link domains to their listing to help improve the accuracy of the results our Green Web Check API delivers.
We’d love your feedback
This functionality is very new in our platform. The process, in its current form, is not as smooth as we’d like it to be and we are looking at ways to streamline it. Generating carbon.txt files automatically for providers, based on the public evidence they have already supplied is one feature we’re currently working on that we hope will significantly ease this process.
We hope that this new functionality can be helpful to verified providers in our dataset, and their customers. If you need support using carbon.txt, have other questions, or want to share some feedback then please do reach out to us using our support form.
