Building a Smarter Specification Platform

An Insight into Our Data Process

Today we wanted to share a little bit of inside knowledge about our processes at SpecifiedBy, specifically in terms of what we are doing with product data.

It’s been an exciting time for us here over the past 6 months or so; we’ve raised seed investment, we’ve been steadily building our team, we’re working on a slick new design for the site and brand and we’ll be launching a few ‘next level’ features over the next couple of months.

But all of this is built upon the idea of having access to good, useable building product data. Unfortunately, there’s been no easy way to get this, but we’ve managed to develop a very scalable process which will allow us to grow or database of useful building product information very quickly and develop lots of exciting new features.

This process is also a simple demonstration of why we’re not a product directory. Directories are filled with static, ‘dumb’ information, which allows for browsing by categories or classifications and provides keyword search, but doesn't allow a specifier to query the information in any intelligent way.

We’re building a Smarter Specification platform, consisting of a range of tools to help specifiers make better, more informed product choices, more efficiently, and ultimately providing us all with a more sustainable built environment.

Here's a look at how we get the data to make this possible...

Step 1 – Data Gathering

When a manufacturer is added to SpecifiedBy, the first step in the process sees our content team gather as much information about the manufacturers’ products as is available through their websites, direct from the manufacturer and third-party websites.

This data includes:

• Images
• Product names & descriptions
• Data sheets & other technical documents
• Sample specifications
• CAD files
• BIM objects
• Operations & maintenance guidance
• Brochures
• Certifications & standards

Everything on SpecifiedBy has its place, making it easy for specifiers to find what they need, whether it is a datasheet or a BIM object. We have created an internal process for gathering and organising this ‘dumb data’ that allows us to carry out the categorising and structuring process very efficiently.

All of this information is uploaded to SpecifiedBy to create product pages for each of the manufacturers’ products and all of this content is made available to specifiers to access and download in this original format.

Step 2 – Extracting the Key Data

Our next step is to extract the key information for each product from all of this raw data, in order to present it in a more standardised, smarter format. Importantly, structuring the data in this way allows us to build some clever software on top of it.

As an example, for a window product, we might look to pull out key properties such as:

• Dimensions
• U-value
• Fire rating
• Acoustic Rating
• Air Tightness
• Material(s)
• Finish(es)

For this part of the process, our technical team creates a specially designed excel file for each manufacturer which lists each of their products by name and provides an input for each property that needs to be entered for each product.

For each product, we have a list of around 260 properties to choose from which cover the following areas:

• Technical properties
• Commercial information
• Materials & Finishes
• Sustainability criteria & properties
• Operations & Maintenance information
• Size & Weight properties
• Physical properties

This information is then attached to each products page on SpecifiedBy, creating a Specification Table.

By having the data structured in these tables, where each property is linked to a database, it will be very easy to use any standard templates (such as the work being carried out by BIM4M2 on Product Data Templates) that become available to upload data to our system.

Getting the data converted into this format does take quite a bit of manual work, but the data already exists, manufacturers have provided datasheets forever, so we're not having to create any new information, just a better way of organising and presenting it.

Step 3 – Bulk uploading & Manufacturer tools

The final part of the process is to actually get all of this structured data onto SpecifiedBy.

To do this we’ve built two custom software tools. The first is an internal tool (above) which allows our content and technical teams to upload all of this data in large quantities. This allows us to crack on and do this on behalf of manufacturers on SpecifiedBy.

The second tool (below) is the content management user interface that allows manufacturers to add these properties manually themselves. It’s a very straightforward system where you can search and select from our database of predefined properties and then fill in the relevant values.

We’ve also added the functionality to create product ‘variations’. For example, if a window manufacturer offers a double glazed and triple glazed version of the same product, this is obviously going to have an impact on the products performance properties. Our system allows manufacturers to quickly create a variation based on the template they have just created for the ‘standard’ version of the product.

Putting the Data to Use

Having product data in this structured, standardised and well-designed format is very nice and makes a specifiers life a bit easier, but we plan to really make this data work!

Below are just a few of the uses we are currently working on.

Data Extractor

The first simple use will be to allow specifiers to extract the data in various formats. This could range from a simple CSV file, to standardised PDTs (when they become available), and even to BIM compliant formats.

Comparisons

Having this standardised formatting has allowed us to work on developing a really slick product comparison feature, where you can essentially view 2 or more products side by side and see how the compare based on a similar set of specification criteria.

Advanced Search

This is our big one, which we are currently testing. You can read a bit more about this is a little review Paul Wilkinson wrote on his blog.

Because we have created a predefined list of product properties (which is still growing) that manufacturers use to share the data about their products, our property tables are much more than just nice and user friendly.

We are creating a live, searchable database of building product information and we’re building a search engine on top of that.

This means a specifier can search, windows for example, by U-value, fire rating, air tightness, dimensions or any other property that has been added. And it’s not just a case of putting in some values and hoping for the best. The properties which can be used to filter results are only those that are attached the product category you are searching within and the values you can enter are pulled straight from the database.

Using the window example again, if we had 50 window products and the lowest U-value was 0.5 W/m2k and the highest was 2.0 W/m2k, you would have a slider to set a value or range between those two values.

Then, as new filters are applied the search results are updated in real-time.

Possible API

One other idea we are looking at is creating an API to allow others to access this data and perhaps build their own applications and tools or incorporate it into existing tools. If you are interested in this, let us know in the comments below.

Conclusion

These processes and features are something we’ve been working on for quite a while and are learning a lot from all of the time. It’s one of these things where if you sit down and think about exactly how it will work and what is required to achieve it, it becomes overwhelming and you probably never start.

We decided to just start and figure it out as we go along. It has taken a little while to get everything working just how we want it, but we now have structured data for well over 2,000 products and will soon be launching our advanced search functionality on the back of amazing feedback from those who have trialled it so far.

If you are interested in trying our advanced search or have any questiona about our process, leave a comment below or get in touch directly on darren@specifiedby.com

09549