Some people have asked about what’s behind morecloth.com, what I’m trying to do here and how the software works.



Maybe I should start by saying what I’m not trying to do. I’m not trying to:

(1) Come up with come up anything definitive – any model way of sorting or presenting fabric. There’s plenty of room for other approaches and other websites built around the wealth of fabric at Etsy and elsewhere.

(2) Turn morecloth into a for-profit undertaking. This is a side project for me, something I started impulsively when I learned, a few months ago, that Etsy had created an API (application programming interface), i.e. an easy way for programmers to get at the data on Etsy’s site.

(3) Create a mysterious “black box” situation (see picture above). The software behind this site is pretty simple. I’m not publishing the source code – at least not yet – because it’s in flux and a bit of a mess right now. But read on for an explanation of how it works.

The morecloth site is static: just plain xhtml pages that get regenerated and replaced every week or two. I host them on a low-cost shared server and generate them offline on my home computer, a desktop machine running GNU/Linux (Debian Lenny) that I got for $98.75 on ebay, shipping included. I do know about Moore’s law and falling technology prices, but it still kind of amazes me that this project is so affordable, that the economic barriers are so low.

To create initial versions of the morecloth pages I use a bunch of Python scripts, some of which invoke the ImageMagick image processing library. I then review these pages and edit them into final versions using more Python and ImageMagick and a touch of JavaScript. Here’s the procedure:

(1) Download fabric listings using the Etsy API. (How to query Etsy is explained at developer.etsy.com.)

(2) Extract Etsy listing IDs and image names from the downloaded data using regular expressions.

(3) Fetch images from Etsy and gather color data from them. I don’t save the downloaded images; I just use a Python script to grab them off the Etsy site and process them on the fly with ImageMagick. Invoking Imagemagick from the commandline (possible from within Python, thanks to Python’s subprocess module), I quantize the images (i.e. reduce colors in them to a manageable number), create histograms from the reduced-color images, and store the histogram data with the corresponding Etsy listing ID and image name. This may be a slow, clunky way of doing things – there are more direct ways to work with ImageMagic using Perl or C – but speed isn’t a big consideration, since all the image processing takes place offline.

(4) Group listings according to color. There’s no end of ways to do this. But first it’s important to reduce the number of duplicate fabrics. Viewers probably don’t want to see a page with a bunch of identical or nearly identical pictures. Fortunately, most duplicates can be eliminated without visual inspection, simply by having a script compare the histogram data.

Once the listings are thinned down to eliminate most duplicates, functions can be applied that map the listings to numerical values. One such function, for example, looks at the average color of a fabric’s image and returns a number representing the lightness or darkness of that average color. Given a population of fabric listings and numerical a function defined on those listings, the population can be divvied up into groups by various statistical procedures – many available through the “stats” module for Python. I’ve barely had a chance to play with the possibilities, but it’s clear that there are many ways to represent a population as a collection of non-overlapping sets and that, given several such representations, sets from one can be intersected with sets from another to capture a variety of color-related properties of interest in a population of fabrics.

(5) Clean up the color groups. Once a group is assembled, it may still have some problems. I still have to lay eyes on the actual images and discard anything that detracts from the group as a whole. I try to catch images that include highly visible watermarks or non-fabric objects (e.g., tape measures or coins used to indicate scale). I also try to spot any duplicate fabrics still hanging around or anything that just looks out of place – any fabric disconcertingly different from its neighbors. Some of these visual judgments may be susceptible to automation, but probably not all.

To do this clean-up operation I use Python to create a temporary web page (for local viewing only) for each proposed color group. This page includes some JavaScript and a form with a text box: clicking on a fabric image calls a JavaScript function that removes the image from the page and adds the corresponding Etsy listing ID to the text box. Later, when the form is submitted, a Python cgi script writes to disk IDs of all the listings deemed inappropriate for the group. After all the color groups have been inspected, another Python script creates the final pages that appear on morecloth, along with the bands of colors seen on the index page.

There are a few refinements and add-ons to this process: Python scripts for blending, splitting, and combining color groups. But that’s the basic idea behind morecloth.

Oops! I forgot to say why I’ve taken on this project. Well, the fabrics are pretty. The statistical stuff is potentially interesting. I feel generally friendly toward Etsy sellers. And – who knows? – maybe this will generate some interest in a separate for-profit textile project I’m launching later this year. (More on that later…)