Features and weights: how computers learn

houses-skinyWe want to come up with a way to predict how much a house costs. Let’s use three measurements in our prediction: number of bedrooms, number of bathrooms, and number of floors. Here’s a simple prediction: the cost of a house is equal to the number of bedrooms + the number of bathrooms + the number of floors, all multiplied by $100,000. So we would predict a 3 bed, 2 bath bungalow to cost (3+2+1)*$100,000 or $600,000. Seems reasonable. Another way of writing this prediction would be to distribute the $100,000:

3bed*$100,000 + 2bath*$100,000 + 1floor*$100,000 = $600,000

But I think the number of bedrooms plays a bigger factor in the cost of a house, and the number of floors a smaller factor. I might revise the prediction:

3bed*$150,000 + 2bath*$100,000 + 1floor*$50,000 = $700,000

So we get a slightly different prediction. We can write out these predictions in a generic form:

fbed*wbed + fbath*wbath + ffloor*wfloor = cost

The f’s stand for feature. A feature is just something we can measure to use in the prediction. The w’s stand for weight. A weight is just the value we multiply the feature by. We can call this equation our house pricing model.

In the last example I just guessed what some good weights might be. But if we had some actual examples of what types of houses were selling for what prices, we could let a computer solve for the weights that make our prediction the most accurate. Let’s say there are some existing houses on the market like this:

2bed, 1.5bath, 1floor for $430,000.
4bed, 3bath, 2floor for $980,000.
3bed, 2bath, 1.5floor for $660,000.

So then we can write these equations (we don’t know the weights yet):

2bed*wbed + 1.5bath*wbath + 1floor*wfloor = $430,000
4bed*wbed + 3bath*wbath + 2floor*wfloor = $980,000
3bed*wbed + 2bath*wbath + 1.5floor*wfloor = $660,000

Now we can get a computer to tell us what values of wbed, wbath, and wfloor make the left hand side of these equations as close as possible to the right hand side. In this case, the computer spits out wbed = 54,400, wbath = 228,000 , and wfloor = 27,200 as the weights (check out this short code snippet to see how easy it is to get a computer to do this; you don’t have to know how to code to be able to follow along). Then we can use those values in our general prediction equation to predict the cost of any house as long as we know the three features of that house (number of bedrooms, bathrooms, and floors). Let’s predict the price of our 3 bed, 2 bath bungalow. The equation is:

3bed*54,400 + 2bath*228,000 + 1floor*27,200, which comes to $646,400. There you have it!

This is machine learning. A computer learns how to predict something (the cost of a house) based on the data we give it (the features and costs of the three houses on the market).

If you can think of a prediction problem in terms of quantifiable measures (features), and if you can supply some examples to a computer (like the three houses on the market), it’ll figure out the weights for you. Then you can use those weights to make predictions.

Want to predict if a skin lesion is cancerous?

width_of_lesion*wwidth + hight_of_lesion*whight + depth_of_lesion*wdepth = 1 if cancerous, 0 if not

Want to predict Google’s stock price for tomorrow?

price_of_GOOG_today*wtoday_price + avg_price_of_GOOG_last_month*wmonth_avg + NASDAQ_avg_today*wnasdaq_avg_today = tomorrow’s stock price

Want to predict if that’s a dog in that picture?

pixel_one_brightness_level*wpixel_one + pixel_two_brightness_level*wpixel_two + pixel_three_brightness_level*wpixel_three + … and so on = 1 if a dog, 0 if not

These are kind of toy examples, but not really. If you had enough data to give the computer to learn off of, you might get some decent predictions. You can probably come up with better features to use for these predictions as well.

Machine learning is going to be a very important part of our future. Already it is being used to predict what people are saying (Siri/Google Now), predict what objects are in pictures (Google/Facebook facial recognition), and predict what cars and pedestrians are doing (Google’s self-driving cars). These are the types of problems we usually rely on our brains to solve, but computers are getting much better at them: they can use more complicated models, they are faster, and they’re not biased.

Humans are going to need to figure out what features to supply our computers so that they can listen for us, see for us, drive us around, and in general, make our lives easier. To prepare for this, it will be good for us to start looking at prediction problems in everyday life the way our machine learning algorithms see them: in terms of features and weights.

PS There’s a lot more to machine learning than what I described here. These were examples of supervised learning only, and I simplified things: there are more complex models (we just used a linear one here – and not with best practices either), there are usually many more features, and usually many more examples with which to train the model. But I think it’s a good primer.

Leave a Reply

Your email address will not be published.