How to Use Google Prediction API to Estimate a Sale Price for Your Home

This blog post is based on Martin Omander's workshop at Silicon Valley Code Camp, which I attended on Saturday, October 8:

Store your data, predict the future

Speaker: Martin Omander
Level: Intermediate | Room: 3525 | 9:45 AM Saturday

• Make your smart applications smarter with Google's Prediction API. Take advantage of Google's machine learning algorithms to make recommendations, analyze Twitter, detect spam, classify documents, identify languages and more.
• Store your applications' data securely and efficiently in Google's data centers with the Storage API.

The real estate home price data file mentioned below was compiled by Martin from Redfin data for the Rex Manor neighborhood of Mountain View, California.

If you've ever wanted to buy a house or sell your home, and would like a reasonable estimate for what the ultimate sale price will be in this market, you can get a real estate agent's take, or you can use Google Prediction API (henceforth Prediction) to give you another estimated price, given available market data. The idea is, you grab a number of sale price records for the ZIP code you want to buy or sell in, feed the data into Prediction, and Prediction will spit out a number based on the data you fed it. Besides spitting out scalar values, Prediction also does classification. To get your feet wet, here is a beginner tutorial that figures out, in an automated way, whether some text you pass in is English, French or Spanish:

http://code.google.com/apis/predict/docs/hello_world.html

Assuming you were able to successfully use Google's Prediction API, in conjunction with Google Storage and Google's APIs Explorer Tool, to classify samples of text that you pass in, the next step is to predict a sale price for your home.

1. Download Martin's houses.csv data here
2. Using the same steps in the tutorial above, upload the houses.csv data into Google Storage, and then use the data to train Prediction
3. After training is complete, we want to get a predicted price. To do this, we want to pass in the values for a house that we want to buy or sell. Let's say we're trying to sell a 3 bedroom, 3 bathroom, 1800 square foot, built in 1960, single family residence. I found that when I tried to pass in the following as a value for the csvInstance key, I got an error:

   3 3 1800 1960 house

   I then tried the same thing with commas, but that didn't work either. What I found finally worked is, in the APIs Explorer tool, when specifying the value of the csvInstance, you need to count how many field values you are putting in, let's call it n (in this case, n equals 5), then provision that same number by clicking on "Add" n times, and manually insert the values in order, like so:

    {
      "input": {
        "csvInstance": [
          "3",
          "3",
          "1800",
          "1960",
          "house"
        ]
      }
    }

Caveat for those who want to read in or import additional data sets: if you want Prediction to spit out a number, i.e., a scalar value, one thing to remember is that the column associated with the scalar value that you want a prediction on (in this case, home sale price) should be the first column inside the data file you feed into Prediction. Martin said that when he was putting together the home sale price data, he had to manually copy and paste from Redfin into his spreadsheet program, then move the column for the home sale price to be the first column, and then exported the spreadsheet as a csv, i.e., comma separated values, file.

Heather, Rachel, Michelle, Casey, Ann or Tiffany from Account Services or Cardholder Services

Update: David Lazarus at the Los Angeles Times has written two recent articles about this issue, with some news and interesting ideas:

• http://www.latimes.com/business/la-fi-lazarus-20140107,0,1818707.column
• http://www.latimes.com/business/la-fi-lazarus-20140114,0,1753169.column

--

From phonespam.blogspot.com:

Hello. This is Heather at account services. And we're calling in reference to your current credit card account. There's no problems currently with your account. It is urgent however that you contact us concerning your eligibility for lowering your interest rate. Your eligibility expires shortly. So please consider this your final notice. Please press one now on your phone to speak with a live operator and lower your interest rate. Or press two to discontinue further notices. Thank you and have a great day.

I've been roused from sleep by these unsolicited, automated phone calls, all having to do with card member services, or account services, relating to your credit card debt, lowering your interest rates across all lines of credit, repairing your credit, financial services counseling, or debt negotiation. Fraudsters, con men, whatever you want to call them, ultimately, they're looking to get your money. These robocalls piss me off, and if you want to do something more than just hang up, you can get even. I've pressed '1' to get connected to a live agent. The people who get paid by the telephone marketing scammers have learned to be wary of giving out too much information, especially to an angry caller, because they know if too many people complain, the money will stop coming in. However, if you have a little time, you can employ the following strategy to make yourself some money fighting crime, and help bring down these organizations who try to scam people. The next time you get one of these calls, here's what you should do:

1. Pretend that you are interested. The first two things the representative may ask you are your name and how much credit card debt you have. You can use a variant of your name, or create a new name for yourself (in order to not arouse suspicion, try to make your new name similar to your real name, so that it's easier to remember, such as starting with the same sound or letter. If your name is Mike, then Mark or Malcolm. If John, Jerry or Jesus). Tell the representative you have credit card debt of $16,000 (which is about the national average), so that he or she will get excited about possibly doing a balance transfer with you over the phone, or getting your credit card number so he or she can bill your credit card for "services".
2. Do not give out any of your personal information, such as your credit card number. Inform the representative that you have some discomfort doing business over the phone, and you would like to do some research before you go forward. Tell the telemarketer, "I have a fear of scams, and would like to make sure this is a legitimate business", and "I would like to take advantage of that 4 to 8% rate, but before I do, can you tell me what is your company's name and physical location?" To lull the scammer into thinking you are sincere, I might go so far as to give out my ZIP code and the toll-free number of my credit card company (Craig in the comments below prudently advises retaining your expired credit card for just this occasion, or perhaps switching around the last four digits of an old credit card), but when you are prompted for your credit card number, stall them until they give you some kind of traceable information, such as an official company name, their callback phone number, and of course their physical location.
3. Ask for, and record, the company's name, the name of the person you are talking to, get a call-back number and an address, and note the time and the date of the call.
4. Report the call and any of the information you have to the authorities.

Enough people who complained were finally able to identify and bring down three telephone scammer companies, including Mutual Consolidated Services in Tacoma, Washington, and multiple companies in Florida:

http://blogs2.startribune.com/blogs/whistleblower/2009/06/26/hunt-for-rachel-from-cardholder-services-takes-me-to-a-company-in-florida-but-she-wasnt-there/
http://www.kirotv.com/money/18910460/detail.html
http://www.ftc.gov/opa/2010/07/mutualconsol.shtm
http://www.ftc.gov/os/caselist/0923190/index.shtm

These are three URLs I've used to lodge complaints about the calls:

https://esupport.fcc.gov/ccmsforms/form1088.action
https://complaints.donotcall.gov/complaint/complaintcheck.aspx
https://www.ftccomplaintassistant.gov/

Depending on your state, you can possibly make yourself some extra cash by suing the telemarketing company in small claims court if you are on the "do not call" registry:According to the California Attorney General:

In order to file a complaint, you must know either the name or the phone number of the company that called you. Our office cannot trace the phone call you received and obtain this information on your behalf. You also must provide the date that the company called you and your registered phone number. You may provide your name and address, but it's not required for you to submit a complaint.

According to the FCC:

Some states permit you to file law suits in state court against persons or entities violating the do-not-call rules. You may be awarded $500 in damages or actual monetary loss, whichever is greater. The amount may be tripled if you are able to show that the caller violated the rules willfully and knowingly. Filing a complaint with the FCC does not prevent you from also bringing a suit in state court.
States also can bring a civil law suit against any person or entity that engages in a pattern or practice of violating the TCPA or FCC rules. You can contact your state Attorney General’s office or consumer protection agency with particular complaints, or to encourage such suits.

Happy hunting!

Update: Some new variants of this automated robocall telephone scam start out with:

1. This is Visa Mastercard member services. Congratulations, you now qualify for a lower interest rate on all your credit card accounts. Press the number 2 to speak to a customer service agent, or press 3 to decline this offer.
2. "Hi, this is Tiffany with Account Services, calling in reference..."
3. "We have closed the file on your annual credit card review, and you should have received mail..."
4. "This is an important message from cardmember services..."
5. "Hi, this is Anne with Account Services with some good news!"

how to file a stay of judgment when you've been found guilty of a traffic violation

Update: After a year and a half battle, I won my case on appeal in Superior Court, as detailed in this latest blog entry: Fight your red light ticket, and win!

Let's say you got a traffic ticket, a red light camera photo ticket, and you decide to fight it. You do your due diligence, poring over the resources available at Fight Your Ticket & Win in California and highwayrobbery.net, and the first two times you appear for arraignment, in front of the same judge, you request that a motion hearing be scheduled so you can compel the city of Emeryville to provide discovery, in the form of the high resolution photos and video captured by the Redflex cameras. Both times, the judge either ignores or refuses your request, and continues the matter, i.e., gives you an additional 30 to 60 days before you have to reappear during arraignment in order to enter a plea. The third time, the judge is exasperated that you persist in your request for a motion hearing, and he demands that you enter a plea. You plead Not Guilty, and the judge, perhaps cognizant that you won't skip town, since you showed up all 3 times like you were supposed to, does not order you to put up bail (i.e., the fine of $446 in order to clear the citation), but instead, does you the favor of releasing you on 'O' 'R', i.e., on your own recognizance (most people try to ask the judge for no bail, and are usually denied, although the judge may grant you extra time to put up the bail). You have a trial, where you are found guilty, and the judge orders you to pay the fine, and gives you one month to do so. You may now want to appeal the trial court ruling, because you believe the trial court judge made one or more errors when deciding to rule against you. If you plan to appeal the ruling, you may want to avoid paying the fine until the appeals process is over, hopefully with the appeals court judge ruling in your favor.

The government alleges that, last year, on June 1, 2010, I didn't stop before turning right at the intersection of 40th Street and Horton Street in Emeryville. 17 days later, on June 18, I received a citation in the mail, a notice of traffic violation of California Vehicle Code 21453a, "Failure to Stop at Red Light". Since this case is still on appeal, I have to be wary of what I say, for fear of hurting my case (the government can read, just as well as you or I can). I pled Not Guilty, and appeared in court a total of 8 times, 6 out of 8 in front of the same judge, Commissioner Taylor Culver, to argue my case and lose, then ask for a stay of judgment, and then for a hearing on settlement of the statement on appeal. I spent quite a bit of time reading and writing to get a sense of what my legal foundation is and what obstacles I might face, as I carefully prepared my case. Often, you'll find that when you're arguing a case in traffic court, the system is set up against you, and you will need all the help you can to navigate a process that seems prejudiced in favor of the police, the municipality where the alleged violation took place, and the corporations, such as Redflex, that the cities and the municipalities are in contract with. Eventually, despite all your preparation and strong arguments, the judge may be unresponsive to what you have to say, and very well rule against you (the logic may be, perhaps, that if the trial court were to actually listen to your arguments, then that may only encourage people to fight their tickets, instead of swallowing your pride and forcing yourself to submit to what I believe is an aggressive government tactic to squeeze money out of motorists). Well, during trial, despite my preparation and what I thought to be strong arguments for why my case should be dismissed, or at the very least why I should be found not guilty, the judge ruled against me, and I was ordered to pay $466 in one month.

You can now do one of two things: pay the fine, and when you win, petition to get your money back, or avoid paying the fine by filing a stay of judgment, asking the court to postpone the sentence while you appeal the trial court's ruling. I opted for the latter, as paying such a large amount for me would be a hardship. At first I asked the trial court to stay the judgment, and not surprisingly, the same trial court judge denied my motion. I then submitted a motion to the appeals court to stay the judgment, the template of which was provided to me by the editor of highwayrobbery.net, but the court sent me a letter saying the motion was denied, without prejudice ('without prejudice' means you are allowed to resubmit the motion to the court). The appeals court's given reason for why my motion was denied had to do with my not following the procedure spelled out by the California Rules of Court, which lays out the rules you must abide by, i.e., the technicalities you must follow, when you want the higher court to consider your motion. In this case, this is the denial letter that I got, copied verbatim:

Petitioner Chen's "Application for Stay of Sentence Pending Resolution Of Appeal To the Appellate Department" is denied, without prejudice. CRC 8.930 et seq. governs proceedings in the appellate division for writs of mandate, certiorari, or prohibition.

I was left scratching my head, wondering what I did wrong. I went to my local law library, where I was able to obtain a copy of the specific passage pointed at by the reference California Rules of Court 8.930. Reading further into the rule, in 8.931, I found the following:

A person who is not represented by an attorney and who petitions the appellate division for a writ under this chapter must file the petition on Petition for Writ (Misdemeanor, Infraction, or Limited Civil Case) (form APP-151). For good cause the court may permit an unrepresented party to file a petition that is not on form APP-151.

Reading it over, I finally figured out I had to file what is known as a writ of mandate. After some googling, particularly for california appeals, writ of mandate, I was led to the proper form, APP-151:

http://www.courts.ca.gov/documents/app151.pdf

I went ahead and filled out the form, resubmitted my motion, crossed my fingers, and hoped for the best. Four weeks later, I got the following letter in the mail, copied verbatim:

Petitioner Chen's Petition for Writ is granted.
IT IS ORDERED that the judgment and payment of the $466.00 fine be stayed pending resolution of Petitioner Chen's appeal to the appellate division.

The best thing about filing for a stay of judgment, and appealing a traffic court ruling, is that a traffic infraction technically falls under the aegis of the criminal justice system, where you are entitled to a free defense. Therefore, all the paperwork described herein, such as the appeal or the writ of mandate, is free to file. Unlike the paperwork you file in civil court, where you do have to pay, sometimes hundreds of dollars per filing, filing the aforementioned paperwork with the appeals court regarding a traffic violation incurs no fees.

Update: After a year and a half battle, I won my case on appeal in Superior Court, as detailed in this latest blog entry: Fight your red light ticket, and win!

Flower Thieves

On a Thursday afternoon of last year, August 19, 2010, I walked outside and noticed that the pretty light-pink flowers in the pictures above had been removed without my permission and were now conspicuously absent. I have these plants growing in my front yard, and somebody had taken the blooms while they were still flower buds, before they had a chance to open up and blossom. Where the flower buds had been, I saw only the headless portion of remaining stalk, the flower bud and long stem missing. I suspected then that someone may have come onto the property, possibly in the dark of night or early morning, and stolen the budding flowers from the bulbous plants growing on the property, and that's why my flower buds were no longer there. Upon further examination, I saw that the top ends of some of the headless stalks had uneven edges, whereas some had clean cut edges, indicating that whoever stole the flower buds from the property used two different methods to remove the desired items, perhaps knife, scissors, or other sharp-bladed object on some, and snapping off a few of the others. The person(s) who did this left one stalk already in bloom, possibly because the buds are of greatest value when they have not bloomed yet. Flower thief? Flower poacher? Blossom bandit? Bloom burglar? I'm not sure, but in researching why someone would take my flowers, I ascertained that the scientific name for the plant in question is Lycoris squamigera, otherwise known as surprise lily, resurrection lily, or naked lady, and the ones pictured are the short-stamen variety. I tried to take a photo of what happened to my plants, but at the time my digital camera was not working correctly. Even though the perpetrator would unlikely ever be caught (I mean, how do you trace stolen flowers when I didn't do anything to mark them in any way) and it would probably be a waste of time, I went online and filed an Oakland police report. Then I stewed in my own righteous anger, mad at whoever would do such a dastardly thing as steal someone else's flowers from their front yard. I counted 11 of my flower buds taken from me.

Upon further investigation, I discovered I am not the only one this has happened to. Many parties, besides me, from people in homes with flowers in their gardens to destinations like the San Francisco Golden Gate Park, have reported flower theft, or flower poaching, or bloom snipping, or blossom burglary. Apparently, flowers can bring you $1 to $5 each, and the unscrupulous florists who buy the ill-gotten goods may not want to ask a lot of questions. Whoever thought your friendly neighborhood floral arrangement specialist and bouquet shopkeeper might be some shady criminal dealing in stolen goods? The police are onto the thefts, though. Thieves and the florists who buy from them have gotten caught with covertly-marked stems, and some plants worth as much as $1000 per have microchips embedded inside to indicate that they belong to someone else.

Canon is awesome

I want to let you all know about some pleasantly surprising, then, rather amazing, customer service I received, from the consumer and business technology company Canon. Back in October 10, 2003, I purchased a Canon Powershot S400 digital camera with $300 my parents had given me for my birthday. I really liked the camera, and got to use it a lot, but then last year, I started noticing some problems. The problem I was having was, whenever I tried to take a picture, I saw something like this (thanks, Deepak Prakash, for the photo):

The image on the LCD monitor on the camera, and in subsequent photographs I took, was smeared, blurred, washed-out, and tinted pinkish-red. Other times, I noted a warped, noisy, false-color image, then the LCD monitor would go dark on me.

I was ready to toss the camera and buy a new one, but in a last bid attempt to save money and salvage what I had, on Friday, August 20, 2010, I called Canon, and from a conversation I had with Canon representative Mike, in Chesapeake, Virginia, he told me the following (and I paraphrase):

This particular camera has a service notice, for an issue with an image sensor, a CCD, charge-coupled device. That sensor can fail. What I need to do is send the camera to Canon, and they will repair it for free. I need to give Mike my information, and get a case created for me. Before I send the camera in, remove the battery, memory card, and strap from the device. Wrap camera in bubble wrap, and put in a cardboard box. Mike will send 2 emails to you, the first has instructions on what to do and where to send it to, along with an evaluation form that I print out and put inside the box, and the 2nd email will come from UPS, with a free shipping label, prepaid and insured. Once the camera is received, it will take 2 to 3 days for the Canon technicians to check in the camera and evaluate it; they will send me an email with a repair order and status, and then 5 to 7 days to repair and return to you. Return shipping is via Fedex, and I will need to sign for it. The advisory for this service notice went out in 2007, but you weren't registered, so you didn't receive the notice

Lo and behold, on Thursday, September 9, 2010, I received a repaired Canon S400 camera, which has been working for me just fine since then. I thought it was worth mentioning that, other than the inconvenience of my not having the camera during the time it was in transit, being repaired, and shipped back to me, the whole repair process cost me nothing, and even the shipping was completely paid for by Canon. I've come to find that the problem I encountered was common to many digital cameras from many brands that contained a particular Sony-made part:

http://www.imaging-resource.com/badccds.html

I've also come to find out that Canon is slowly phasing out the cameras they will repair, according to a timeline. So if you have a Canon Powershot S400, it looks like you may be out of luck:

http://camerarepair.blogspot.com/2007/11/canon-digital-cameras-showing-black.html

How to Add Google +1 to Your Blogger/Blogspot Presence

The Google Plus social network has been fun to play with so far, and I understand that even though Google+ has been growing really fast, it still has a ways to go before it catches up with Facebook and Twitter. Unlike the latter two, one thing that takes getting used to in Google+ is the lack of a system to send a direct or private message to who you're connected with. There is a work-around, where you can publish to your intended recipient's stream: you click on "Share what's new...", type something, but then when it comes time to 'Share', instead of specifying a circle, you type out the name of the person who you're connected to. Your message then gets published to that person's plus.google.com stream. The problem with this method of approximating direct messaging is, as Google+ warns:

Your content may be shared beyond the bounds which you originally intended if your post is reshared or if someone is mentioned on your post. For instance, if someone is mentioned in a comment, they'll be able to see the entire post even if the post wasn't originally shared with them.

So if you mention someone else in the content of the post, and happen to prepend that third person's name with a '+' or '@', then that person can now read your message, which defeats the purpose of a private, direct message.

That said, Google+ is still pretty cool, and one can argue that because there is this limitation of no direct messaging, Google+ is cleaner and leaner, akin to Twitter's self-imposed 140 character limit (which I, and Farhad Manjoo, think has outlived its usefulness, especially for Twitter conversations).

Before Google Plus came out, Google introduced +1, which is their version of the Facebook 'Like' button. In addition to Google's how to add +1 to your blog, for those on blogger/blogspot, here's what I did:

As stated in the Google document above, you need to copy and paste one of the following 3 possible pairs of XML elements into your blog template:

• <script> and <g:plusone> elements
• <script> and <div> elements
• Two <script> elements, the latter of which makes a call to the render function, as explained in the section Javascript API

They all accomplish the same thing, but if you don't know which one to use, I'd go with the first, or the second, one, either of which are easy to copy, as Google provides the boilerplate for you, without the need for any additional changes from you. Our next steps are for where to paste what we copied:

1. In Blogger's tabs console, click on 'Design'
2. Click on 'Edit HTML', and in the section 'Edit Template', click on 'Expand Widget Templates'
3. Paste anywhere inside. I recommend that you provision a section inside the template so you can keep track of all the changes you've made to the default.

Leccion de Español Tres o Spanish Lesson 3

The Yards (1999, directed by James Gray) with Joaquin Phoenix (as Willie Gutierrez) and Robert Montano (as Hector), where they are speaking in Spanish. Mark Wahlberg is silent onlooker.
Notice how Robert's character code-switches, i.e., changes the language he is using, from English to Spanish. In this context, Joaquin's character, who was conversing in the more familiar Spanish, reverts to English, to distance himself emotionally from the person who he was addressing. We later find out why, as Willie Gutierrez and his crew are, and have been, intentionally sabotaging the trains operated by Hector's company, in order to secure lucrative city contracts.



Hey, Gutierrez, quiero hablar contigo.
Yo tengo nada que decir, Hector.
Oye, yo soy tu amigo, chico. Yo se, uh, lo que tu estas haciendo.
<Willie reverts from Spanish to English>
Hey, nunca vas a ser tan blanco como ellos.

Back in February of last year, I was listening to talk radio, and had a chance to hear an interview with Audrey Nelson, author of Code Switching. I took some notes:

Code switching came out of linguistics, and it refers to someone who is good at and has knowledge of two cultures. Good communicators know how to switch gears. When speaking to audiences predominantly of one typical gender role, Audrey will use different styles; with men: statistics, research, be much more direct. With women: stories, relationship talk, very high level of disclosure within a couple of minutes, i.e., talking about your insecurities, fears and doubts. Women are in the business of social maintenance, to take care of people and relationships. Chit chat initiated by women before a meeting serves a purpose: to relax, make people feel freer, less on-edge, feel connected and bonded, be more civil to each other. When men chit chat, they talk about safe topics like work, sports. You spend more time at work than any other organization. If the verbal and nonverbal contradict, human nature is to base decisions on the observation of non-verbal behavior. Women are much better readers of nonverbal cues than men; excellence in social maintenance means excellence in observing people's feelings, relationships, emotions.

Although her book is mainly about how to communicate effectively in the workplace to a mostly male (or female) audience, the take-away message is to know the needs of your audience (which is what I was referring to in Why You Should Learn Black English). Ms. Nelson went on to state that we make 6 to 8 stereotype conclusions in the first 6 to 8 seconds that we meet each other. Assuming you are fluent enough to pull it off convincingly (one of the biggest hurdles of code-switching), once you demonstrate that you can speak their language, your audience will feel connected and bonded, and be more civil to each other, and more comfortable with you.