Working at Google: Maps

"Oh, I Love Google Maps!"

In the last three episodes, I was in Enterprise and then in Ads, helping them make money hand over fist, sometimes. One of the Owned and Operated Properties I helped out was Google Maps, working with their Monetization team headed by Chip Chapin, an easy-going guy whom I bonded with. So Maps was my next destination. But first, an amusing (I hope) digression:

The Yellow Badges and Google Books

When I started in Maps, I was in GWC-1 (for Google West Campus, near the corner of Rengstorff and Charleston). Occasionally I’d go for a walk around the area, and I’d see a bunch of people hanging around the doors of GWC-3 on their break, smoking cigarettes and looking more like high school students than Googlers. They were book scanners: hourly workers who were paid about as much as Burger King staff (I got that comparison from a Googler who gave a tech talk about the project).

The Atlantic wrote an absolutely great article about Google Books.

You should go read it now, if you want to know the depressing legal reasons why you can’t just go to Google and download an out-of-print and long-forgotten book, even though they have it, scanned and ready.

When I first joined, I went to a number of internal talks about how book scanning worked, and I thought it was incredibly clever. It’s described here, and they got a patent on it.

The machine projects an infrared grid on the pages, which doesn’t show up on the photo but lets the software “flatten out” the page. Thus, book-scanning does not require cutting off the binding and destroying the book. They could take a book off the shelf, scan it, and put it back none the worse. They did that for entire university libraries, like the University of Michigan’s.

The machines required a human to sit there, flip the page, and take a picture. Over and over. That’s what the book scanners did. It was not what you’d call skilled labor.

They had yellow badges (at Google, regular employees had badges with no color, contractors had a red background, and interns had blue). The symbolism of yellow apparently was lost on the HR people who chose that color.

The Yellow Badges were not permitted to go anywhere else on the Google campus, and they couldn’t even get into the microkitchens in their own building.

I found that I could ask anyone at Google if they’d ever seen a Yellow Badge, and no one had even heard of them. They really were invisible. I wish I had talked to some of them when I had the chance. Now that I’m retired and walking the dog twice a day, I talk to lots of “normal” people who don’t work in high tech, and they’re some of the most fun and interesting people in Silicon Valley.

Eventually, book scanning was moved somewhere off campus, and GWC-3 was renovated for normal Google use. My own project moved over there.

As The Atlantic article says, Google has a vast storehouse of millions of the world’s books, and no one can read them; not even Google employees! Hard to believe, isn’t it? Congress could pass some legislation to clear up this mess, but I don’t think they’re even considering it.

Enough on that; onward to Maps.

Maps

The first thing I noticed about being in Maps was, when I told someone outside of Google what I did, they always said, “Oh, I love Google Maps!” This was a change from the days in Ads, when they would say, “I never click on ads!” I would respond, “Well, someone does, or we wouldn’t have all this free food!”

Chip Chapin was the project leader of a group that was changing My Maps, the facility that lets you create your own map. Specifically, they were adapting it to use Fusion Tables. It seemed like a straightforward way to get introduced to Maps, plus I got along with Chip.

For an example of My Maps: if you’re having a wedding with out-of-town guests, you’d make a map with the church, the reception site, the hotel for the guests, the airport, and so forth, and distribute it to the invitees. I’ve used it for vacations: for the places I planned to visit, hotels I was going to stay in, etc.

In 2010, My Maps looked like this

Note that “My Maps” is right there next to “Get Directions.” In 2022, if you click the “hamburger menu” to the left of “Search Google Maps”

You get a list of all your saved maps

There was a directive to convert My Maps to Fusion Tables, but don’t ask me why. Apparently someone thought this would be a good idea. Fusion Tables has now been discontinued, leaving the state of Arizona, among others, in the lurch, since they had committed to it.

If you’re suspecting, “They have more engineers than they know what to do with,” I wouldn’t argue.

Maps Monetization

There was a larger effort to make money off Maps, which had always been a money-pit (or “loss leader” in retail terms). The idea was that a lot of businesses need accurate geographical data and Google should provide it, for a fee of course. Natural resource companies, developers, governmental units, solar installers, power-generation companies… the list goes on. Unfortunately, there already is a big player in that space called ArcGIS.

It’s time for a little digression about customer lock-in:

There are some extremely expensive software packages, like SAP (for business), Oracle Applications, SAS Analytics, and IBM mainframe software of all varieties, which are excruciatingly difficult to learn but have almost every function you can imagine. Users become fanatically loyal to them, because they’ve invested so much effort and money that they’re basically hostages. There is a whole industry of SAP consultants, for example: expensive experts that you have to hire after you’ve installed SAP. They hang around for years. These systems become impossible to get rid of once you’ve installed them.

“Stockholm Syndrome” would be another word for it. ArcGIS is one of those hostage software systems. People who use it cannot imagine using anything as déclassé as Google Maps. That’s for consumers, not serious business people like them, thank you.

There’s a book about how companies with a locked-in market like that get dethroned: The Innovator’s Dilemma. Generally, it goes like this:

1. A tiny company makes an inferior product with cheaper technology, which captures the low-end of the market. It does most of what the big product does, and it’s much cheaper.

2. The big company doesn’t take the newcomer seriously, because it’s so inferior. In fact, the big company’s profit margins improve, because it wasn’t making much money on the low end.

3. The tiny company uses its profits to improve its product, and begins taking more of the big company’s market. They call this “moving up the value chain.”

4. Eventually, it captures essentially all of the market, and the big company is out of business.

I was excited to re-enact this with ArcGIS. Their product was incredibly difficult to learn, while Google’s was consumer-friendly. The opportunity was obvious: maybe Google Maps is not as comprehensive as ArcGIS, but it’s way cheaper and easier.

That was my thought, anyway. I thought it was the management’s, too. It turned out the executives were more interested in fighting with each other.

Chip led a group which included the Crisis Response Team, which helps when there’s an earthquake, flood, tsunami, or other disaster. This is especially relevant to Maps, for obvious reasons: a disaster map needs to show where the relief efforts are, where survivors might be lost, etc. Those needs overlapped with the needs of “regular” users of My Maps, so rather than create a separate tool, they could add Crisis Response features to My Maps.

A Weird Meeting

At some point we had probably the weirdest meeting of my entire career.

I mentioned that the Crisis Response Team was part of our group. Apparently Chip was not pleasing the Powers That Be. Maybe as a more senior guy, he had opinions, and wasn’t as easily persuaded as a young, naïve person.

It was proposed that Ka-Ping, who was the Crisis Team leader, an extremely good engineer, and a very precocious lad, become our overall leader. Now, normally when there’s a coup attempt like this, it all happens behind closed doors and is announced as a fait accompli. It is never, never debated openly.

Nonetheless, that’s what happened. We went around the table, with Chip and Ka-Ping both present, and spoke our piece about whether he should take over or not (I said No and supported Chip). Ka-Ping did not win. I still cannot fathom why anyone would want to humiliate himself like that.

Management is Imposed

As I said, Chip was apparently not pleasing the higher-ups, and an outside person, Louis Perrochon,

was brought in as a second-level manager. Louis, in turn, hired a guy named Igor Benko

to be our manager, and a 20-something named Chris Ouk, as our project leader. Chris had no background in Maps. Igor had no background in Maps. Maybe that made them ideal as people who would be malleable.

At any rate, with the management turmoil over, we could proceed on a full-blown My Maps rewrite. This was significantly more work than the original plan to just convert to Fusion Tables.

There were about six of us, as I recall, plus a User Experience (UX) expert and a Product Manager. Most of the work was in JavaScript on the user end, but for some reason I can’t recall, I didn’t want to do that, and kept working in Java, which meant the backend server.

The server includes the database, and at Google, it seemed like there was a new database project every month. BigTable is widely publicized outside of Google, but there are myriad systems built on top of it. Maps was using one of those. Silly me, I thought they actually cared about attracting third party developers and wanted a database API that was attractive to them.

My mistake. What they actually cared about was getting along with the rest of the Geo team. A new, business-friendly API was something they didn’t care about at all. I should have listened to the conventional wisdom.

I did write one feature you can still use! Let’s start with the My Maps. Here’s the Excel I’m importing into Maps:

Austin Fatal Accident

14.7KB ∙ XLSX file

Download

Download

We’ll get to that. Here’s what My Maps looks like, with one of mine:

A MyMap can have layers, which I’m not really using here. You get to this screen by clicking “open in MyMaps” in the previous screenshot.

Dates: All these screen shots were captured in December 2022. The software might have changed by the time you read this, but right now you can still do it.

Follow along here; this might be something you actually want to do and didn’t know how! All these features were the subject of innumerable discussions with the product manager and almost everyone else.

Obligatory note: of course I didn’t do all of this by myself. Hardly anything in Engineering is a solo effort. Furthermore, I’m certain that lots of people who came after me fixed bugs and improved it.

Adding Your Data to a MyMaps Layer

Your own spreadsheet is what we call “user-generated content” (UGC) and it has to be handled like an unexploded bomb, initially. There are thousands of hackers who’d dearly love to inflict some kind of harm on Google, so before any processing can be done on their data, we have to inspect it, gingerly. I had to learn the proper way of handling UGC.

I’ve added a layer to my map called “Austin fatal accidents” and I have a CSV from data.world that I want to import:

So I click “Import” and after providing my file, I get this dialog. The software has to know which columns your address is in, if it can’t guess (the City and State columns are present, but they don’t show in that picture):

So I select Address, City, and State and hit Continue. Now I have to name each point. I decide it will be the person who got in the accident:

I hit Finish and we get this:

Let’s click on the easternmost accident, near the Tesla factory:

Errors

Now, if you look over on the left, you see a message “1 row couldn’t be shown on the map.” This is the problem that occupied us for months, and I think I did three or four different versions that were not usable.

We can try to translate the user’s address into latitude-longitude pairs, but it’s not always possible, so we have to indicate that to the user somehow. How we do it was the hard problem. I think they came up with the best way after I left.

The answer (which, full disclosure, I didn’t do initially), was this:

MyMaps puts a red “alert” marker on the offending column. In this case, it was that “6200 N IH 35 SB SVRD FLYOVER TO EB E US HWY 290 EB” could not be translated. Now you could edit that cell to get it to pass.

Maybe you see that red flag and think “of course, that’s obvious!” All I can say is, “Everything is obvious, once you see it.”