from: Maps API Usage Questions <maps-api-usage-questions@google.com> date: Thu, Aug 26, 2010 at 2:41 AM

Hi All,

It has been brought up to our attention that your application(at http://code.google.com/p/gmapcatcher/) seems to be caching or extracting content from Google Maps/Earth in a way that might be against our Terms of Service, available at http://maps.google.com/help/terms_maps.html

Please note that according to these Terms you shouldn't use the service in a manner that gives you or any other person access to mass downloads or bulk feeds of any content, including but not limited to numerical latitude or longitude coordinates, imagery and visible map data. You may also refer to the permission guidelines for Google Maps and Google Earth available at http://www.google.com/permissions/geoguidelines.html

Please stop this practice and respond to us. If you have any questions feel free to reply to this email.

Kind Regards, The Google Maps Team

Comment deleted

i think removing capability of bulk download in code will solve the problem. it's ok to view and cache viewed tiles. it's will be enough.

I agreed. I never used bulk download feature also. viewing and caching tiles are basic functions for any apps using google api out there. so I think it will be fine.

Isn't the trouble that many map providers allow bulk downloads, so we have to simply disallow bulk downloads from google - inter alia; however for an individual to allow such downloads, all they'll have to do is to modify/comment one python statement, hence it's virtually as simple as clicking a confirm button? Is it even right to make such a distinction - to distinguish those who are sufficiently technically self-confident to modify one python statement from those who won't?

Aside from the obvious fact that the web is not - yet! :-) - a proprietary domain, so it seems legally infeasible for a provider to legally limit the volume of non-malicious access to their website / public URLs, even though they could technically do so were it not a serious limitation of their commercial potential;

Moreover I feel it's kind of unrealistic - for the 'modify one python statement' reason - for us as devs to force google's bullyboy policy down the throats of eventual less-technical gmapcatcher users

well that's my 2c worth at least :-)

Today I got a reply from the Maps API Compliance Team:

from Maps API Usage Questions date Tue, Sep 7, 2010 at 12:34 AM subject Re: [#684120005] Google Notification to GMapCatcher

Hi Helder,

The code ( http://code.google.com/p/gmapcatcher/source/detail?r=928# ) that you have added warns the user that the action they are initiating is not permitted by the Terms of Use. However it still gives the user the option of proceeding. Consequently we believe it does not comply with section 2e of the Google Maps/Earth Terms of Service which states:

"you must not ... use the Products in a manner that gives you or any other person access to mass downloads or bulk feeds of any Content"

The key phrase here is "or any other person". By allowing the user to continue to download Maps even after they have been warned that this is not permitted, the developers are giving the user access to mass downloads or bulk feeds of this content.

So you are still not in compliance. Please stop this practice as soon as possible.

Kind Regards, Ruchi, The Maps API Compliance Team

You live in Florida, I gather? They allow jury trial in civil cases in Florida, perhaps you should remind google of that :-D

ps before you write back, could be worth thinking of alternate hosting such as github

more specifically, 'we believe' is not law, it is google bullying; it is not 'the developers' who "giv[e people] access to bulk downloads", it is 'the internet', perhaps 'the internet' should be the defendant :-D

Im surprised Ruchi didn't mention 2a :)

Well, web is not a proprietary domain but google maps data and services are. And by using it via web browser or google earth people agree with terms of use. Whether it is or it is not "limitation of their commercial potential" doesn't matter at all. Owner of the data sets the terms.

What makes URL "private"? :) So if somebody wants to put 2 computers on internet - he needs to make connection between 2 branches of his small dynamic growing company - it means - hallo people, anybody can get to the computers! They are mine, but I connected them to internet so you are right to do anything to them. Have a fun :) Mark, are you sure this is OK? I don't think so.

Does github make any change?!

well, git allows parallel branches, plus is generally more advanced than svn; there's a git for windows although how good it is I'd need to check; macports too; github allows bug tracking, downloads, etc, so is similar in most ways save that arbitrariness won't pull the plug :-)

As for the law, standa, what do you define as 'data ownership'? Copyright? Say for instance, for the sake of simplification, google has purchased the copyright to the images - reasonably true for the sat images, a simplification for the maps/overlays; copyright law specifically allows people to cache data, that is all we allow people to do; Say someone were making additional use of / publishing / etc the data, that would potentially be a breach of copyright, not the caching of it; even there, there is a grey area in terms of web content, as copyright is often deemed effectively waived when data is made accessible; what do you think for instance the web archive does?

As for the notion of connecting your own computers, well legally every non-malicious chancer would be allowed to download such data [as they could reasonably claim was non-maliciously requested] in a generally authorized manner; However practically, all data aside from data that a server is publicizing to the web, would be considered intended kept private - ie unauthorized; a grey area would be for instance windows folder sharing defaults

Now as for authorization / private URLs, well it's reasonably simple; a password-protected area is prima facie 'unauthorized' while files made freely accessible from a web server's http protocol port 80 are prima facie authorized;

This is when we reach the question of the proprietary web; google, in claiming the right to define what it thinks of as authorization, would bring the web to its knees, as for instance say I put up a website, including in a relatively obscure location some terms that say no-one who has ever belonged to a right-wing political party is allowed to surf my website at all, then I could potentially sue anybody who had some kind of right-wing affiliation who surfed my website? Utterly ridiculous, it's up to me to at least put up some kind of technical hurdle, a notification page at the very least, before claiming my right to filter surfers; google's lack of such a page simply serves their own commercial interests - it would discourage surfers - so our addition of such a 'page' is entirely the correct way of handling it; heck there's not even a link to the terms from maps.google.com, you have to basically specifically search for them

To return to their specific contentions, it's clear that 'gives [a] person access to bulk downloads' is precisely what the web does, when google refuses to implement a technical barrier to that - it's up to them to do so, not us, although they won't for obvious commercial reasons, hence the bullying; unless they're thinking of paying us, we should hold firm as we're independent of them, we have no reason for serving their whims

ps 'by using it via web browser [..] people agree with terms of use' is not right, people need to at least view the page that contains the terms, possibly click an accept button/check, to truly agree; even then there is some question whether 'click-through' agreements are really valid

For similar reasons, unless we had an API key, we're not bound by the API terms either, hence the apparent misunderstanding in that we're written to from someone in the 'API compliance team'

As for 2a, hopefully even they notice that we are basically a browser, albeit a very specific kind of browser

pps as for 2a, no worries; after you've given in to their interpretation of the legal effect of 2e, 2a would be the future bone of contention :-D

Mark, thank you for long explanation.

IMO using warning dialog box that allows users to continue is not enough. User must not have option to download at all. If he/she changes source code it's his/her responsibility.

heck there's not even a link to the terms from maps.google.com, you have to basically specifically search for them

Not true. When you display web interface to googlemaps (maps.google.com or your own webpage using "Google Maps JavaScript API V3" - for example see http://code.google.com/intl/en/apis/maps/documentation/javascript/basics.html paragraph "Region Localization") there is line at the bottom of the map. Starts "(C) 2010 Google - Imagery..." ends with link to "Terms of Use".

When you display web presentations of big corporations like www.microsoft.com, www.oracle.com, www.ibm.com, www.sony.com, www.verizon.com,... - there is always link to Terms of Use or Terms or something very similar. I don't think all this corps. with army of lawyers have this link because they just like it. I would be very careful about it.

when google refuses to implement a technical barrier to that

Not true. Lets have for example URLs of the tiles: http://mt0.google.com/vt/lyrs=m@132&hl=en&x=322&y=701&z=11&s=Gal http://mt0.google.com/vt/lyrs=m@132&hl=en&x=696&y=899&z=11&s=Gal (Got links by network sniffer using my web browser.) It is very simple to create script using fer example wget. Script creates URLs for tiles with (322 <= x <= 696) and (701 <= y <= 899). One can get this way tiles for much of North America. Can you tell me, how many tiles you downloaded this way? :) You didn't get all? What happened? They have technical barrier? Damn it.

we should hold firm I agree. But I'm not convinced yet.

Hi Standa,

thanks for helping me notice the link to the terms - I've noticed it before, it must have been hiding in camouflage when I looked :-)

I seem to have managed to put together a shell script to download tiles - in fact it was unstoppable wget must involve thread/fork

!/bin/bash

server=0 for h in {322..696} do for v in {701..899} do let "server += 1" let "server %= 3" wget -U "new sample browser version 0.0.1.1" "http://mt$server.google.com/vt/lyrs=m@132&hl=en&x=$h&y=$v&z=11" done done

basically their 'technical restriction' is at most a speed bump, no more than that

My main feeling is that quick capitulation serves us no purpose at all; google should already be wary of potential adverse publicity, that would be all the more true were there ever a real threat of legal action involving a rather dodgy legal basis; besides a jury, at least, should be willing to think of david vs goliath; so basically our only real risk is of needing to change hosting; sourceforge is a possible too; aside from all that even the vague possibility of judgment in google's favor would be for pennies, the additional cost of the server load that could be proven to result from our program :-D

Best

Mark

I think you guys are doing the right thing by not giving in to Google's request to remove the bulk download functionality. They are providing the information online, so if a user makes a decision to cache what they download that is legal I think, but also the users responsibility, gmapcatcher is just a download tool and can't be illegal. cheers, Jamie

Ad technical barrier - explanation: Several months ago using python and downloading tiles using URl different only in 'x' and 'y' (parameter 's' was set to constant value) I was able to download only few tiles (100 or may be up to thousand) after that I started to get HTTP error reply with some kind of 'violating' message. Next few days I was unable to download any useful data.

I am unable to get the same error now. So I'd like to take back my argument - existing technical barrier.

If I get to it somehow, I let you know.

standa.

Some interesting points are being made here to be sure, and its good to see you dudes/dudettes are not buckling to the pressure, keep on rocking. If people are limited from downloading certain amounts of data, then what is to say everyone could not download a quota or certain area of 'the world', zip it up and then share such information via torrents on the internet?

This would be helping you out kindly Google would it not? If it is hogging bandwidth that is denying a fair service to others as you claim. Then we should all download a small chunk and share those small chunks with everyone else, and then your bandwidth problems would be solved Google.

Liberating the world from the hands of the corporation. Sounds kinda catchy i reckon. But then what happens if you hurt one corporation? Another one usually gains. We need to bring them all down together at the same time, maybe ;).

But in the meantime liberating the world sounds cool to me. Such information should not be for sale, it should be available for all. As humankind. I thought Google according to your very humane sounding idealism, that you 'wern't evil'.

I suggest you refrain then from threatening people who are trying to help humanity, and not just line their pockets with gold.

Lots of love - Dan xx

i don't think that we are should ignore google's request. maps is their property. so they are have rights to say on which conditions they are allow to use their product. easy way to violate somebody's rights does not give a rights to do such. i think there is should be compromise. for example, if google want to show ads to those who view the map, so maybe it better to add google's ads when show google's maps. or something else.

pokrash, when someone knocks on your door with a hammer, you tend to answer them negatively; let google make a sensible request for a reasonable acceptable compromise then we'll possibly be more persuadable; as for me, I'm not convinced advertising beside the maps is appropriate, my thinking is that in fact this is a wake-up call to google to improve the way it adds placement to maps - hotspots [would need API], advertiser visibility/emphasis etc rather than lists beside the map;

As for 'ownership', as I've already said it's a term that is ill-fitted to internet serving of data, while the use of the word 'rights' to refer to a corporation's profit motive is kind of dangerous, as there is no real parallel to for instance individual human rights

@Standa, there's a 404 forbidden - the message says 'suspected automation' - unless you vary the server number; even so, the 'technical hurdle' that I was thinking of was more in the nature of a password-protected / captcha-protected access to the tiles, that would identify an 'authorized' zone, an important notion in the law of the internet; generally, similar to the prominence / inevitability of click-to-acccept of the terms, it's an incremental factor, I'd say there's a clear flavor of google not bothering very much as all such hurdles would limit the number of 'customers'

@Mark: Thanks :) I was trying even constant server name, but was unable to get the 404. If I understand correctly this 'suspected automation' is what I was looking for and it is base for my argument - this is technical barrier. Whether it is enough or not for a low, i don't know.

A couple of points if i may.

Firstly perhaps some sort of discussion feature/board e.t.c would be a useful appendix to this current website, as is, its only really issues that one can talk about. An area for suggestions/discussion to improve things might be a good shout maybe?

Secondly aRe there plans to implement a shape down of the code, so as to reduce the ability to use the mass download function anytime soon? If so i take it, it would be preferable to stick with an older version of the program merely to download the tiles? Then switch to a newer version for the usability improvements.

Lastly, and this is a bit of a change of scenery, is it possible to download the tiles and | the download directly into a zipping/archiving program of some kind? I've noticed when zipping up the various 'levels' of maps that the amount one can compress the files/tiles is quite significant. I.e the entire level 5 or 6 of maps tiles(can't remember which) is about 80GB in size (at least) uncompressed but only about 10GB once compressed. This method of piping the tiles directly into a compressor of some sorts, would mean that one could archive vast amounts of files into a relatively small space.

I suppose the next issue would be if your program could retrieve these files from the archive relatively quickly, it would make for an extremely efficient way of storing files, and having access to a significant 'database' of mappery on a relatively small harddrive.

Just an idea anyways.

Keep up the good work fellas.

• Dan

Latest email from Google Maps Team:

Hi,

As mentioned in our previous email, it has been brought up to our attention that your application might be in violation of our Google Maps and Earth API Terms of Service, available at http://code.google.com/apis/maps/terms.html

While we are unable to give you legal guidance in this matter, we remind you that Google reserves the right to suspend or terminate your use of the service at any time. Google also reserves the right to exercise or enforce any legal right or remedy contained in the Terms of Service.

Please stop this practice and respond to us within 3 days. If we do not hear from you, we might be forced to initiate legal action against you.

Sincerely, The Google Maps Team

Threaten.

How Google to take 'legal action' to us? Our project is hosted at Google, they can attend us and remove the feature by themselves.

However, for our safety, I suggest us to remove the bulk downloading buttons in our next version.

for our safety sounds distinctly as though it's giving in to bullying though?

Myself I'd resist, although I won't speak for everyone - how brave are we all feeling? :-D

keep your local files up-to-date as we may need to upload them to github/sourceforge :-)

remember, too, that 'our safety' is from a legal action that would net google at most pennies, plus considerable criticism for its disproportionate methods

So what exactly do the google terms of service demand from this project? Do they dislike the client software, or the amount of bytes downloaded from their free service? Caching data happens in every browser. There are many more http clients than just the browsers, each of them being able to store the downloaded images as local files (==caching). I do not know what google desires.

Oh how i love corporations, for all the suffering they cause my fellow human-beings.

I have a point i should like to make regarding proceedings in a court of law.

As far as i am aware corporations are always named in a dispute i.e 'googliath vs David'.

Now how in a dispute/court of law is it fair to have a corporation/conglomerate, which consists of many people stand against one single person? A corporation is not a human-being therefore to allow such proceedings to take place is surely fundamentally unjust is it not?

We could also add the fact that the corporations are the ones passing 'laws' that are beneficial to themselves in the first place, using such methods as lobbying e.t.c to pass laws that suit them and their vested interests. This has truly corrupted and rotted away the very integrity of our law-making systems has it not?

A law to say you can't share with your fellow human-beings (DRM e.t.c) wonder who passed that 'law' through. I sincerely doubt it was the 'people'.

When this parasite (corporations in general) has infected our law systems and culture to such a degree that they have a monopoly or total control of certain areas, what is stopping them from setting the prices as they see fit, stripping down freedoms, and ultimately enslaving people?

We are starting to see the begginnings of this turn of events come into fruition. Car insurance, you checked how much that costs lately? Gas/electric all suddenly rising and being blamed on the recession.

Its like a big excuse though for corporations isn't it... the recession (that they caused in the first place)...the raising of electric/gas utility bills..corporations answer = well its the reccession innit geez, need to tax you more...car insurance premiums being raised to extortionate rates = corporations bullshit excuse = well its the winter innit and also yeah people are claiming more bogusly,= thatll do..theyll buy that... = hike the prices.

Job agencies getting their grubby little mitts on a monopoly of labour force. You been to the jobcentre lately? 90 per cent of the jobs (granted a guess but check it out for yourselves) are via agencies. So basically they take a cut of your wages and strip you pretty much completely of your rights (i.e they can fire you at any point, without really any reason). And if you refuse any jobs on the grounds that you are being exploited? Well apparently they can cut your benefits. But in response to that i say this...these agencies are services which you (the customer) use in order to acquire employment, it would be a guess but i would say it is against your human-rights to have to accept a service, and that you can choose whether or not to put business in that agencies way or not. If your benefits are cut because you didn't apply for a job via the agency, i would say you are probably in a fairly good position to argue the above point.

Of course the counter-measures already installed for not accepting such employment would be :- well he/she doesn't want to work and as such due to negative stigmatisation is outcast. This is insanity!! and i say to those people, what happens when everyone has no choice but to accept these jobs and the choice is taken away from them? Then these agencies have total control of workers. In that pretty much every job will become via an agency and people will be able to be hired and fired in a heartbeat, and 'worker rights' will no longer exist.

For the sake of the next generations to come we cannot and should not allow this to happen. We cannot just keep burying our heads in the sand and say, well what can we do, its pointless e.t.c. Thats a loser mentality and that is exactly what this world would want you to think. Instead we all need to start taking responsiblity for this situation in our own little ways, whatever they may be, and do what we can no matter how small it may seem, to make things better, that is our duty as human-beings is it not?

Then if thats our duty we are all failing pretty heinously right now aren't we. So we all need to buckle down and do what we can, to re-empower ourselves as a people, even if it seems small or insignificant at first. Even if it involves making some sacrifices along the way.

I tell you if they keep squeezing and squeezing us, theyre gonna get a shock pretty soon, and their outright greed will be their own downfall.

What happens when these corporations that have monopolies over certain 'areas' merge into giant conglomerates? Then more and more power becomes concentrated into fewer and fewer hands. Surely this would end in a 1984-like scenario? Except its not really communism that would of caused such a scenario as 1984, which it would have you believe is it? It is in fact the current system which the western world uses, which logically results in such an end-game scenario.

Capitalism is the syringe, corporations are the flesh eating bacteria.

So 1984 was always just propoganda and along with other 'paintings' put in place over the course of the last century, it would have people believe that communism (i.e working together, look at the deriatives and use some common sense ;) commune, communal e.t.c) will result in such a gloomy scenario, when ironically as we see in our current world situation, it is going to be capitilism to create such a place.

We always have hope however, and we can make a change, this world would have you believe that you cannot, but i tell you that is total bull-shit. We the people can decide when enough is enough, and anyone who tells you otherwise, needs to open their eyes and do something about it.

I do realise however that everyone is entitled to their own views and beliefs and saying such things could be compared to being a 'armchair warrior' e.t.c. At the end of the day it is you dudes facing the possible repercussions for your actions and no amount of moral support could or should ever change that fact.

In such a situation it would be prudent to do some risk-assessment, is the risk really worth it? I find it sad that Google is being all petty about the mass-download function and wanting to threaten legal-action. But then there are of course other courses of action that can be utilized.

For example as i earlier mentioned, say you did in fact limit the bulk download function, which is reasonable conisdering you are in the firing line. What would stop a massive amount of people (i.e anyone who uses googlemaps) from caching and then sharing their unique individual information with one another on a network other than google. i.e bit-torrent e.t.c, this would be helpful to google because as they mentioned it is the 'server-load' that they are so worried about, and in a way the above actions would actually decrease google's server load, would it not?

Anyways whatever course of action/non-action you dudes choose to persue, i have to say i appreciate the work you've done with your program its well cool.

• Dan

Don't take my word for it about capitalism though,

Einstein was saying it 60 years ago!

http://www.monthlyreview.org/598einstein.php

yay for middle-ground-ism :-D

This is great program! Hard to find similar as google tends to be pretty protective as you can see. Could you just program out bulk download for google maps and keep bulk download for all others? Might get google off your back and avoid a lot of hassle. I want to keep this around and see it continue to be improved! Keep up good work.

some further points

– non-cooperation is an appropriate way of handling harmful behavior; the alternative involves bearing some responsibility for the adverse consequences — think of what you'll say to your grandchildren when they say 'what did you do during the war of the proprietary internet?' :-)

– A restriction of gmapcatcher would lead reasonably directly to a fork of the project, such as 'freemapcatcher' :-D while it's far less probable that people would fork to make a more restricted project

Hi everyone,

First, thanks for your feedback and comments on this issue. I've had the opportunity to try and clarify our reasons for our Terms of service with the lead developer, and wanted to provide the following summary (we try to keep our terms as short and readable as possible, but it's still a couple pages):

• The overall reasons for our restrictions on tile access are many, including the fact some of our content providers we license data from require we allow access only via our published API's
• We do not allow any download/offline caching of tiles, outside of standard browser caching that takes place when viewing tiles through the API.
• I do not see any way your project goal (downloading tiles) could be compatible with our existing terms for tile access.

I am aware that a number of alternative tile providers have terms that are less restrictive than ours, and allow direct tile access, Meanwhile, some providers (including us) limit access to tiles and other content only if you use the API directly.

I realize there is a lot of interest in accessing the content directly, but I appreciate your efforts as developers of open source software to do your best to follow our terms.

I also hope that despite the incompatibility with this particular project, you find our overall set of free services useful, and that you create other projects that can make use of them while still following our terms for accessing the content.

Thanks,

-Josh

Hi All

Today is a very sad day; I just committed a change (see r1128) to remove GOOG from the map services menu, and also from the default in the config file.

Helder

The future is relatively free, however, as in new news, the US Geological Survey has made access to all sat images 100% free, as in Beer :-)

http://www.newswise.com/articles/view/574813/

suggesting the possibility of some kind of 'app style' improvement to gmapcatcher, that for a subscription to cover the cost of cloud servers, would provide the sat images directly; overlay, say OSM data, then the result is freedom - as in speech - from the domineering stance of some of the googles of this world :-)

Issue 274 has been merged into this issue.

I repost my question here:

Google Map now allow users to download Maps for offline browsing. Can we put Google Map back to gmapcatcher?

http://techcrunch.com/2011/07/07/google-maps-for-android-now-lets-you-download-maps-for-offline-viewing-hallelujah/

Although the Android Application does now let you pre-cache a set of data, we still do not offer this ability through our API. But we realize it is a popular feature request, but the same reasons noted in comment #30 above still apply.

Issue 383 has been merged into this issue.

Issue 383 has been merged into this issue.

Issue 383 has been merged into this issue.

Issue 383 has been merged into this issue.