https://support.google.com/gemini/answer/16836988
“[Gemini app activity] is used to improve Google services for everyone, including by training generative AI models, if your Keep Activity setting is on.“
https://support.google.com/gemini/answer/13594961?hl=en#human_review
“A subset of chats are reviewed by human reviewers”
https://support.google.com/gemini/answer/13594961?hl=en#pn_retain_data
“Chats reviewed by human reviewers [...] are not deleted when you delete your activity. Instead, they are retained for up to three years.”
https://support.google.com/gemini/answer/13594961?hl=en#pn_data_usage
“Please don’t enter confidential information that you wouldn’t want a reviewer to see or Google to use to improve our services, including machine-learning technologies.”
To disable their training on your chat sessions, you need to turn off “Keep Activity” in Gemini’s settings. Turning that off means you’ll lose any chat sessions after 72 hours.
I learned this too late, thinking that having a paid subscription = not being turned into the product. Apparently paying is not enough.
Thank you to this user on HN for mentioning in another thread: https://news.ycombinator.com/item?id=50355424
PSA: This is true of all the frontier model providers even if you disable activity tracking. They are all constantly monitoring your interactions for problematic behavior.
[0]: https://www.nytimes.com/wirecutter/advice/ai-chat-privacy/
[1]: https://www.404media.co/inside-project-lily-the-humans-readi...
[2]: https://www.winknews.com/news/woman-arrested-after-ai-threat...
For an AI system to be “completely safe”, like a self-driving car, it must map the full environment it operates in and be taught the “right way” to handle every conceivable scenario.
So when you present it with novel data or a situation, that constitutes an “AI safety event”, which per their ToS is a carve out for training on your session.
Further, go read the ToS carefully about what you opt out of. You’ll note there’s nary a mention of tool calls. You’re only opting out inputs (what you type) and outputs from the model from being trained on. All the content of the tool calls made while running on your system are fair game for training.
I had it switched off:
"Hey Google, remind me at 7pm <something>"
"OK, your reminder is set for 7pm"
Next day:
"Hey Google, why didn't you remind me at 7pm"
"I can't do anything with your phone unless you enable chat history."
WTAF!
Google has this pattern of all or nothing approach.
Maps refuses to show your *current* location if you don't enable location accuracy i.e. if you won't allow it to spy on your WiFi or Bluetooth it refuses to show your current location at all. GPS, what's that?
We can disable notifications and even phone access but physical activity is the one permission that cannot be disabled for play services !?
Google apps have a lot of WTFs!
There's no reason they can't have a mix of "this is popular in general" and "your subs and likes suggest you'd like this" on the front page - they just don't want people to disable watch history. They are required to offer that as an option, but they'll never stop pestering you about it.
That one I do like. YouTube’s homepage (at least without an account and with history off) is a cesspool of uninteresting clickbait garbage so I’m glad to have it off. I do agree they seem to think of it as some kind of punishment¹, which makes it all the more satisfying that it works as I want it.
¹ It’s now entirely blank for me, but I remember when they started doing this (it wasn’t always the case) they showed a message that suggested not having a homepage was undesirable.
I know this idea is over the head of probably 75% of people, so forgive me if this is too dense to understand, but "full payment" is the sum of all forms of payment (usually a combo of data, ads, and money). The seller of a product/service is the one who determines the payment component ratios, and the buyers decide if they want it or not. This applies to everything from streaming services, to app subscriptions, to TV sales.
So while true that you are giving money to Google, it is also true that you are paying the rest of the "full payment" with your data.
Google offers the option to pay the "full payment" with money too. But often companies do not, and only offer other ratios of "full payment".
Don't feel bad for not grasping this, seemingly everyone else is unable to get it without having it spelled out.
I found out later that Goog411 was created to primarily train on the voice prompts from the users of the service. Left a really bad taste.
Call it what it is, please. User-hostile behaviour that compels you to turn their tracking on if you want to use any of the practically helpful services.
I'm a super privacy conscious person, and even I end up keeping many of these things on simply because I want to use for eg google assistant.
Thanks for the reminder OP!
If Gemini has access to app data like gallery, mail etc that can also be subjected to human review? I imagine that will also be part of the context.
They are vague when they say 'don't enter'. Is it only prompts or does it also cover other data too.
https://github.com/moooff/HermitUI
They can use it with a singel click for there confidential stuff.
https://moooff.github.io/HermitUI/dist/hermit-ui-wllama.html...
I just sorta assumed that Google was hoovering up anything I type into its AI and don't tell it anything too personal or sensitive.
Is anyone under the illusion that our data is not being collected, whether it's Google or literally even the seemingly innocuous TVs in our home that have been found to be spying on us 24/7?
Literally the opposite. People don't read privacy policies, even ones designed to be readable.
The privacy policy says it grabs everything, and there's nothing that says they'll stop just because you pay them.
> A subset of chats are reviewed by human reviewers
This is expected these days for threads which may make the provider liable. Just look at the recent death case due to bad GPT medication recommendation.
> Chats reviewed by human reviewers [...] are not deleted when you delete your activity. Instead, they are retained for up to three years.
They're being retained for legal purposes.
> Please don’t enter confidential information...
This is so they don't accidentally preserve it for review, but also they don't want you to share private info that the agent will post somewhere without an explicit approval.
Those points make sense for them, regardless what other T&C rules apply.
But there's two reasons to bring cautionary attention to it anyway:
1. As a user, I may not be okay with those terms as they stand and may not have the legal insight to have intuited them. In as much as I have a choice, because of those terms, I may want to use Gemini less/differently or look at other providers that have crafted less shrewd and defensive legalese and processes.
2. As a executive or manager at Google, whether working directly in feature management, or advertising, or government compliance, these same terms define the framework for anything I want to develop. What "makes sense" from a defensive legal standpoint today can the open door for an exploitative or otherwise unexpected use later.
I get what you're saying here in terms of plausible or justifiable "innocence", but that probably shouldn't get in the way of some critical personal reflection for users.
> They're being retained for legal purposes.
It’s a significantly wider usage than that:
A subset of chats are reviewed by human reviewers (including Google’s trained service providers) to help improve Google services. These include Gemini models, other generative AI models that power Gemini Apps, and technologies that help reduce unintended activations.
Reviewers assess if Gemini Apps’ responses were low-quality, inaccurate, or harmful. They also suggest better responses. We use their suggestions to help our models respond better in the future.
That’s in addition to the liability language:
Chats are also reviewed to keep Gemini safe and secure for everyone and to address violations of the Google Terms of Service, including the Generative AI Prohibited Use Policy. Reviewed chats are retained for up to three years.
https://support.google.com/gemini/answer/13594961?hl=en#huma...
Some important things to take away from this:
* They’re claiming to use chat transcripts for both training and for human review to improve training, not just for liability.
* They don’t define “subset”. It could be most chats.
* They make no promises of anonymizing content, just “disconnecting” it from the account.
* This is happening for free and paid user subscriptions, by default, and the only way to stop it is to turn off chat history altogether.
Don't care for what, store it away audit save if you have to, talk to laywers in case you need to read through them but don't let random people read my stuff.
And in this case stuff = AI discussion which feels to me similiar to my google docs documents im hoping/expecting Google also to not read
But this would require an admission of limitations of the technology.
Do people sue Google if search links to a page that has misleading info that results in someone taking a wrong drug? Nope.
Do we sue the browser maker when the info on a given Web page is malicious? Also no.
So, I don't understand why we treat chatbot providers any different. Everyone should be expected not to believe a word they say without corroboration.
Some chats are saved for 14-30 days IF they trigger a safety classifier, but they are never used to train future models or retained after that period, the objective is to verify safety. Simple as that.
If you are a business user, like in the case of Azure OpenAI, the analysis and priorities are quite different though, but the mechanism seems to be the same.
The primary purpose has always been improving search overall -- traditionally this has meant updating query interpretation, ranking, presentation, etc for smaller and smaller query slices ("Ah, when we show the mayor and the population in response to [Barcelona] people just revise their query to [Barcelona FC], so we should make sure to include latest match result on the top. Not so for [New York]."), and running tests on every product change to see what people like. In this case nobody cares about any _individual_ user's data, only the collective. This is basically the original "collective intelligence" contract of PageRank, albeit hillclimbing on billions of searches and clicks instead of using link structure. (Old question: does original PageRank count as machine learning, or not?)
Second is personalization -- things like "Oh, my flight or my delivery is expected on date X, handy that gmail popped a reminder on my calendar, good work Google." Or searching for a restaurant in Maps and seeing results that are biased towards the kinds of restaurants you already like, that kind of thing.
When it comes to revenue the purpose of these isn't to target ads, it's just to have you get value from and use the product, and when you are ready to spend money this is where you come to search for the thing, whatever it is.
Finally there is limited use of a limited set of data for ads relevance. There's higher CPC for ads you might actually be interested in (and if you are looking to spend money, is finding a willing seller who has compelling terms via an ad such a bad thing?) But ad revenue is mostly downstream of users being satisfied with the core product and using it more, whatever that product is. (Search, Maps, AI chat, etc)
(Bonus. My Search pro tip re: ads is leverage the modern stack and spell out your intent with a few more words when you query. We're pretty good at not serving ads for non-commercial queries. But old-school super short "searchese" queries make it hard to guess whether your intent is commercial or not.)
It is actually, but the shady part is that this is not (just) to target ads to you, though that certainly happens… it is, as OP explained, so that they can better capture query intent AND your behavior (clicks, dwell time, subsequent searches, etc) that they then use to refine and rerank search results, which improves Google Search and keeps you using it, which becomes a virtuous flywheel that competitors can’t catch up to.
One way for competitors to break through is to buy traffic, but well Google blocked that too by using its monopoly profits to pay more than anybody else (Apple and Mozilla search deals.)
All of this came out in the Search Antitrust trial. Look up NavBoost for instance.
Anytime you aren't paying full price for something, you are paying the rest with something else. We know the real cost because there is an API that comes with all the confidentiality, and that is much more expensive.
FYI, this is true for the other labs too.
I don't think people understand that while consumers are dumb, uneducated, and broke, enterprise is a cash cow with a experts and a legal team.
If you serve them, like Google does on a massive scale, these ToS "gotchas" will do you in faster than you can say "lawsuit".
six months later
Quietly update the ToS to allow them to retroactively use all of your history and data going forward, attempt to fly under the radar and/or gaslight users that it's always been that way
“ This license is for the limited purpose of operating the services, which means allowing the services to work as designed and creating new features and functionalities. ”
https://policies.google.com/terms?hl=en-US#toc-permission
The training exclusion has always been a distraction. These service providers have plenty of other uses for your data.
actually even gemma locally is great for a lot of chat purposes.
eg Gemini enterprise has a note under each chat session explicitly saying it’s not used to improve the models
Local models are also a thing. The Qwen 3.8 I'm running on a spare computer is definitely not training on my inputs.
also the same cases for antrophic and open ai
why would anyone trust company that literally need data to improve their product ????
You’d maybe want to counter with something like 'But Google/Amazon/whoever just wants to sell me stuff!'. What makes you think that Chinese B2C would just pass on that opportunity?
I also don’t know that I agree with your idea that manipulation is a greater risk from foreign actors. You're right to point out that it can be a problem from either, but local actors have a particularly direct incentive to shape the opinions and behavior of the population whose views affect their own power and interests. I would not argue that means foreign actors are harmless, I just don't agree they are necessarily worse either.
Just like how Kim Jong Un killed his brother using dumb pawns to actually carry out the murder.
So if your head has not grasped this reality, then they are already two steps ahead of your "What could they possibly do".
This is also the reason why congress keeps banning Chinese enterprise. It's not because they care about privacy, it's because they care about people being turned into pawns.