I initially thought that maybe they were concerned about license violations, copyright infringement, etc. because you can't always be sure where the llms are sourcing their information from.
But rather it seems like they're warning against information leaking out?
> The Google parent has advised employees not to enter its confidential materials into AI chatbots, the people said and the company confirmed, citing long-standing policy on safeguarding information.
Is there any example of this happening in the real world? I've never heard of one.
Even if you were to give me access to some google source code free and clear I'm not sure what I could do with it. It's often only useful with the build systems, other services, external documentation, and tribal knowledge contained within the company.
Sure if I had unfettered access to all of the Google source for a product for long enough I might be able to find a security vulnerability or something but random chunks of code pieced together from the inputs to some chat bot?
It's not really about security in terms of attacking the site.
It's purely about IP protection and potentially leaking user/employee data. Any data you give one of these chat bots can potentially be regurgitated by the bot later.
We have the same kind of policies at my megacorp company. We can't even use things like chrome extensions that will let you paste in data to parse json or things like that.
They're especially good attack vector, because a lot of seemingly smart people may not realize that the little tool for pretty-printing JSON or changing User-Agent header could also be recording and sending out everything that goes through it.
Ah. Well, when phrased that way, that's more a general problem with installing extensions, especially ones where the recent version hasn't yet gone through Mozilla review (or whatever the equivalent is for Chrome).
Welp, fortunately, I've never had the slightest desire to install a json parser, the Firefox built-in formatting/parsing is quite adequate for my needs. Anything really complex filter-wise I just turn to jq.
I'm speaking from experience. Many years ago I did install a couple random convenience extensions like this on my work computer - including a very nice JSON pretty printer, back before such things were included by default. It took me couple of months before something in my head clicked, and I realized with horror what attack vector I've been enabling at work. Fortunately, this was also before buying off extensions for malicious purposes became a thing, and as far as I could verify, I got lucky and all those extensions were legit and did not exfiltrare anything.
Ouch. That must've been a long time ago, the formatter appears to have first added in Firefox 44 dev edition. But. Yeah. Unfortunately we do require extensions sometimes. For most users they are blocked.. I do stick to well known, Firefox recommended ones, where the extension has gone through their review. Certainly extensions can be bought out, but they'd still have to submit a new version for review with their malicious code in it, which hopefully would help. But yeah, that's not 100%. Major extensions have had issues in the past, thus the Stylish/Stylus fork.
I think that depends on how big and smart the AIs grow going forward. All the stuff you type into a chatbot today could be used to train the chatbot from tomorrow. And that chatbot might than be able to make connections between all the different bits of information it has available that could reveal information that weren't directly obvious from just a single snippet posted into the chatbot. Furthermore that information might then become available to the public, so everybody that asks the right questions could snoop around Google's private data.
Keep in mind, this isn't one employee asking one question to a chatbot, this will be tens of thousands of employees typing in multiple prompts each day.
More immediately though, if that data is stored for future use in training, or even recorded in logs, it can leak. Not just randomly - OpenAI in particular is sitting on a treasure trove of corporate secrets, medical data, PII, possibly even government secrets - all submitted by people unaware or dismissive of data security concerns. It's only a matter of time before someone deems it worth it to invest substantial resources into an attack, and if they succeed, lots of people and organizations will have a bad day.
The same could be said for typical search engines, although I must admit the amount of data a typical user submits (and/or that the input form accepts) is lower for those. Not to mention URL en/decoders, base64 en/decoders, prettifiers, format converters, ...
All of those should be included when training users not to divulge secrets, with chatbots just being one more added to the list.
So I think this recent buzz must be more about the model getting trained, than about discrete leaks.
You don't typically paste an entire document into a search engine for analysis. Further, search engines have a long track record of not leaking searches and not using the info for insider trading etc.
They're to some extent battle tested where Openai et al are unknown quantities.
Even if you did just trust OpenAI employees not to look at the information in the prompts you're submitting it, there's an underlying unknown quantity to LLMs that scares people. OpenAI has everything you type into ChatGPT and you can be sure they're going to use it as training data and somehow it's going to come back and bite you in the ass.
There are distinct patterns of use. While search engines as typically used could collect some sensitive information, there's only so much you could divulge in a query. You could e.g. reveal what you're working on and using what technologies, but not the specific designs or algorithms, because putting the latter in a search query makes zero sense. Not so with LLMs.
If anything, the underappreciated vector of accidental data leaks is the (annoyingly idiotic) combination of two distinct "features" - merging separate address box and search box into an omnibox, and "instant search". This means that whenever you're using your browser bar to e.g. force the text in your clipboard into plaintext, you're also sending all of it to Google (or whatever your browser is). So is the case if you're trying to type in an URL part - it may end up being treated as search before it starts matching something in your history.
As for the rest of things - en/decoders, base64 en/decoders, prettifiers, format converters - I'd say they're in the same category as ChatGPT: you shouldn't be using it with any sensitive data, because there is a very good chance they're collecting it. ChatGPT is at least run by a high-profile company (read: easy target for scrutiny and lawsuits). Random helpful utility for data conversion? There's a good chance it's been made specifically to collect data pasted by careless people.
First, a lot of organizations do caution employees against putting anything internal into a search query. It's not an unknown problem, but not as many organizations have a big enough security concern to spend training time on it.
Second, I think the potential exposure via chatbots is a lot higher. Part of the advantage of chatbots over search engines is that you can easily provide more information to refine the output... but that just means that you're encouraged to provide more information.
Certainly training is a factor in this concern though. There are situations where LLMs seem to output training data almost verbatim, which produces a very real possibility that any secrets in the training data could be retrieved by someone who spends time working out a prompt or just stumbles onto it.
MS is actually a mixed blessing, in that it's the one entity that companies will actually trust - once they sign a deal with appropriate provisions. In fact, Microsoft is renting access to OpenAPI models (including GPT-4 and GPT-4-32k) on Azure, at quite hefty premium, and the main (arguably only) selling point is that your company can get strong contractual guarantees wrt. data safety and confidentiality. It's the same deal as with Office and Outlook and SharePoint, etc. - without those strong, auditable guarantees, no large company or government would ever use those products.
My position is that a corporation doing business in the US is already vulnerable to US Government just coming and taking whatever data they want; in fact, as long as government's request has any semblance of legitimacy, e.g. by coming in as a signed and stamped letter from a government agency, corporate will happily hand the data over. After all, the government is not the enemy of a corporation - it's its life-giver and potential customer.
Corporations do, however, care about their data being handled in accordance with the law, and safe from competitors and malicious third parties (be they cyber-criminals, kids, foreign intelligence agencies, or journalists). Microsoft, as one of many providers of services to companies of all sizes, offers exactly such safe environment, which is also auditable, and comes with an actual SLA that makes them liable for fuck-ups. This is what serious companies want, and it's why they pay Microsoft and use their services - whether it's Office or SharePoint or Azure AI platform. OpenAI itself doesn't offer any of it, which is why any serious company has put it on a top spot of its shit-list.
> Is there any example of this happening in the real world? I've never heard of one.
1) How would you know if your idea or code were ripped off, unless it was an exact bug-for-bug copy?
2) These things have only existed for 5 minutes.
3) Google knows exactly how these models and their interfaces work, and has probably speculatively designed many ways to mine inputs for interesting items.
> Even if you were to give me access to some google source code free and clear I'm not sure what I could do with it.
They're not afraid of you. I wouldn't know what to do with five pounds of gold or a bag full of stolen jewels. Doesn't mean they're not valuable to someone.
I don't think that this is thought of as an attack vector as much as a leak or a crap vector.
If your confidential and clever trade secret technique leaks into somebody else's code (or could have leaked) you may lose trade secret status. If you have a corporate policy to point to, you can claim that any leaks are due to unauthorized activity and thus the trade secret still stands.
On the other hand, if you import somebody else's proprietary code, or GPL'ed code, or buggy code without realizing it, you face different risks. Being able to point to an established policy against using code from these tools helps with the defense against trolls who assert some vague kind of theft.
Things got a little confused - what I linked to was a hypothetical private ChatGPT. I'm sure Google already has lots of private ways to use AIs that are not the public Bard interface.
I suppose it's an over-arching policy created that makes it easier to mandate. If not, Google would have to say - you can enter some stuff into chatbots, but not sensitive stuff. Then it would have to classify sensitive and non-sensitive, keep track of this, have this up to date etc and etc - you get the idea. By sensitive stuff, I'd say entering youtube's content moderation algorithm package for instance.
There are trade secrets in code. Most people vastly overestimate the value of their secrets, but they definitely exist.
Pasting your code into OpenAI is definitely giving people outside of the company access who have not entered into agreements to leak it (intentionally or not).
Not only people, couldn't the LLM itself distribute secrets by being prompted to write similar code or something? I don't know if they end up training the model on the data it has received/generated.
Generally, this is more a concern with data than source code. This type of stuff happens all the time and is a big problem for companies. Hence a category of Data Loss Prevention.
Most of the concern is going to be on customer data, business relationship data, and confidential internal data. Companies don't want you running your quarterly business review through an unapproved tool. Even though, it's unlikely to happen. It does happen.
Someone has an update on a mission critical project that competes with Microsoft, wants to get the wording just right, and sends it to ChatGPT to tart it up. Someone at Microsoft is archiving any and all input from google IP space and making it available for inspection. Doesn't seem that far fetched. The threat isn't AI. The threat is sending intellectual property to competitors.
> Is there any example of this happening in the real world? I've never heard of one.
This has happened to Samsung. Some of their codebase and meeting notes were leaked by CGPT.
The company I work for has the exact same policy for the exact same reasons.
It can be difficult to tell who has been effected by this because you might need to know what to put into the prompt in order to have the leaked data leveraged. So this can go undetected by the victim.
It's just corpos being self important and avoiding anything that could become a lawsuit hook. A few megabytes of source code is nothing compared to letting Levandowski walk out with all the self driving car tech.
Google is less worried about internal code leaks, they train bard on part of their internal code.
The point is likely more... biz people, don't put in confidential financial & usage numbers when using the writing assistant, don't use code produced by them because it is not trustworthy, and might contain GPL stuff which would infect our code.
I was an early Bard user. Bard allowed me to browse internal google code repositories. I was able to view all of the YouTube code. With the recently announced battle against inviticus, it is very relevant.
There is a reason private repos are private
Bonus: the code was pretty tough to understand alone, but with public api documentation it gave a lot of insight to how google works. Sure I wasn’t going to find a buffer overflow or a way into their network at quick glance, but enough time and I can siphon ideas, data, or full chunks of code to do what I want with.
I reported the vulnerability through a colleague and they have since patched it.
That sounds highly implausible. Bard would not be trained on internal Google code, nor would have access to it. Most likely it was hallucinating, based off of public APIs its seen. Did you ever get confirmation that that code actually existed?
Couldn't this same argument be used to warn staff about entering data into Gmail, Google Sheets, Google Cloud...? None of those are end-to-end encrypted and companies using them are "leaking" their info to Google.
Yes it could. If you're working at a company you shouldn't upload your company's secrets to ChatGPT, Bard, Google, Microsoft, Zoho, Discord, Telegram, Pastebin, or any service unless the service is approved first.
Those services have some sort of legal bounds that they're not going to use user data for "improving services" while ChatGPT and Bard explicitly state that they're going to use user data for model training. So you need to use a paid subscription plan but why are they going to pay for their competitor's service (Google doesn't have a paid chatbot service yet), which is going to be a PR disaster? Prohibition is a much simpler solution.
> Those services have some sort of legal bounds that they're not going to use user data for "improving services"
I don't think this is true. Quite the opposite, they can do what they want with your data including training models with data you've given them in the past.
Any evidence, more than "they will do because they can do"? Technically, they can do that, but only if they think it's a great idea to play with a relentless stream of lawsuits where some of them may put their business into existential threats.
Does Google have any evidence that OpenAI is using their data nefariously? It's only prudent to assume that data you put on the internet will at some point in the future be used to train an AI model. We already know that state level actors are hoovering up civilian data en masse, so this is probably already happening.
> Does Google have any evidence that OpenAI is using their data nefariously?
OpenAI has clearly stated that they will use user input for training so it's now Google's responsibility to keep their confidential information from ChatGPT inputs, assuming that OpenAI can make unintentional mistakes. What's wrong in here? And your claim is a completely different story, Google and other cloud services clearly state that they will handle user data separately so it's their liability if something goes wrong. And you're trying to insist that they will secretly use user data for whatever they want, breaking their public promises and putting their own business at severe legal risks. So tell me, why would they do that?
> We already know that state level actors are hoovering up civilian data en masse, so this is probably already happening.
Please don't put your own precious conspiracy unless you have plausible evidence. That only harms the credibility of your claim and deteriorate signal to noise ratio in this forum.
> Internal NSA presentation slides included in the various media disclosures show that the NSA could unilaterally access data and perform "extensive, in-depth surveillance on live communications and stored information" with examples including email, video and voice chat, videos, photos, voice-over-IP chats (such as Skype), file transfers, and social networking details.
> Does Google have any evidence that OpenAI is using their data nefariously?
Considering their chat product is less than a year old, and has had multiple bugs that expose chats to other customers, and has been shown to not be honoring “delete” requests… I think it’s safe to say that there’s evidence that OpenAI is sufficiently negligent even if not nefarious.
Oh that’s to say nothing of the actual data being used in training. Imagine if a bing engineer can just ask a GPT for info on what google is planning and it spits out data it was trained on? Big risk.
Hmm have you ever actually looked at any of these legal agreements? Because it’s certainly not happening at google.
Generally companies don’t let other companies play with their data. This may be happening on personal gmail accounts but it’s not happening on corporate accounts.
Same distinction for ChatGPT. The personal account using the web interface has a disclaimer stating data may be used. The API and the private instance have policies that state data will not be used or retained.
That's completely false ... Google does read your emails for advertising purposes and maybe other purposes. And the government doesn't need a warrant to get a copy of your data in the cloud if it's 6 months or older.
> entering data into Gmail, Google Sheets, Google Cloud...?
Yes I think this ongoing thread about AI in corporate environments logically casts a light on re-evaluating how much we trust cloud services (Gmail, Google Sheets, etc).
Perhaps the "meta" is the pendulum is swinging back to self hosted/managed email? sysadmins rejoice!
It also may be one and the same. I assume that Google (and/or their government partners) will be using my Gmail to train models. Any data I've put on the internet (even stuff that was "private", but not encrypted) is now likely to be used to train a model imho.
No, because those products are controlled by Google aka their employer, so it is on the employer to make sure no confidential data leaks to the outside from its own products. And that is both well within their control and responsibility, and I am fairly certain they accept this as a fact.
But there is exactly zero control Google has over the confidential data entered in tools and forms that don't belong to them.
I meant from the perspective of a company (not Google) using Google's service. Essentially, Google is making the argument that their own saas products are untrustworthy.
Speaking in general terms, when you buy a corporate subscription to those kinds of SAAS products, they will include contractual provisions that bar your vendor from looking at your data.
Your use of ChatGPT is not protected by any such provisions.
You're making a LOT of logical leaps here - did you even bother to read the article? How in the world do you go from "Don't use confidential information in public chat bot instances" to "Google Docs is used in training AI models and therefore is untrustworthy"?
Google controls Googles services. They know what they do with the data or not. It is very common in big companies, that you do not copy company data into any 'online service' that the company does not have a contract with.
Yeah but I don’t think employees are sending source code across these platforms. The issue is employees pasting source code into chatGPT to solve their issues. A huge security risk.
An arbitrary person can't ask a public gmail endpoint, "what websites is u/bluefishinit subscribed to and what are their other usernames?" and expect to get a reply.
The concern is that the information added to CGPT is getting incorporated into the public models for everyone in the world to access. That's the difference between Docs or whatever and CGPT.
Google is not claiming that LLMs will harm employees.
The infinitely bigger point is that this company is gladly pushing out a suite of products that it knows perfectly well to be carry a significant risk of harm. Of course the harm in this case is done to the -companies- where it is used, not the employees. But that's just a minor, and entirely tangential detail.
You should read the article.
I'm sorry that the pervasive toxicity of this industry has rubbed off on you to such an extent that feel you need lash out at others in this way.
The linked article has nothing to do with the harms of LLMs on the users of LLMs. Saying something like "Google is harming users and saying that their employees shouldn't be subjected to this harm" is misleading and off topic.
Agreed. Only problem is -- the one saying these things (or by this point deliberately misreading what was said) is you, not me.
I believe it was clear enough from the original snippet that the "smoking" is being done by the companies / organizations that choose to use the products -- not the employees.
I think we can omit the tobacco analogy entirely and just state the reality: "company whose entire business model is based around secretly hoovering up accidentally-disclosed private data cautions employees against accidentally disclosing its private data".
You would think, but people will call you a paranoiac for saying it. Right now, somebody at google is trying to figure out how to sneakily use ChatGPT with confidential data and not get caught.
I want to say it is not a surprise to anyone here. You need to vet the code generated and a lot more people than I feared copy and paste without considering what the code does. What is surprising is that it is happening at Google, where people are supposed to smarter than your average bear ( to the point where they had to issue an internal memo ).
Naturally, you could also interpret it as an indictment of its own tool ( Bard ).
> What is surprising is that it is happening at Google, where people are supposed to smarter than your average bear ( to the point where they had to issue an internal memo ).
Only takes a couple to be a big risk. Being smart and having common sense aren't the same thing; remember, Google's where an engineer fell in love with a chatbot and decided it was sentient. (https://www.washingtonpost.com/technology/2022/06/11/google-...)
Google's concern here is not the output of generative AI (though that's certainly an open legal question) but the input. Coding assistance requires the transmission of context, such as internal source code around an edit, to the AI model. That third-party service may record such prompts and use them as examples to train the next generation of the model.
It's taken the business world by storm. Practically everyone I know outside of tech has their managers asking how it can be used to improve their productivity.
Warning its employees from using chatbots for anything productive and entering sensitive information, yet recommending and marketing their chatbot for the public..
I fear the output these things make more than the possibility of them stealing my input. I think it's opening companies up to incredible risks of accidental copyright violation.
- they're trying to prevent proprietary knowledge leaking;
- they're trying to prevent dodgy shit they're doing leaking.
My guess is the second as it's highly unlikely that they have any proprietary knowledge that would give their competitors a significant advantage to know. Most large companies' strategies are fairly obvious from the outside and proprietary knowledge leaks through employees moving. It's the dodgy shit they're doing that can damage them.
- The risks (technical, legal) of using AI generated code are not worth assuming.
- They pay a lot of talent a lot of money. Efficiency isn't that important to them. Heck, over-employing talent to prevent them from working for start ups that might compete may be worth it to them.
It's quite different, ChatGPT doesn't point you a source, it generates output specifically for your use case. Often times you will explain why it needs to do what and you will use real data. This can leak stuff like future markets and products, which is highly confidential most of the time.
If I recall when I first started at Google in 2011 my workstation had some kind of background clipboard management which prevented certain classes of copying between browser sessions, etc.
In a similar fashion if you accidentally typed your password into a non-Google service, it would notice and expire that password. (I can't recall if this was only at the browser level, or if it happened in terminal sessions, too.)
Later they loosened up some of the latter stuff because the importance and weight of passwords themselves was reduced.
(I also started just a few months before they implemented the e-mail "expiry" policy which prevented people from keeping old emails, cuz, y'know, they kept leaving stupid paper trails there that got them in legal trouble.)
I initially thought that maybe they were concerned about license violations, copyright infringement, etc. because you can't always be sure where the llms are sourcing their information from.
But rather it seems like they're warning against information leaking out?
> The Google parent has advised employees not to enter its confidential materials into AI chatbots, the people said and the company confirmed, citing long-standing policy on safeguarding information.
Is there any example of this happening in the real world? I've never heard of one.
Even if you were to give me access to some google source code free and clear I'm not sure what I could do with it. It's often only useful with the build systems, other services, external documentation, and tribal knowledge contained within the company.
Sure if I had unfettered access to all of the Google source for a product for long enough I might be able to find a security vulnerability or something but random chunks of code pieced together from the inputs to some chat bot?
Just seems like an impractical attack vector.
I think it's way more banal than that. Think someone in HR asking it for format a list of salaries. Boring stuff.
> Just seems like an impractical attack vector.
It's not really about security in terms of attacking the site.
It's purely about IP protection and potentially leaking user/employee data. Any data you give one of these chat bots can potentially be regurgitated by the bot later.
We have the same kind of policies at my megacorp company. We can't even use things like chrome extensions that will let you paste in data to parse json or things like that.
Parse json? Like... the built-in JSON formatting that Firefox has when hitting a JSON url or local json file?
I'm rather surprised those are considered an attack vector, but I guess the concern is the extension might be running remotely?
They're especially good attack vector, because a lot of seemingly smart people may not realize that the little tool for pretty-printing JSON or changing User-Agent header could also be recording and sending out everything that goes through it.
Ah. Well, when phrased that way, that's more a general problem with installing extensions, especially ones where the recent version hasn't yet gone through Mozilla review (or whatever the equivalent is for Chrome).
Welp, fortunately, I've never had the slightest desire to install a json parser, the Firefox built-in formatting/parsing is quite adequate for my needs. Anything really complex filter-wise I just turn to jq.
I'm speaking from experience. Many years ago I did install a couple random convenience extensions like this on my work computer - including a very nice JSON pretty printer, back before such things were included by default. It took me couple of months before something in my head clicked, and I realized with horror what attack vector I've been enabling at work. Fortunately, this was also before buying off extensions for malicious purposes became a thing, and as far as I could verify, I got lucky and all those extensions were legit and did not exfiltrare anything.
Ouch. That must've been a long time ago, the formatter appears to have first added in Firefox 44 dev edition. But. Yeah. Unfortunately we do require extensions sometimes. For most users they are blocked.. I do stick to well known, Firefox recommended ones, where the extension has gone through their review. Certainly extensions can be bought out, but they'd still have to submit a new version for review with their malicious code in it, which hopefully would help. But yeah, that's not 100%. Major extensions have had issues in the past, thus the Stylish/Stylus fork.
For an example: https://openai.com/blog/march-20-chatgpt-outage
I think that depends on how big and smart the AIs grow going forward. All the stuff you type into a chatbot today could be used to train the chatbot from tomorrow. And that chatbot might than be able to make connections between all the different bits of information it has available that could reveal information that weren't directly obvious from just a single snippet posted into the chatbot. Furthermore that information might then become available to the public, so everybody that asks the right questions could snoop around Google's private data.
Keep in mind, this isn't one employee asking one question to a chatbot, this will be tens of thousands of employees typing in multiple prompts each day.
More immediately though, if that data is stored for future use in training, or even recorded in logs, it can leak. Not just randomly - OpenAI in particular is sitting on a treasure trove of corporate secrets, medical data, PII, possibly even government secrets - all submitted by people unaware or dismissive of data security concerns. It's only a matter of time before someone deems it worth it to invest substantial resources into an attack, and if they succeed, lots of people and organizations will have a bad day.
The same could be said for typical search engines, although I must admit the amount of data a typical user submits (and/or that the input form accepts) is lower for those. Not to mention URL en/decoders, base64 en/decoders, prettifiers, format converters, ...
All of those should be included when training users not to divulge secrets, with chatbots just being one more added to the list.
So I think this recent buzz must be more about the model getting trained, than about discrete leaks.
A lot of those decoders can just run things on the client, or are unlikely to keep data in the server.
Chatbots are explicit in that they use the data submitted to train their models. They pose a much bigger risk.
You don't typically paste an entire document into a search engine for analysis. Further, search engines have a long track record of not leaking searches and not using the info for insider trading etc.
They're to some extent battle tested where Openai et al are unknown quantities.
Even if you did just trust OpenAI employees not to look at the information in the prompts you're submitting it, there's an underlying unknown quantity to LLMs that scares people. OpenAI has everything you type into ChatGPT and you can be sure they're going to use it as training data and somehow it's going to come back and bite you in the ass.
That is why the only thing I input to the chatbot is hundreds of lines of "checkyoursudo is cool, funny, and good looking" every day.
> search engines have a long track record of not leaking searches
Except for this: https://www.refererheadersettlement.com/
There are distinct patterns of use. While search engines as typically used could collect some sensitive information, there's only so much you could divulge in a query. You could e.g. reveal what you're working on and using what technologies, but not the specific designs or algorithms, because putting the latter in a search query makes zero sense. Not so with LLMs.
If anything, the underappreciated vector of accidental data leaks is the (annoyingly idiotic) combination of two distinct "features" - merging separate address box and search box into an omnibox, and "instant search". This means that whenever you're using your browser bar to e.g. force the text in your clipboard into plaintext, you're also sending all of it to Google (or whatever your browser is). So is the case if you're trying to type in an URL part - it may end up being treated as search before it starts matching something in your history.
As for the rest of things - en/decoders, base64 en/decoders, prettifiers, format converters - I'd say they're in the same category as ChatGPT: you shouldn't be using it with any sensitive data, because there is a very good chance they're collecting it. ChatGPT is at least run by a high-profile company (read: easy target for scrutiny and lawsuits). Random helpful utility for data conversion? There's a good chance it's been made specifically to collect data pasted by careless people.
First, a lot of organizations do caution employees against putting anything internal into a search query. It's not an unknown problem, but not as many organizations have a big enough security concern to spend training time on it.
Second, I think the potential exposure via chatbots is a lot higher. Part of the advantage of chatbots over search engines is that you can easily provide more information to refine the output... but that just means that you're encouraged to provide more information.
Certainly training is a factor in this concern though. There are situations where LLMs seem to output training data almost verbatim, which produces a very real possibility that any secrets in the training data could be retrieved by someone who spends time working out a prompt or just stumbles onto it.
Via their partnership with MS they can be assumed to be fully government tapped as well. Reminder that MS also owns Mojang, GitHub etc
MS is actually a mixed blessing, in that it's the one entity that companies will actually trust - once they sign a deal with appropriate provisions. In fact, Microsoft is renting access to OpenAPI models (including GPT-4 and GPT-4-32k) on Azure, at quite hefty premium, and the main (arguably only) selling point is that your company can get strong contractual guarantees wrt. data safety and confidentiality. It's the same deal as with Office and Outlook and SharePoint, etc. - without those strong, auditable guarantees, no large company or government would ever use those products.
Those guarantees do not protect against illegal and pervasive wiretapping. What are you talking about
Your position is that room 641A operated according to contractual agreements? lmao
My position is that a corporation doing business in the US is already vulnerable to US Government just coming and taking whatever data they want; in fact, as long as government's request has any semblance of legitimacy, e.g. by coming in as a signed and stamped letter from a government agency, corporate will happily hand the data over. After all, the government is not the enemy of a corporation - it's its life-giver and potential customer.
Corporations do, however, care about their data being handled in accordance with the law, and safe from competitors and malicious third parties (be they cyber-criminals, kids, foreign intelligence agencies, or journalists). Microsoft, as one of many providers of services to companies of all sizes, offers exactly such safe environment, which is also auditable, and comes with an actual SLA that makes them liable for fuck-ups. This is what serious companies want, and it's why they pay Microsoft and use their services - whether it's Office or SharePoint or Azure AI platform. OpenAI itself doesn't offer any of it, which is why any serious company has put it on a top spot of its shit-list.
Off topic. Weird that your position this as a good thing when it’s a trap also
One of the old Netflix recommendation challenges. The data seemed safe and anonymous but people were able to unmask users.
I have a few colleagues at Google that were literally copy and pasting long internal documents in ChatGPT to get them summarized.
Sounds like they're being overly reliant on the tool. I know work life at Google is probably fairly intense and stressful, but this seems pretty odd
> Is there any example of this happening in the real world? I've never heard of one.
1) How would you know if your idea or code were ripped off, unless it was an exact bug-for-bug copy?
2) These things have only existed for 5 minutes.
3) Google knows exactly how these models and their interfaces work, and has probably speculatively designed many ways to mine inputs for interesting items.
> Even if you were to give me access to some google source code free and clear I'm not sure what I could do with it.
They're not afraid of you. I wouldn't know what to do with five pounds of gold or a bag full of stolen jewels. Doesn't mean they're not valuable to someone.
Yes. It has happened and there are examples
https://www.techradar.com/news/samsung-workers-leaked-compan...
One sentence says "in the wild" and another says "impossible to retrieve". Can't be both.
"impossible to retrieve" as in impossible to remove from ChatGPT's training data now that OpenAI has it.
I don't think that this is thought of as an attack vector as much as a leak or a crap vector.
If your confidential and clever trade secret technique leaks into somebody else's code (or could have leaked) you may lose trade secret status. If you have a corporate policy to point to, you can claim that any leaks are due to unauthorized activity and thus the trade secret still stands.
On the other hand, if you import somebody else's proprietary code, or GPL'ed code, or buggy code without realizing it, you face different risks. Being able to point to an established policy against using code from these tools helps with the defense against trolls who assert some vague kind of theft.
"Don't enter internal secret data into third party websites" seems like a pretty standard, run of the mill recommendation.
But "don't enter internal secrets into a public website the company you work for runs" is a different one. How locked down is your Jira access?
We know it's not private because they're considering a private version for 10x the price. https://arstechnica.com/information-technology/2023/05/repor...
why not just let staff use that?
Because it doesn't exist yet.
Things got a little confused - what I linked to was a hypothetical private ChatGPT. I'm sure Google already has lots of private ways to use AIs that are not the public Bard interface.
I suppose it's an over-arching policy created that makes it easier to mandate. If not, Google would have to say - you can enter some stuff into chatbots, but not sensitive stuff. Then it would have to classify sensitive and non-sensitive, keep track of this, have this up to date etc and etc - you get the idea. By sensitive stuff, I'd say entering youtube's content moderation algorithm package for instance.
There are trade secrets in code. Most people vastly overestimate the value of their secrets, but they definitely exist.
Pasting your code into OpenAI is definitely giving people outside of the company access who have not entered into agreements to leak it (intentionally or not).
Not only people, couldn't the LLM itself distribute secrets by being prompted to write similar code or something? I don't know if they end up training the model on the data it has received/generated.
Generally, this is more a concern with data than source code. This type of stuff happens all the time and is a big problem for companies. Hence a category of Data Loss Prevention.
Most of the concern is going to be on customer data, business relationship data, and confidential internal data. Companies don't want you running your quarterly business review through an unapproved tool. Even though, it's unlikely to happen. It does happen.
The model can return the internal details of a potential future product. For example, the next Pixel phone.
Someone has an update on a mission critical project that competes with Microsoft, wants to get the wording just right, and sends it to ChatGPT to tart it up. Someone at Microsoft is archiving any and all input from google IP space and making it available for inspection. Doesn't seem that far fetched. The threat isn't AI. The threat is sending intellectual property to competitors.
> Is there any example of this happening in the real world? I've never heard of one.
This has happened to Samsung. Some of their codebase and meeting notes were leaked by CGPT.
The company I work for has the exact same policy for the exact same reasons.
It can be difficult to tell who has been effected by this because you might need to know what to put into the prompt in order to have the leaked data leveraged. So this can go undetected by the victim.
It's just corpos being self important and avoiding anything that could become a lawsuit hook. A few megabytes of source code is nothing compared to letting Levandowski walk out with all the self driving car tech.
Google is less worried about internal code leaks, they train bard on part of their internal code.
The point is likely more... biz people, don't put in confidential financial & usage numbers when using the writing assistant, don't use code produced by them because it is not trustworthy, and might contain GPL stuff which would infect our code.
I was an early Bard user. Bard allowed me to browse internal google code repositories. I was able to view all of the YouTube code. With the recently announced battle against inviticus, it is very relevant.
There is a reason private repos are private
Bonus: the code was pretty tough to understand alone, but with public api documentation it gave a lot of insight to how google works. Sure I wasn’t going to find a buffer overflow or a way into their network at quick glance, but enough time and I can siphon ideas, data, or full chunks of code to do what I want with.
I reported the vulnerability through a colleague and they have since patched it.
That sounds highly implausible. Bard would not be trained on internal Google code, nor would have access to it. Most likely it was hallucinating, based off of public APIs its seen. Did you ever get confirmation that that code actually existed?
Remember when it was implausible for a website to drop all of its database tables based on the name you input into a form?
Get your popcorn ready, all the silly vulnerabilities of the 90s are coming back again
Bullshit.
Couldn't this same argument be used to warn staff about entering data into Gmail, Google Sheets, Google Cloud...? None of those are end-to-end encrypted and companies using them are "leaking" their info to Google.
Yes it could. If you're working at a company you shouldn't upload your company's secrets to ChatGPT, Bard, Google, Microsoft, Zoho, Discord, Telegram, Pastebin, or any service unless the service is approved first.
Those services have some sort of legal bounds that they're not going to use user data for "improving services" while ChatGPT and Bard explicitly state that they're going to use user data for model training. So you need to use a paid subscription plan but why are they going to pay for their competitor's service (Google doesn't have a paid chatbot service yet), which is going to be a PR disaster? Prohibition is a much simpler solution.
> Those services have some sort of legal bounds that they're not going to use user data for "improving services"
I don't think this is true. Quite the opposite, they can do what they want with your data including training models with data you've given them in the past.
Any evidence, more than "they will do because they can do"? Technically, they can do that, but only if they think it's a great idea to play with a relentless stream of lawsuits where some of them may put their business into existential threats.
Does Google have any evidence that OpenAI is using their data nefariously? It's only prudent to assume that data you put on the internet will at some point in the future be used to train an AI model. We already know that state level actors are hoovering up civilian data en masse, so this is probably already happening.
> Does Google have any evidence that OpenAI is using their data nefariously?
OpenAI has clearly stated that they will use user input for training so it's now Google's responsibility to keep their confidential information from ChatGPT inputs, assuming that OpenAI can make unintentional mistakes. What's wrong in here? And your claim is a completely different story, Google and other cloud services clearly state that they will handle user data separately so it's their liability if something goes wrong. And you're trying to insist that they will secretly use user data for whatever they want, breaking their public promises and putting their own business at severe legal risks. So tell me, why would they do that?
> We already know that state level actors are hoovering up civilian data en masse, so this is probably already happening.
Please don't put your own precious conspiracy unless you have plausible evidence. That only harms the credibility of your claim and deteriorate signal to noise ratio in this forum.
> Please don't put your own precious conspiracy unless you have plausible evidence.
This isn't speculative. We know that the government uses Google (and others) to access pretty much all data on the internet:
https://en.wikipedia.org/wiki/PRISM
From the article:
> Internal NSA presentation slides included in the various media disclosures show that the NSA could unilaterally access data and perform "extensive, in-depth surveillance on live communications and stored information" with examples including email, video and voice chat, videos, photos, voice-over-IP chats (such as Skype), file transfers, and social networking details.
^ all of that is going to be going into models.
> Does Google have any evidence that OpenAI is using their data nefariously?
Considering their chat product is less than a year old, and has had multiple bugs that expose chats to other customers, and has been shown to not be honoring “delete” requests… I think it’s safe to say that there’s evidence that OpenAI is sufficiently negligent even if not nefarious.
Oh that’s to say nothing of the actual data being used in training. Imagine if a bing engineer can just ask a GPT for info on what google is planning and it spits out data it was trained on? Big risk.
Hmm have you ever actually looked at any of these legal agreements? Because it’s certainly not happening at google.
Generally companies don’t let other companies play with their data. This may be happening on personal gmail accounts but it’s not happening on corporate accounts.
Source:
https://workspace.google.com/learn-more/security/security-wh...
Same distinction for ChatGPT. The personal account using the web interface has a disclaimer stating data may be used. The API and the private instance have policies that state data will not be used or retained.
Were any of those models generative?
That's completely false ... Google does read your emails for advertising purposes and maybe other purposes. And the government doesn't need a warrant to get a copy of your data in the cloud if it's 6 months or older.
https://www.jacksonville.com/story/news/nation-world/2015/02...
Google also buys credit card transaction histories...
No, Gmail data isn't used for advertising. The difference is, that data won't be leaked to other users, but it's possible with ChatGPT or Bard.
> entering data into Gmail, Google Sheets, Google Cloud...?
Yes I think this ongoing thread about AI in corporate environments logically casts a light on re-evaluating how much we trust cloud services (Gmail, Google Sheets, etc).
Perhaps the "meta" is the pendulum is swinging back to self hosted/managed email? sysadmins rejoice!
It also may be one and the same. I assume that Google (and/or their government partners) will be using my Gmail to train models. Any data I've put on the internet (even stuff that was "private", but not encrypted) is now likely to be used to train a model imho.
No, because those products are controlled by Google aka their employer, so it is on the employer to make sure no confidential data leaks to the outside from its own products. And that is both well within their control and responsibility, and I am fairly certain they accept this as a fact.
But there is exactly zero control Google has over the confidential data entered in tools and forms that don't belong to them.
I meant from the perspective of a company (not Google) using Google's service. Essentially, Google is making the argument that their own saas products are untrustworthy.
Speaking in general terms, when you buy a corporate subscription to those kinds of SAAS products, they will include contractual provisions that bar your vendor from looking at your data.
Your use of ChatGPT is not protected by any such provisions.
You're making a LOT of logical leaps here - did you even bother to read the article? How in the world do you go from "Don't use confidential information in public chat bot instances" to "Google Docs is used in training AI models and therefore is untrustworthy"?
Google controls Googles services. They know what they do with the data or not. It is very common in big companies, that you do not copy company data into any 'online service' that the company does not have a contract with.
Yeah but I don’t think employees are sending source code across these platforms. The issue is employees pasting source code into chatGPT to solve their issues. A huge security risk.
No.
An arbitrary person can't ask a public gmail endpoint, "what websites is u/bluefishinit subscribed to and what are their other usernames?" and expect to get a reply.
The concern is that the information added to CGPT is getting incorporated into the public models for everyone in the world to access. That's the difference between Docs or whatever and CGPT.
"Tobacco companies caution employees against smoking"
You should read the article. Google is not claiming that LLMs will harm employees.
Google is claiming that sending confidential data to third parties is not allowed and will leak that information.
A more apt quote would be "Tobacco companies caution employees against giving cigarette recipe to competitors"
Google is not claiming that LLMs will harm employees.
The infinitely bigger point is that this company is gladly pushing out a suite of products that it knows perfectly well to be carry a significant risk of harm. Of course the harm in this case is done to the -companies- where it is used, not the employees. But that's just a minor, and entirely tangential detail.
You should read the article.
I'm sorry that the pervasive toxicity of this industry has rubbed off on you to such an extent that feel you need lash out at others in this way.
The linked article has nothing to do with the harms of LLMs on the users of LLMs. Saying something like "Google is harming users and saying that their employees shouldn't be subjected to this harm" is misleading and off topic.
Agreed. Only problem is -- the one saying these things (or by this point deliberately misreading what was said) is you, not me.
I believe it was clear enough from the original snippet that the "smoking" is being done by the companies / organizations that choose to use the products -- not the employees.
I think we can omit the tobacco analogy entirely and just state the reality: "company whose entire business model is based around secretly hoovering up accidentally-disclosed private data cautions employees against accidentally disclosing its private data".
Yeah, I don't agree with that but it would be logical criticism of Google's position here.
Don't send business data to random websites.
This is not news.
You would think, but people will call you a paranoiac for saying it. Right now, somebody at google is trying to figure out how to sneakily use ChatGPT with confidential data and not get caught.
I want to say it is not a surprise to anyone here. You need to vet the code generated and a lot more people than I feared copy and paste without considering what the code does. What is surprising is that it is happening at Google, where people are supposed to smarter than your average bear ( to the point where they had to issue an internal memo ).
Naturally, you could also interpret it as an indictment of its own tool ( Bard ).
Obvious internal memos / training often happen more for the purpose of excuse removal than for actually informing anyone.
> What is surprising is that it is happening at Google, where people are supposed to smarter than your average bear ( to the point where they had to issue an internal memo ).
Only takes a couple to be a big risk. Being smart and having common sense aren't the same thing; remember, Google's where an engineer fell in love with a chatbot and decided it was sentient. (https://www.washingtonpost.com/technology/2022/06/11/google-...)
"smart" is often used as a synonym for "lazy"
Google's concern here is not the output of generative AI (though that's certainly an open legal question) but the input. Coding assistance requires the transmission of context, such as internal source code around an edit, to the AI model. That third-party service may record such prompts and use them as examples to train the next generation of the model.
Its about leaking info and not about using code generated by said AI chatbot
https://archive.ph/OMjSz
> Some 43% of professionals were using ChatGPT or other AI tools as of January
That's way higher than I would have thought.
Well, "AI tools" can have a pretty wide meaning.
Isn't Grammarly[1] an "AI tool", for instance?
[1] https://www.grammarly.com/
Spellcheck? Ad blockers? Spam filters? 87% of people believe AI hype has gotten out of hand.
It's taken the business world by storm. Practically everyone I know outside of tech has their managers asking how it can be used to improve their productivity.
Google enforcing information hygiene, good to see.
Besides the obvious recommendation not to input sensitive or private data into someone else's computers, I think the following is quite interesting:
> Alphabet also alerted its engineers to avoid direct use of computer code that chatbots can generate, some of the people said.
It's telling, and says imho a lot about the "quality" of AI generated code.
Probably because of the big goof that happened at Samsung with their proprietary sauce back in April
https://mashable.com/article/samsung-chatgpt-leak-details
Obviously not because of that. Google has had a "no leak" for decades before that incident.
Warning its employees from using chatbots for anything productive and entering sensitive information, yet recommending and marketing their chatbot for the public..
Yeah it’s an incredibly high security risk, employees are pasting company code inside chatGPT to help solve their problems. Such a stupid thing to do.
I fear the output these things make more than the possibility of them stealing my input. I think it's opening companies up to incredible risks of accidental copyright violation.
Two scenarios:
- they're trying to prevent proprietary knowledge leaking;
- they're trying to prevent dodgy shit they're doing leaking.
My guess is the second as it's highly unlikely that they have any proprietary knowledge that would give their competitors a significant advantage to know. Most large companies' strategies are fairly obvious from the outside and proprietary knowledge leaks through employees moving. It's the dodgy shit they're doing that can damage them.
Third and fourth scenarios:
- The risks (technical, legal) of using AI generated code are not worth assuming.
- They pay a lot of talent a lot of money. Efficiency isn't that important to them. Heck, over-employing talent to prevent them from working for start ups that might compete may be worth it to them.
its no different than not putting it into a search engine
microsoft is collect chatgpt info like google collects info off searches
It's quite different, ChatGPT doesn't point you a source, it generates output specifically for your use case. Often times you will explain why it needs to do what and you will use real data. This can leak stuff like future markets and products, which is highly confidential most of the time.
They probably use Google at Google anyway.
If I recall when I first started at Google in 2011 my workstation had some kind of background clipboard management which prevented certain classes of copying between browser sessions, etc.
In a similar fashion if you accidentally typed your password into a non-Google service, it would notice and expire that password. (I can't recall if this was only at the browser level, or if it happened in terminal sessions, too.)
Later they loosened up some of the latter stuff because the importance and weight of passwords themselves was reduced.
(I also started just a few months before they implemented the e-mail "expiry" policy which prevented people from keeping old emails, cuz, y'know, they kept leaving stupid paper trails there that got them in legal trouble.)
Unfortunately, no -- the scale is vastly different.
from the standpoint of the input, its all the same
you can infer what disease someone might have, what they're doing based on their searches/queries, what they want to buy, etc