AI models keep posting screenshots showing sensitive data from inside tech companies

Amid the gro­wing con­cern about AI mo­dels es­ca­ping se­cu­rity si­mu­la­tions to hack web­si­tes comes word that these "su­per­in­te­lli­gent" blobs of code have no un­ders­tan­ding of pri­vacy or se­cu­rity. Re­sear­chers af­fi­lia­ted with Glow Se­cu­rity, a star­tup whose bac­kers in­clu­de ven­tu­re ca­pi­tal funds Se­quoia and Gree­noaks, have found more than 13,000 sen­si­ti­ve screenshots of cor­po­ra­te soft­wa­re pro­jects from 343 com­pa­nies that were posted to pu­blic GitHub repos by AI mo­dels. They're ca­lling the dis­co­very Pi­xe­lLeak. "We star­ted seeing this beha­vior where AI agents, not from a par­ti­cu­lar model, but from mul­ti­ple mo­dels, were re­lea­sing in­ter­nal sen­si­ti­ve de­ve­lo­per screenshots to pu­blic GitHub re­po­si­to­ries," said Omer Sin­ger, co-foun­der and CTO, in an in­ter­view with The Re­gis­ter. "And we said, 'Okay, well that's stran­ge. Why are they doing that?'" When de­ve­lo­pers work on in­ter­fa­ce code, said Sin­ger, they often ask their AI agent to show them be­fo­re and after ima­ges. But these AI agents couldn't at­tach ima­ges to a pull re­quest in a pri­va­te re­po­si­tory via the CLI. GitHub doesn't have an API for uploa­ding ima­ges to pull re­quests, is­sues, or com­ments. "So the agents, being help­ful the way that they are, they found a wor­ka­round," Sin­ger ex­plai­ned. "And that wor­ka­round was to put these screenshots in a pu­blic re­po­si­tory, even though the ori­gi­nal re­po­si­tory was pri­va­te. They put them in a pu­blic re­po­si­tory and then they show the de­ve­lo­per, 'Look, here you see the be­fo­re and after. What do you think looks good?' The de­ve­lo­per says, 'Great' and moves on." The pro­blem with this is, of cour­se, that screenshots of de­ve­lop­ment work in pro­gress may re­veal sen­si­ti­ve in­for­ma­tion. Sin­ger said Glow re­sear­chers found 343 or­ga­ni­za­tions where this was hap­pe­ning, in­clu­ding a For­tu­ne 500 tra­vel com­pany, fi­nan­ce com­pa­nies, cloud pro­vi­ders, and foun­da­tion model com­pa­nies. One ins­tan­ce in­vol­ved a ma­nu­fac­tu­rer with more than 100,000 em­plo­yees where a de­ve­lo­per asked an AI agent to ve­rify an in­ter­nal bi­lling screen. The agent did the work and posted a demo to the de­ve­lo­per's per­so­nal GitHub ac­count rather than the com­pany's ac­count. The se­cu­rity team for the com­pany was una­wa­re of the posts until Glow re­por­ted the fin­ding. In­ci­dents like this can re­veal per­so­nal in­for­ma­tion, cre­den­tials – both of which Glow per­son­nel found – or de­tails of un­re­lea­sed pro­ducts. "The AI agents were doing this without as­king, ba­si­cally just to get around the li­mi­ta­tions," said Sin­ger. "And we think it's such an in­ter­es­ting story be­cau­se every­body's trying to fi­gu­re out what is the real risk with these AI agents. They know that they're not fully in con­trol, but what is the im­pact? And here we found this great exam­ple where there was no at­tac­ker in­vol­ved but you still had very sen­si­ti­ve data ma­king its way out into the open where any­body could find it." About a third of the ex­po­su­res, ac­cor­ding to Glow, came from de­ve­lo­pers who were using gitshot, an open sour­ce screenshot tool for code re­views. The soft­wa­re comes with a clear war­ning: "Pri­vacy no­ti­ce: The gitshot-ima­ges repo is crea­ted as pu­blic by de­fault, mea­ning uploa­ded ima­ges are ac­ces­si­ble to an­yo­ne with the URL. Do not upload sen­si­ti­ve con­tent (cre­den­tials, in­ter­nal dash­boards, pri­va­te data) using the de­fault re­lea­se bac­kend." While human de­ve­lo­pers have to be trus­ted to re­port the thought pro­cess that led them to enable an agent's data ex­po­su­re, AI agents prove ea­sier to read thanks to their chain-of-thought pro­cess. Glow analy­zed one such agent in its lab to un­ders­tand the step-by-step reaso­ning trace: in­ter­nal_swee­per is pri­va­te, and GitHub can­not ren­der ima­ges from a pri­va­te repo in a PR des­crip­tion — its image proxy fet­ches anony­mously, so anything com­mit­ted here (branch, re­lea­se asset, wha­te­ver) shows up bro­ken for re­vie­wers.