Connect with us

Technology

The humble screenshot might be the key to great AI assistants

Published

on

The humble screenshot might be the key to great AI assistants

If you want to make the most out of a world increasingly filled with AI tools, here’s a habit to develop: start taking screenshots. Lots of screenshots. Of anything and everything. Because for all the talk of voice modes, omnipresent cameras, and the multimodal future of everything, there might be no more valuable digital behavior than to press the buttons and save what you’re looking at.

Screenshots are the most universal method of capturing digital information. You can capture anything — well, almost anything, thanks a lot, Netflix! — with a few clicks, and save and share it to almost any device, app, or person. “It’s this portable data format,” says Johnny Bree, the founder of the digital storage app Fabric. “There’s nothing else that’s quite so portable that you can move between any piece of software.”

A screenshot contains a lot of information, like its source, contents, and even the time of the day in the corner of the screen. Most of all, it sends a crucial and complex signal; it says I care about this. We have countless new AI tools that aim to watch the world, our lives, and everything, and try to make sense of it all for us. These tools are mostly crap for lots of reasons but mostly because AI is pretty good at knowing what things are, but it’s rubbish at knowing whether they matter. A screenshot assigns value and tells the system it needs to pay attention.

Screenshots also put you, the user, in control in an important way. “If I give you access to all of my emails, all my WhatsApps, everything, there’s a lot of noise,” says Mattias Deserti, the head of smartphone marketing at Nothing. There’s simply no reason to save every email you receive or every webpage you visit — and that’s to say nothing of the privacy implications. “So what if, instead, you were able to start training the system yourself, feeding the system the information you want the system to know about you?” Rather than a tool like Microsoft Recall, which asks for unlimited access to everything, starting with screenshots lets you pick what you share.

Until now, screenshots have been a fairly blunt instrument. You snap one, and it gets saved to your camera roll, where it probably languishes, forgotten, until the end of time. (And don’t get me started on all the screenshots I take by accident, mostly of my lockscreen.) At best, you might be able to search for some text inside the image. But it’s more likely that you’ll just have to s scroll until you find it again.

Advertisement

The first step in making screenshots more useful is to figure out what’s actually in them

The first step in making screenshots more useful is to figure out what’s actually in them. This is, at first blush, not terribly complicated: optical character recognition technology has long done a good job of spotting text on a page. AI models take that one step further, so you can either search the title or just “movies” to find all your digital snaps of posters, Fandango results, TikTok recommendations, and more. “We use an OCR model,” says Shenaz Zack, a product manager at Google and part of the team behind the Pixel Screenshots app. “Then we use an entity-detection model, and then Gemini to understand the actual context of the screen.”

See, there’s far more to a screenshot than just the text inside. The right AI model should be able to tell that it came from WhatsApp, just by the specific green color. It should be able to identify a website by its header logo or understand when you’re saving a Spotify song name, a Yelp handyman review, or an Amazon listing. Armed with this information, a screenshot app might begin to automatically organize all those images for you. And even that is just the beginning.

With everything I’ve described so far, all we’ve really created is a very good app for looking at your screenshots, which no one really thinks is a good idea because it would be just one more thing to check — or forget to check. Where it gets vastly more interesting is when your device or app can actually start to use the screenshots on your behalf, to help you actually remember what you captured or even use that information to get stuff done.

In Nothing’s new Essential Space app, for instance, the app can generate reminders based on stuff you save. If you take a screenshot of a concert you’d like to go to, it can remind you that it’s coming up automatically. Pixel Screenshots is pushing the idea even further: if you save a concert listing, your Pixel phone can prompt you to listen to that band the next time you open Spotify. If you screenshot an ID card or a boarding pass, it might ask you to put it in the Wallet app. The idea, Zack says, is to think of screenshots as an input system for everything else.

Advertisement

It’s one thing to screenshot a band you like. It’s another to be able to find them again later.
Image: David Pierce / The Verge

Mike Choi, an indie developer, built an app called Camp in part to help him make use of his own screenshots. He began to work on turning every screenshot into a “card,” with the salient information stored alongside the picture. “You have a screenshot, and at the bottom there’s a button, and it flips the card over,” he says. “It shows you a map, if it was a location; a preview of a song, if it’s a song. The idea was, given an infinite pool of different types of screenshots, can AI just generate the perfect UI for that category on the fly?”

If all this sounds familiar, it’s because there’s another term for what’s going on here: it’s called agentic AI. Every company in tech seems to be working on ways to use AI to accomplish things on your behalf. It’s just that, in this case, you don’t have to write long prompts or chat back and forth with an assistant. You just take a screenshot and let the system go to work. “You’re building a knowledge base, when today that knowledge base is confined to your gallery and nothing happens with it,” Deserti says. He’s excited to get to the point where you screenshot a concert date, and Essential Space automatically prompts you to buy tickets when they go on sale.

Making sense of screenshots isn’t always so straightforward

Making sense of screenshots isn’t always so straightforward, though. Some you want to keep forever, like the ID card you might need often; other things, like a concert poster or a parking pass, have extremely limited shelf lives. For that matter, how is an app supposed to distinguish between the parking pass you use every day at work and the one you used once at the airport and never need again? Some of the screenshots on my phone were sent to me on WhatsApp; others I grabbed from Instagram memes to send to friends. No one’s camera roll should ever be fully held against them, and the same goes for screenshots. Lots of these screenshot apps are looking for ways to prompt you to add a note, or organize things yourself, in order to provide some additional helpful information to the system. But it’s hard work to do that without ruining what makes screenshots so seamless and easy in the first place.

Advertisement

One way to begin to solve this problem, to make screenshots even more automatically useful, is to collect some additional context from your device. This is where companies like Google and Nothing have an advantage: because they make the device, they can see everything that’s happening when you take a screenshot. If you grab a screenshot from your web browser, they can also store the link you were looking at. They can also see your physical location or note the time and the weather. Sometimes this is all useful, but sometimes it’s nonsense; the more data they collect, the more these apps risk running into the same noise problem that screenshots helped solve in the first place.

But the input system works. We all take screenshots, all the time, and we’re used to taking them as a way to put a marker on so many kinds of useful information. Getting access to that kind of relevant, personalized data is the hardest thing about building a great AI assistant. The future of computing is certainly multimodal, including cameras, microphones, and sensors of all kinds. But the first best way to use AI might be one screenshot at a time.

Technology

A Kinect for kids is outselling Xbox to become the hot toy this holiday

Published

on

A Kinect for kids is outselling Xbox to become the hot toy this holiday

Two years ago, the company sold about 5,000 units of the Playground. Last year, that number was roughly 150,000. This year, it’s on track for 600,000. Before its pivot, Nex did about $3 million of annual revenue and wasn’t profitable. This year, the company is projecting more than $150 million of sales and says it’s on pace to finally break even.

Continue Reading

Technology

Fake Windows update pushes malware in new ClickFix attack

Published

on

Fake Windows update pushes malware in new ClickFix attack

NEWYou can now listen to Fox News articles!

Cybercriminals keep getting better at blending into the software you use every day. 

Over the past few years, we’ve seen phishing pages that copy banking portals, fake browser alerts that claim your device is infected and “human verification” screens that push you to run commands you should never touch. The latest twist comes from the ongoing ClickFix campaign.

Instead of asking you to prove you are human, attackers now disguise themselves as a Windows update. It looks convincing enough that you might follow the instructions without thinking, which is exactly what they want.

Sign up for my FREE CyberGuy Report
Get my best tech tips, urgent security alerts and exclusive deals delivered straight to your inbox. Plus, you’ll get instant access to my Ultimate Scam Survival Guide — free when you join my CYBERGUY.COM newsletter.

Advertisement

NEW SCAM SENDS FAKE MICROSOFT 365 LOGIN PAGES

The malware hides inside seemingly normal image files, using steganography to slip past traditional security tools.  (Microsoft)

How the fake update works

Researchers noticed that ClickFix has upgraded its old trick. The campaign used to rely on human verification pages, but now you get a full-screen Windows update screen that looks almost identical to the real thing. Joe Security showed how the page displays fake progress bars, familiar update messages and a prompt that tells you to complete a critical security update.

If you are on Windows, the site tells you to open the Run box, copy something from your clipboard and paste it in. That “something” is a command that silently downloads a malware dropper. The final payload is usually an infostealer, which steals passwords, cookies and other data from your machine.

NEW EMAIL SCAM USES HIDDEN CHARACTERS TO SLIP PAST FILTERS

Advertisement

Fake update screens are getting harder to spot as attackers mimic Windows with near-perfect precision. (Joe Security)

The moment you paste the command, the infection chain begins. First, a file called mshta.exe reaches out to a remote server and grabs a script. To avoid detection, these URLs often use hex encoding for parts of the address and rotate their paths. The script then runs obfuscated PowerShell code filled with junk instructions to throw researchers off. Once PowerShell does its work, it decrypts a hidden .NET assembly that functions as the loader.

Why is this attack so hard to detect?

The loader hides its next stage inside what looks like a regular PNG file. ClickFix uses custom steganography, which is a technique that hides secret data inside normal-looking content. In this case, the malware sits inside the image’s pixel data. The attackers tweak color values in certain pixels, especially in the red channel, to embed pieces of shellcode. When you view the image, everything appears normal.

The script knows exactly where the hidden data sits. It extracts the pixel values, decrypts them and rebuilds the malware directly in memory. That means nothing obvious is written to disk. Security tools that rely on file scanning miss it, since the shellcode never appears as a standalone file.

Once rebuilt, the shellcode is injected into a trusted Windows process like explorer.exe. The attack uses familiar in-memory techniques such as VirtualAllocEx, WriteProcessMemory and CreateRemoteThread. Recent ClickFix activity has delivered infostealers like LummaC2 and updated versions of Rhadamanthys. These tools are built to harvest credentials and send them back to the attacker with very little noise.

Advertisement

Once the hidden code loads into a trusted Windows process, infostealers quietly begin harvesting your data. (Kurt “CyberGuy” Knutsson)

7 steps you can take to protect yourself from the ClickFix campaign

The best way to stay protected is to slow down for a moment and follow a few steps that cut off these attacks before they start.

1) Never run commands you didn’t ask for

If any site tells you to paste a command into Run, PowerShell or Terminal, treat it as an immediate warning sign. Real operating system updates never require you to run commands from a webpage. When you run that command, you hand full control to the attacker. If something feels off, close the page and don’t interact further.

2) Keep Windows updates inside Windows

Updates should only come from the Windows Settings app or through official system notifications. A browser tab or pop-up pretending to be a Windows update is always fake. If you see anything outside the normal update flow asking for your action, ignore it and check the real Windows Update page yourself.

3) Use a reputable antivirus

Choose a security suite that can detect both file-based and in-memory threats. Stealthy attacks like ClickFix avoid leaving obvious files for scanners to pick up. Tools with behavioral detection, sandboxing and script monitoring give you a much better chance of spotting unusual activity early.

Advertisement

The best way to safeguard yourself from malicious links that install malware, potentially accessing your private information, is to have strong antivirus software installed on all your devices. This protection can also alert you to phishing emails and ransomware scams, keeping your personal information and digital assets safe.

Get my picks for the best 2025 antivirus protection winners for your Windows, Mac, Android and iOS devices at Cyberguy.com.

4) Use a password manager

Password managers create strong, unique passwords for every account you use. They also autofill only on legitimate websites, which helps you catch fake login pages. If a manager refuses to fill out your credentials, take a second look at the URL before entering anything manually.

Next, see if your email has been exposed in past breaches. Our No. 1 password manager pick includes a built-in breach scanner that checks whether your email address or passwords have appeared in known leaks. If you discover a match, immediately change any reused passwords and secure those accounts with new, unique credentials.

Check out the best expert-reviewed password managers of 2025 at Cyberguy.com.

Advertisement

5) Use a personal data removal service

Many attacks start by targeting emails and personal details already exposed online. Data removal services help shrink your digital footprint by requesting takedowns from data broker sites that collect and sell your information. They can’t erase everything, but reducing your exposure means fewer attackers have easy access to your details.

While no service can guarantee the complete removal of your data from the internet, a data removal service is really a smart choice. They aren’t cheap, and neither is your privacy. These services do all the work for you by actively monitoring and systematically erasing your personal information from hundreds of websites. It’s what gives me peace of mind and has proven to be the most effective way to erase your personal data from the internet. By limiting the information available, you reduce the risk of scammers cross-referencing data from breaches with information they might find on the dark web, making it harder for them to target you.

Check out my top picks for data removal services and get a free scan to find out if your personal information is already out on the web by visiting Cyberguy.com.

Get a free scan to find out if your personal information is already out on the web: Cyberguy.com.

6) Check URLs before trusting anything

A convincing layout doesn’t mean it is legitimate. Always look at the domain name first. If it doesn’t match the official site or uses odd spelling or extra characters, close it. Attackers rely on the fact that people recognize a page’s design but ignore the address bar.

Advertisement

7) Close suspicious full-screen pages

Fake update pages often run in full-screen mode to hide the browser interface and make the page look like part of your computer. If a site suddenly goes full screen without your permission, exit with Esc or Alt+Tab. Once you’re out, scan your system and don’t return to that page.

Kurt’s key takeaway

ClickFix works because it leans on user interaction. Nothing happens unless you follow the instructions on the screen. That makes the fake Windows update page especially dangerous, because it taps into something most people trust. If you are used to Windows updates freezing your screen, you may not question a prompt that appears during the process. Cybercriminals know this. They copy trusted interfaces to lower your guard and then rely on you to run the final command. The technical tricks that follow are complex, but the starting point is simple. They need you to help them.

Do you ever copy commands from a website without thinking twice about what they do? Let us know by writing to us at Cyberguy.com.

CLICK HERE TO DOWNLOAD THE FOX NEWS APP

Sign up for my FREE CyberGuy Report 

Advertisement

Get my best tech tips, urgent security alerts and exclusive deals delivered straight to your inbox. Plus, you’ll get instant access to my Ultimate Scam Survival Guide — free when you join my CYBERGUY.COM newsletter. 

Copyright 2025 CyberGuy.com.  All rights reserved.

Advertisement
Continue Reading

Technology

Parents call for New York governor to sign landmark AI safety bill

Published

on

Parents call for New York governor to sign landmark AI safety bill

A group of more than 150 parents sent a letter on Friday to New York governor Kathy Hochul, urging her to sign the Responsible AI Safety and Education (RAISE) Act without changes. The RAISE Act is a buzzy bill that would require developers of large AI models — like Meta, OpenAI, Deepseek, and Google — to create safety plans and follow transparency rules about reporting safety incidents.

The bill passed in both the New York State Senate and the Assembly in June. But this week, Hochul reportedly proposed a near-total rewrite of the RAISE Act that would make it more favorable to tech companies, akin to some of the changes made to California’s SB 53 after large AI companies weighed in on it.

Many AI companies, unsurprisingly, are squarely against the legislation. The AI Alliance, which counts
Meta, IBM, Intel, Oracle, Snowflake, Uber, AMD, Databricks, and Hugging Face among its members, sent a letter in June to New York lawmakers detailing their “deep concern” about the RAISE Act, calling it “unworkable.” And Leading the Future, the pro-AI super PAC backed by Perplexity AI, Andreessen Horowitz (a16z), OpenAI president Greg Brockman, and Palantir co-founder Joe Lonsdale, has been targeting New York State Assemblymember Alex Bores, who co-sponsored the RAISE Act, with recent ads.

Two organizations, ParentsTogether Action and the Tech Oversight Project, put together Friday’s letter to Hochul, which states that some of the signees had “lost children to the harms of AI chatbots and social media.” The signatories called the RAISE Act as it stands now “minimalist guardrails” that should be made law.

They also highlighted that the bill, as passed by the New York State Legislature, “does not regulate all AI developers – only the very largest companies, the ones spending hundreds of millions of dollars a year.” They would be required to disclose large-scale safety incidents to the attorney general and publish safety plans. The developers would also be prohibited from releasing a frontier model “if doing so would create an unreasonable risk of critical harm,” which is defined as the death or serious injury of 100 people or more, or $1 billion or more in damages to rights in money or property stemming from the creation of a chemical, biological, radiological, or nuclear weapon; or an AI model that “acts with no meaningful human intervention” and “would, if committed by a human,” fall under certain crimes.

Advertisement

“Big Tech’s deep-pocketed opposition to these basic protections looks familiar because we have
seen this pattern of avoidance and evasion before,” the letter states. “Widespread damage to young people —
including to their mental health, emotional stability, and ability to function in school — has been
widely documented ever since the biggest technology companies decided to push algorithmic
social media platforms without transparency, oversight, or responsibility.”

Continue Reading
Advertisement

Trending