Connect with us

Technology

Apple’s next nebulous idea: smart home robots

Published

on

Apple’s next nebulous idea: smart home robots

Humanoid robots are one of those dreams that sometimes feel like we’re on the precipice of realizing. Boston Dynamics has its Atlas robot, and Tesla is pursuing robotics, while companies like Mercedes, Amazon, and BMW are or will be testing robots for industrial use. But those are all very expensive robots performing tasks in controlled environments. In the home, they might still be far off.

Enter Apple. Mark Gurman at Bloomberg has said its robotics projects are under the purview of former Google employee John Giannandrea, who has been in charge of Siri and, for a time, the Apple Car. With the car project canceled, Vision Pro launched, and “Apple Intelligence” around the corner, is that the next big thing?

According to his information, any humanoid Apple robot is at least a decade away. Still, simpler ideas may be closer — a smaller robot that might follow you around or another idea involving a large iPad display on a robotic arm that emotes along with the caller on the other end with head nods and the like.

Many, if not most, homes are dens of robot-confounding chaos.

A mobile robot is tricky, though; what in the world would Apple do with a home robot that follows me around? Will it play music? Will it have wheels, or will it walk? Will I be expected to talk to AJAX or SiriGPT or whatever the company names its chatbot? Or, given Apple’s rumored OpenAI deal, some other chatbot?

Advertisement

Ballie in 2020 (left) vs. Ballie in 2024 (right).
Screenshots: YouTube

For that matter, what form will it take? Will it fly? Will it have wheels? Will it be a ball? Can I kick it?

Its form factor will be at least as important as its smarts. Houses have stairs, furniture that sometimes moves, clothes that end up on the floor, pets that get in the way, and kids who leave their stuff everywhere. Doors that opened or closed just fine yesterday don’t do so today because it rained. A haphazard kitchen remodel 20 years ago might mean your refrigerator door slams into the corner of the wall by the stairs because why would you put the refrigerator space anywhere else, Dave? But I digress.

Based on what little detail has trickled out, Apple’s robotics ideas seem to fit a trend of charming novelty bots we’ve seen lately.

Samsung’s “Bot Handy” robot.
Image: Samsung
Advertisement

One recent example is Samsung’s Bot Handy concept, which looks like a robot vacuum with a stalk on top and a single articulating arm, meant to carry out tasks like picking up after you or sorting your dishes. There’s also the cute ball-bot, Ballie, that Samsung has shown off at a couple of CES shows. The latest iteration follows its humans and packs a projector that can be used for movies, video calls, or entertaining the family dog.

Amazon’s Astro is an expensive way to get a beer.
Photo by Jennifer Pattison Tuohy / The Verge

Meanwhile, Amazon’s $1,600 home robot with a tablet for a face, Astro, is still available by invitation only. It is charming, in a late 90s Compaq-computer-chic aesthetic sort of way, but it’s not clear that it’s functionally more useful than a few cheap wired cameras and an Echo Dot.

LG “AI Agent” robot from CES 2024.
Image: LG

LG says its Q9 “AI Agent” is a roving smart home controller that can guess your mood and play music for you based on how it supposes you’re feeling. I’m very skeptical of all of that, but it has a handle, and I do love a piece of technology with a built-in handle.

Advertisement

I still want a sci-fi future filled with robotic home assistants that save us from the mundane tasks that keep us from the fun stuff we would rather do. But we don’t all live in the pristine, orderly abode featured in Samsung’s Ballie video or the videos Apple produces showing its hardware in personal spaces. Many normal homes are dens of robot-confounding chaos that tech companies will have a hard time accounting for when they create robots designed to follow us or autonomously carry out chores.

There are other paths to take. Take the Ring Always Home Cam, which will be very noisy judging from the demo videos, but it could also be useful and even good. While putting aside the not-insignificant privacy implications for a moment, it seems promising to me mostly because of the mobility and that it’s only designed to be a patrolling security camera.

That kind of focused functionality means it’s predictable, which is what makes single-purpose gizmos and doodads work. After some experimentation, my smart speakers are where they hear me consistently or are the most useful, and I can put my robot vacuums in the rooms I know I’ll keep clean enough that they won’t get trapped or break something (usually).

The robot vacuums I have — the Eufy Robovac L35 and a Roomba j7 — do an okay job, but they sometimes need rescuing when they find my cat’s stringy toys or eat a paperclip (which are somehow always on the floor even though I never, ever actually need one or even know where we keep them).

I have a kid, see, and preparing the way for them in other parts of the house is just adding more work to the mix. That’s fine for me because the two rooms in their charge are the ones that need vacuuming the most, so they’re still solving a problem, but it waves at the broader hurdles robotic products face.

Advertisement

And it’s not all that clear that AI can solve those problems. A New York Times opinion piece recently pointed out that despite all the hand-wringing about the tech over the last year and a half, generative AI hasn’t proven that it will be any better at making text, images, and music than the “mediocre vacuum robot that does a passable job.”

Given the generative AI boom and rumors that Apple is working on a HomePod with a screen, a cheerful, stationary smart display that obsequiously turns its screen to face me all the time seems at least vaguely within the company’s wheelhouse. Moving inside the house and interacting with objects is a trickier problem, but companies like Google and Toyota have seen success using generative AI training approaches for robots that “learn” how to do things like make breakfast or quickly sort items with little to no explicit programming.

It’ll be years, maybe even decades, before Apple or anyone else can bring us anything more than clumsy, half-useful robots that blunder through our homes, being weird, frustrating, or broken. Heck, phone companies haven’t even figured out how to make notifications anything but the bane of our collective existence. They’ve got their work cut out for them with homes like mine, where we’re just one busy week away from piles of clutter gathering like snowdrifts, ready to ruin some poor robot’s day.

Continue Reading
Advertisement
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Technology

Meta releases Threads API for developers to build “unique integrations”

Published

on

Meta releases Threads API for developers to build “unique integrations”

The Threads API is now available, meeting a promised launch by the end of June. The free API will allow developers to build “unique integrations” into Threads, and potentially even result in third-party apps for Meta’s competitor to what was previously known as Twitter.

“People can now publish posts via the API, fetch their own content, and leverage our reply management capabilities to set reply and quote controls, retrieve replies to their posts, hide, unhide or respond to specific replies,” explains Jesse Chen, director of engineering at Threads.

Chen says that insights into Threads posts are “one of our top requested features for the API,” so Meta is allowing developers to see the number of views, likes, replies, reposts, and quotes on Threads posts through the API. Meta has published plenty of documentation about how developers can get started with the Threads API, and there’s even an open-source Threads API sample app on GitHub.

Meta has been testing the Threads API with a small number of developers: Grabyo, Hootsuite, Social News Desk, Sprinklr, Sprout Social, and Techmeme. These test integrations have allowed sites like Techmeme to automate posting to Threads, or Sprout and Hootsuite customers to feed Threads posts into the social media management platform.

We’re now waiting to see if developers will be able to easily build a third-party Threads app with this new API that’s not connected to a social media management platform. The existing fediverse beta could help with that, allowing Threads users to access posts through Mastodon clients and share content to Mastodon servers. The current beta of the fediverse integration doesn’t let users view replies and follows from the fediverse though, so it’s far from being feature complete as an alternative to third-party Threads apps.

Advertisement
Continue Reading

Technology

Sims competitor Life by You has been canceled

Published

on

Sims competitor Life by You has been canceled

The game, which was first revealed in 2023, sounded impressive: it was designed to allow for the entire town to be simulated in real-time and have no loading screens. However, based on a forum post by Paradox’s deputy CEO Mattias Lilja, the game had some issues that may not have been easily fixable even with additional time for development.

“A few weeks back, we decided to hold off on an Early Access release in order to re-evaluate Life by You, as we still felt that the game was lacking in some key areas,” Lilja says. “Though a time extension was an option, once we took that pause to get a wider view of the game, it became clear to us that the road leading to a release that we felt confident about was far too long and uncertain.”

Lilja says that the game “had a number of strengths,” but the company realized that “when we come to a point where we believe that more time will not get us close enough to a version we would be satisfied with, then we believe it is better to stop.”

The Life by You team hasn’t been the only one trying to make a new take on The Sims: former XCOM developers recently launched Midsummer Studios to develop a new life sim game of their own. But EA is hard at work on more Sims as well, developing a new free-to-play Sims game codenamed Project Rene.

Continue Reading

Technology

Apple’s fancy new CarPlay will only work wirelessly

Published

on

Apple’s fancy new CarPlay will only work wirelessly

Apple’s been talking about its next generation of CarPlay for two years now with very little to show for it — the system is designed to unify the interfaces on every screen in your car, including the instrument cluster, but so far only Aston Martin and Porsche have said they’ll ship cars with the system, without any specific dates in the mix.

And the public response from the rest of the industry towards next-gen CarPlay has been pretty cool overall. I talk to car CEOs on Decoder quite often, and most of them seem fairly skeptical about allowing Apple to get between them and their customers. “We have Apple CarPlay,” Mercedes-Benz CEO Ola Källenius told me in April. “If, for some of the functions, you feel more comfortable with that and will switch back and forth, be my guest. But to give up the whole cockpit head unit — in our case, a passenger screen and everything — to somebody else? The answer is no.”

That industry skepticism seems to have hit home for Apple, which posted two WWDC 2024 videos detailing the architecture and design of next-gen CarPlay. Both made it clear that automakers will have a lot of control over how things look and work, and even have the ability to just use their own interfaces for various features using something called “punch-through UI.” The result is an approach to CarPlay that’s much less “Apple runs your car” and much more “Apple built a design toolkit for automakers to use however they want.”

See, right now CarPlay is basically just a second monitor for your phone – you connect to your car, and your phone sends a video stream to the car. This is why those cheap wireless CarPlay dongles work – they’re just wireless display adapters, basically.

But if you want to integrate things like speedometers and climate controls, CarPlay needs to actually collect data from your car, display it in realtime, and be able to control various features like HVAC directly. So for next-gen CarPlay, Apple’s split things into what it calls “layers,” some of which run on your iPhone, but others which run locally on the car so they don’t break if your phone disconnects. And phone disconnects are going to be an issue, because next-generation CarPlay only supports wireless connections. “The stability and performance of the wireless connection are essential,” Apple’s Tanya Kancheva says while talking about the next-gen architecture. Given that CarPlay connectivity issues are still the most common issue in new cars and wireless made it worse, that’s something Apple needs to keep an eye on.

Advertisement

There are two layers that run locally on the car, in slightly different ways. There’s the “overlay UI,” which has things like your turn signals and odometer in it. These can be styled, but everything about it is entirely run on your car, and otherwise untouchable. Then there is the “local UI,” which has things like your speedometer and tachometer — things related to driving that need to update all the time, basically. Automakers can customize these in several ways – there are different gauge styles and layouts, from analog to digital, and they can include logos and so on. Interestingly, there’s only one font choice: Apple’s San Francisco, which can be modified in various ways, but can’t be swapped out.

Apple’s goal for next-gen CarPlay is to have it start instantaneously — ideally when the driver opens the door — so the assets for these local UI elements are loaded onto the car from your phone during the pairing process. Carmakers can update how things look and send refreshed assets through the phone over time as well — exactly how and how often is still a bit unclear.

Then there’s what Apple calls “remote UI,” which is all stuff that runs on your phone: maps, music, trip info. This is the most like CarPlay today, except now it can run on any other screen in your car. 

The final layer is called “punch-through UI,” and it’s where Apple is ceding the most ground to automakers. Instead of coming up with its own interface ideas for things like backup cameras and advanced driver-assistance features, Apple’s allowing carmakers to simply feed their existing systems through to CarPlay. When you shift to reverse, the interface will simply show you your car’s backup camera screen, for example:

But carmakers can use punch-through UI for basically anything they want, and even deeplink CarPlay buttons to their own interfaces. Apple’s example here is a vision of multiple colliding interface ideas all at once: a button in CarPlay to control massage seats that can either show native CarPlay controls, or simply drop you into the car’s own interface.

Advertisement

A lot of carmakers are going to take the easy way out here, I think.
Apple

Or a hardware button to pick drive modes could send you to either CarPlay settings, deeplink you into the automaker’s iPhone app, or just open the native car settings:

Apple’s approach to HVAC is also what amounts to a compromise: the company isn’t really rethinking anything about how HVAC controls work. Instead, it’s allowing carmakers to customize controls from a toolkit to match the car system and even display previews of a car interior that match trim and color options. If you’ve ever looked at a car with a weird SYNC button that keeps various climate zones paired up, well, the next generation of CarPlay has a weird SYNC button too.

All of this is kept running at 60fps (or higher, if the car system supports it) by a new dedicated UI timing channel, and a lot of the underlying compositing relies on OpenGL running on the car itself.

All in all, it’s a lot of info, and what feels like a lot of Apple realizing that carmakers aren’t going to just give up their interfaces — especially since they’ve already invested in designing these sorts of custom interfaces for their native systems, many of which now run on Unreal Engine with lots of fun animations, and have Google services like Maps integrated right in. Allowing automakers to punch those interfaces through CarPlay might finally speed up adoption – and it also might create a mix-and-match interface nightmare. 

Advertisement

All that said, it’s telling that no one has seen anything but renders of next-gen CarPlay anywhere yet. We’ll have to see what it’s like if this Porsche and Aston ever arrive, and if that tips anyone else into adopting it.

Continue Reading
Advertisement

Trending